1
|
Dhaygude AD, Ameta GK, Khan IR, Singh PP, Maaliw RR, Lakshmaiya N, Shabaz M, Khan MA, Hussein HS, Alshazly H. Knowledge‐based deep learning system for classifying Alzheimer's disease for multi‐task learning. CAAI TRANSACTIONS ON INTELLIGENCE TECHNOLOGY 2024; 9:805-820. [DOI: 10.1049/cit2.12291] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/28/2023] [Accepted: 06/21/2023] [Indexed: 08/25/2024] Open
Abstract
AbstractDeep learning has recently become a viable approach for classifying Alzheimer's disease (AD) in medical imaging. However, existing models struggle to efficiently extract features from medical images and may squander additional information resources for illness classification. To address these issues, a deep three‐dimensional convolutional neural network incorporating multi‐task learning and attention mechanisms is proposed. An upgraded primary C3D network is utilised to create rougher low‐level feature maps. It introduces a new convolution block that focuses on the structural aspects of the magnetic resonance imaging image and another block that extracts attention weights unique to certain pixel positions in the feature map and multiplies them with the feature map output. Then, several fully connected layers are used to achieve multi‐task learning, generating three outputs, including the primary classification task. The other two outputs employ backpropagation during training to improve the primary classification job. Experimental findings show that the authors’ proposed method outperforms current approaches for classifying AD, achieving enhanced classification accuracy and other indicators on the Alzheimer's disease Neuroimaging Initiative dataset. The authors demonstrate promise for future disease classification studies.
Collapse
Affiliation(s)
| | - Gaurav Kumar Ameta
- Department of Computer Science & Engineering Parul Institute of Technology Parul University Vadodara Gujarat India
| | | | | | - Renato R. Maaliw
- College of Engineering Southern Luzon State University Lucban Quezon Philippines
| | - Natrayan Lakshmaiya
- Department of Mechanical Engineering Saveetha School of Engineering SIMATS Chennai Tamil Nadu India
| | - Mohammad Shabaz
- Model Institute of Engineering and Technology Jammu J&K India
| | - Muhammad Attique Khan
- Department of Computer Science HITEC University Taxila Pakistan
- Department of Computer Science and Mathematics Lebanese American University Beirut Lebanon
| | - Hany S. Hussein
- Electrical Engineering Department College of Engineering King Khalid University Abha Saudi Arabia
- Electrical Engineering Department Aswan University Aswan Egypt
| | - Hammam Alshazly
- Faculty of Computers and Information South Valley University Qena Egypt
| |
Collapse
|
2
|
Zuo Q, Wu H, Chen CLP, Lei B, Wang S. Prior-Guided Adversarial Learning With Hypergraph for Predicting Abnormal Connections in Alzheimer's Disease. IEEE TRANSACTIONS ON CYBERNETICS 2024; 54:3652-3665. [PMID: 38236677 DOI: 10.1109/tcyb.2023.3344641] [Citation(s) in RCA: 2] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 05/31/2024]
Abstract
Alzheimer's disease (AD) is characterized by alterations of the brain's structural and functional connectivity during its progressive degenerative processes. Existing auxiliary diagnostic methods have accomplished the classification task, but few of them can accurately evaluate the changing characteristics of brain connectivity. In this work, a prior-guided adversarial learning with hypergraph (PALH) model is proposed to predict abnormal brain connections using triple-modality medical images. Concretely, a prior distribution from anatomical knowledge is estimated to guide multimodal representation learning using an adversarial strategy. Also, the pairwise collaborative discriminator structure is further utilized to narrow the difference in representation distribution. Moreover, the hypergraph perceptual network is developed to effectively fuse the learned representations while establishing high-order relations within and between multimodal images. Experimental results demonstrate that the proposed model outperforms other related methods in analyzing and predicting AD progression. More importantly, the identified abnormal connections are partly consistent with previous neuroscience discoveries. The proposed model can evaluate the characteristics of abnormal brain connections at different stages of AD, which is helpful for cognitive disease study and early treatment.
Collapse
|
3
|
Yuan Z, Li X, Hao Z, Tang Z, Yao X, Wu T. Intelligent prediction of Alzheimer's disease via improved multifeature squeeze-and-excitation-dilated residual network. Sci Rep 2024; 14:11994. [PMID: 38796518 PMCID: PMC11127948 DOI: 10.1038/s41598-024-62712-w] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/20/2023] [Accepted: 05/21/2024] [Indexed: 05/28/2024] Open
Abstract
This study aimed to address the issue of larger prediction errors existing in intelligent predictive tasks related to Alzheimer's disease (AD). A cohort of 487 enrolled participants was categorized into three groups: normal control (138 individuals), mild cognitive impairment (238 patients), and AD (111 patients) in this study. An improved multifeature squeeze-and-excitation-dilated residual network (MFSE-DRN) was proposed for two important AD predictions: clinical scores and conversion probability. The model was characterized as three modules: squeeze-and-excitation-dilated residual block (SE-DRB), multifusion pooling (MF-Pool), and multimodal feature fusion. To assess its performance, the proposed model was compared with two other novel models: ranking convolutional neural network (RCNN) and 3D vision geometrical group network (3D-VGGNet). Our method showed the best performance in the two AD predicted tasks. For the clinical scores prediction, the root-mean-square errors (RMSEs) and mean absolute errors (MAEs) of mini-mental state examination (MMSE) and AD assessment scale-cognitive 11-item (ADAS-11) were 1.97, 1.46 and 4.20, 3.19 within 6 months; 2.48, 1.69 and 4.81, 3.44 within 12 months; 2.67, 1.86 and 5.81, 3.83 within 24 months; 3.02, 2.03 and 5.09, 3.43 within 36 months, respectively. At the AD conversion probability prediction, the prediction accuracies within 12, 24, and 36 months reached to 88.0, 85.5, and 88.4%, respectively. The AD predication would play a great role in clinical applications.
Collapse
Affiliation(s)
- Zengbei Yuan
- College of Medical Imaging, Jiading District Central Hospital Affiliated Shanghai University of Medicine and Health Sciences, Shanghai, 201318, China
- School of Health Science and Engineering, University of Shanghai for Science and Technology, Shanghai, 200093, China
| | - Xinlin Li
- College of Medical Imaging, Jiading District Central Hospital Affiliated Shanghai University of Medicine and Health Sciences, Shanghai, 201318, China
- School of Health Science and Engineering, University of Shanghai for Science and Technology, Shanghai, 200093, China
| | - Zezhou Hao
- College of Medical Imaging, Jiading District Central Hospital Affiliated Shanghai University of Medicine and Health Sciences, Shanghai, 201318, China
- School of Health Science and Engineering, University of Shanghai for Science and Technology, Shanghai, 200093, China
| | - Zhixian Tang
- College of Medical Imaging, Jiading District Central Hospital Affiliated Shanghai University of Medicine and Health Sciences, Shanghai, 201318, China
| | - Xufeng Yao
- College of Medical Imaging, Jiading District Central Hospital Affiliated Shanghai University of Medicine and Health Sciences, Shanghai, 201318, China.
| | - Tao Wu
- College of Medical Imaging, Jiading District Central Hospital Affiliated Shanghai University of Medicine and Health Sciences, Shanghai, 201318, China
| |
Collapse
|
4
|
Chen Z, Liu Y, Zhang Y, Zhu J, Li Q, Wu X. Shared Manifold Regularized Joint Feature Selection for Joint Classification and Regression in Alzheimer's Disease Diagnosis. IEEE TRANSACTIONS ON IMAGE PROCESSING : A PUBLICATION OF THE IEEE SIGNAL PROCESSING SOCIETY 2024; 33:2730-2745. [PMID: 38578858 DOI: 10.1109/tip.2024.3382600] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 04/07/2024]
Abstract
In Alzheimer's disease (AD) diagnosis, joint feature selection for predicting disease labels (classification) and estimating cognitive scores (regression) with neuroimaging data has received increasing attention. In this paper, we propose a model named Shared Manifold regularized Joint Feature Selection (SMJFS) that performs classification and regression in a unified framework for AD diagnosis. For classification, unlike the existing works that build least squares regression models which are insufficient in the ability of extracting discriminative information for classification, we design an objective function that integrates linear discriminant analysis and subspace sparsity regularization for acquiring an informative feature subset. Furthermore, the local data relationships are learned according to the samples' transformed distances to exploit the local data structure adaptively. For regression, in contrast to previous works that overlook the correlations among cognitive scores, we learn a latent score space to capture the correlations and employ the latent space to design a regression model with l2,1 -norm regularization, facilitating the feature selection in regression task. Moreover, the missing cognitive scores can be recovered in the latent space for increasing the number of available training samples. Meanwhile, to capture the correlations between the two tasks and describe the local relationships between samples, we construct an adaptive shared graph to guide the subspace learning in classification and the latent cognitive score learning in regression simultaneously. An efficient iterative optimization algorithm is proposed to solve the optimization problem. Extensive experiments on three datasets validate the discriminability of the features selected by SMJFS.
Collapse
|
5
|
Wang L, Wang J, Zhu L, Fu H, Li P, Cheng G, Feng Z, Li S, Heng PA. Dual Multiscale Mean Teacher Network for Semi-Supervised Infection Segmentation in Chest CT Volume for COVID-19. IEEE TRANSACTIONS ON CYBERNETICS 2023; 53:6363-6375. [PMID: 37015538 DOI: 10.1109/tcyb.2022.3223528] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/19/2023]
Abstract
Automated detecting lung infections from computed tomography (CT) data plays an important role for combating coronavirus 2019 (COVID-19). However, there are still some challenges for developing AI system: 1) most current COVID-19 infection segmentation methods mainly relied on 2-D CT images, which lack 3-D sequential constraint; 2) existing 3-D CT segmentation methods focus on single-scale representations, which do not achieve the multiple level receptive field sizes on 3-D volume; and 3) the emergent breaking out of COVID-19 makes it hard to annotate sufficient CT volumes for training deep model. To address these issues, we first build a multiple dimensional-attention convolutional neural network (MDA-CNN) to aggregate multiscale information along different dimension of input feature maps and impose supervision on multiple predictions from different convolutional neural networks (CNNs) layers. Second, we assign this MDA-CNN as a basic network into a novel dual multiscale mean teacher network (DM [Formula: see text]-Net) for semi-supervised COVID-19 lung infection segmentation on CT volumes by leveraging unlabeled data and exploring the multiscale information. Our DM [Formula: see text]-Net encourages multiple predictions at different CNN layers from the student and teacher networks to be consistent for computing a multiscale consistency loss on unlabeled data, which is then added to the supervised loss on the labeled data from multiple predictions of MDA-CNN. Third, we collect two COVID-19 segmentation datasets to evaluate our method. The experimental results show that our network consistently outperforms the compared state-of-the-art methods.
Collapse
|
6
|
Agarwal D, Berbís MÁ, Luna A, Lipari V, Ballester JB, de la Torre-Díez I. Automated Medical Diagnosis of Alzheimer´s Disease Using an Efficient Net Convolutional Neural Network. J Med Syst 2023; 47:57. [PMID: 37129723 PMCID: PMC10154284 DOI: 10.1007/s10916-023-01941-4] [Citation(s) in RCA: 2] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/09/2023] [Accepted: 03/20/2023] [Indexed: 05/03/2023]
Abstract
Alzheimer's disease (AD) poses an enormous challenge to modern healthcare. Since 2017, researchers have been using deep learning (DL) models for the early detection of AD using neuroimaging biomarkers. In this paper, we implement the EfficietNet-b0 convolutional neural network (CNN) with a novel approach-"fusion of end-to-end and transfer learning"-to classify different stages of AD. 245 T1W MRI scans of cognitively normal (CN) subjects, 229 scans of AD subjects, and 229 scans of subjects with stable mild cognitive impairment (sMCI) were employed. Each scan was preprocessed using a standard pipeline. The proposed models were trained and evaluated using preprocessed scans. For the sMCI vs. AD classification task we obtained 95.29% accuracy and 95.35% area under the curve (AUC) for model training and 93.10% accuracy and 93.00% AUC for model testing. For the multiclass AD vs. CN vs. sMCI classification task we obtained 85.66% accuracy and 86% AUC for model training and 87.38% accuracy and 88.00% AUC for model testing. Based on our experimental results, we conclude that CNN-based DL models can be used to analyze complicated MRI scan features in clinical settings.
Collapse
Affiliation(s)
- Deevyankar Agarwal
- Department of Signal Theory and Communications and Telematics Engineering, University of Valladolid, Paseo de Belén 15, 47011, Valladolid, Spain.
| | | | - Antonio Luna
- MRI Unit, Radiology Department, HT Médica, Carmelo Torres No. 2, 23007, Jaén, Spain
| | - Vivian Lipari
- European Atlantic University, Isabel Torres 21, 39011, Santander, Spain
| | | | - Isabel de la Torre-Díez
- Department of Signal Theory and Communications and Telematics Engineering, University of Valladolid, Paseo de Belén 15, 47011, Valladolid, Spain
| |
Collapse
|
7
|
Chadebec C, Thibeau-Sutre E, Burgos N, Allassonniere S. Data Augmentation in High Dimensional Low Sample Size Setting Using a Geometry-Based Variational Autoencoder. IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE 2023; 45:2879-2896. [PMID: 35749321 DOI: 10.1109/tpami.2022.3185773] [Citation(s) in RCA: 3] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/15/2023]
Abstract
In this paper, we propose a new method to perform data augmentation in a reliable way in the High Dimensional Low Sample Size (HDLSS) setting using a geometry-based variational autoencoder (VAE). Our approach combines the proposal of 1) a new VAE model, the latent space of which is modeled as a Riemannian manifold and which combines both Riemannian metric learning and normalizing flows and 2) a new generation scheme which produces more meaningful samples especially in the context of small data sets. The method is tested through a wide experimental study where its robustness to data sets, classifiers and training samples size is stressed. It is also validated on a medical imaging classification task on the challenging ADNI database where a small number of 3D brain magnetic resonance images (MRIs) are considered and augmented using the proposed VAE framework. In each case, the proposed method allows for a significant and reliable gain in the classification metrics. For instance, balanced accuracy jumps from 66.3% to 74.3% for a state-of-the-art convolutional neural network classifier trained with 50 MRIs of cognitively normal (CN) and 50 Alzheimer disease (AD) patients and from 77.7% to 86.3% when trained with 243 CN and 210 AD while improving greatly sensitivity and specificity metrics.
Collapse
|
8
|
Semisupervised Bacterial Heuristic Feature Selection Algorithm for High-Dimensional Classification with Missing Labels. INT J INTELL SYST 2023. [DOI: 10.1155/2023/4196920] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 02/24/2023]
Abstract
Feature selection is a crucial method for discovering relevant features in high-dimensional data. However, most studies primarily focus on completely labeled data, ignoring the frequent occurrence of missing labels in real-world problems. To address high-dimensional and label-missing problems in data classification simultaneously, we proposed a semisupervised bacterial heuristic feature selection algorithm. To track the label-missing problem, a k-nearest neighbor semisupervised learning strategy is designed to reconstruct missing labels. In addition, the bacterial heuristic algorithm is improved using hierarchical population initialization, dynamic learning, and elite population evolution strategies to enhance the search capacity for various feature combinations. To verify the effectiveness of the proposed algorithm, three groups of comparison experiments based on eight datasets are employed, including two traditional feature selection methods, four bacterial heuristic feature selection algorithms, and two swarm-based heuristic feature selection algorithms. Experimental results demonstrate that the proposed algorithm has obvious advantages in terms of classification accuracy and selected feature numbers.
Collapse
|
9
|
Li Z, Cong Y, Chen X, Qi J, Sun J, Yan T, Yang H, Liu J, Lu E, Wang L, Li J, Hu H, Zhang C, Yang Q, Yao J, Yao P, Jiang Q, Liu W, Song J, Carin L, Chen Y, Zhao S, Gao X. Vision transformer-based weakly supervised histopathological image analysis of primary brain tumors. iScience 2022; 26:105872. [PMID: 36647383 PMCID: PMC9839963 DOI: 10.1016/j.isci.2022.105872] [Citation(s) in RCA: 13] [Impact Index Per Article: 6.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/02/2022] [Revised: 12/03/2022] [Accepted: 12/21/2022] [Indexed: 12/25/2022] Open
Abstract
Diagnosis of primary brain tumors relies heavily on histopathology. Although various computational pathology methods have been developed for automated diagnosis of primary brain tumors, they usually require neuropathologists' annotation of region of interests or selection of image patches on whole-slide images (WSI). We developed an end-to-end Vision Transformer (ViT) - based deep learning architecture for brain tumor WSI analysis, yielding a highly interpretable deep-learning model, ViT-WSI. Based on the principle of weakly supervised machine learning, ViT-WSI accomplishes the task of major primary brain tumor type and subtype classification. Using a systematic gradient-based attribution analysis procedure, ViT-WSI can discover diagnostic histopathological features for primary brain tumors. Furthermore, we demonstrated that ViT-WSI has high predictive power of inferring the status of three diagnostic glioma molecular markers, IDH1 mutation, p53 mutation, and MGMT methylation, directly from H&E-stained histopathological images, with patient level AUC scores of 0.960, 0.874, and 0.845, respectively.
Collapse
Affiliation(s)
- Zhongxiao Li
- Computer Science Program, Computer, Electrical and Mathematical Sciences and Engineering (CEMSE) Division, King Abdullah University of Science and Technology (KAUST), Thuwal 23955-6900, Kingdom of Saudi Arabia,KAUST Computational Bioscience Research Center (CBRC), King Abdullah University of Science and Technology (KAUST), Thuwal 23955-6900, Kingdom of Saudi Arabia
| | - Yuwei Cong
- Department of Pathology, The First Affiliated Hospital of Harbin Medical University, 23 Youzheng Street, Nangang District, Harbin 150001, People’s Republic of China
| | - Xin Chen
- Department of Neurosurgery, The First Affiliated Hospital of Harbin Medical University, Harbin, Heilongjiang Province 150001, China
| | - Jiping Qi
- Department of Pathology, The First Affiliated Hospital of Harbin Medical University, 23 Youzheng Street, Nangang District, Harbin 150001, People’s Republic of China,Corresponding author
| | - Jingxian Sun
- Department of Neurosurgery, The First Affiliated Hospital of Harbin Medical University, Harbin, Heilongjiang Province 150001, China
| | - Tao Yan
- Department of Neurosurgery, The First Affiliated Hospital of Harbin Medical University, Harbin, Heilongjiang Province 150001, China
| | - He Yang
- Department of Neurosurgery, The First Affiliated Hospital of Harbin Medical University, Harbin, Heilongjiang Province 150001, China
| | - Junsi Liu
- Department of Neurosurgery, The First Affiliated Hospital of Harbin Medical University, Harbin, Heilongjiang Province 150001, China
| | - Enzhou Lu
- Department of Neurosurgery, The First Affiliated Hospital of Harbin Medical University, Harbin, Heilongjiang Province 150001, China
| | - Lixiang Wang
- Department of Neurosurgery, The First Affiliated Hospital of Harbin Medical University, Harbin, Heilongjiang Province 150001, China
| | - Jiafeng Li
- Department of Neurosurgery, The First Affiliated Hospital of Harbin Medical University, Harbin, Heilongjiang Province 150001, China
| | - Hong Hu
- Department of Neurosurgery, The First Affiliated Hospital of Harbin Medical University, Harbin, Heilongjiang Province 150001, China
| | | | - Quan Yang
- Department of Neurosurgery, The First Affiliated Hospital of Harbin Medical University, Harbin, Heilongjiang Province 150001, China
| | - Jiawei Yao
- Department of Neurosurgery, The First Affiliated Hospital of Harbin Medical University, Harbin, Heilongjiang Province 150001, China
| | - Penglei Yao
- Department of Neurosurgery, The First Affiliated Hospital of Harbin Medical University, Harbin, Heilongjiang Province 150001, China
| | - Qiuyi Jiang
- Department of Neurosurgery, The First Affiliated Hospital of Harbin Medical University, Harbin, Heilongjiang Province 150001, China
| | - Wenwu Liu
- Department of Neurosurgery, The First Affiliated Hospital of Harbin Medical University, Harbin, Heilongjiang Province 150001, China
| | - Jiangning Song
- Biomedicine Discovery Institute and Department of Biochemistry and Molecular Biology, Monash University, Melbourne, VIC 3800, Australia,Monash Data Futures Institute, Monash University, Melbourne, VIC 3800, Australia
| | - Lawrence Carin
- Computer Science Program, Computer, Electrical and Mathematical Sciences and Engineering (CEMSE) Division, King Abdullah University of Science and Technology (KAUST), Thuwal 23955-6900, Kingdom of Saudi Arabia,Corresponding author
| | - Yupeng Chen
- School of Microelectronics, Southern University of Science and Technology, Shenzhen 518055, PR China,Corresponding author
| | - Shiguang Zhao
- Department of Neurosurgery, The First Affiliated Hospital of Harbin Medical University, Harbin, Heilongjiang Province 150001, China,Department of Neurosurgery, Shenzhen University General Hospital, Shenzhen, Guangdong Province 518100, China,Corresponding author
| | - Xin Gao
- Computer Science Program, Computer, Electrical and Mathematical Sciences and Engineering (CEMSE) Division, King Abdullah University of Science and Technology (KAUST), Thuwal 23955-6900, Kingdom of Saudi Arabia,KAUST Computational Bioscience Research Center (CBRC), King Abdullah University of Science and Technology (KAUST), Thuwal 23955-6900, Kingdom of Saudi Arabia,Corresponding author
| |
Collapse
|
10
|
Lv J, Wang Z, Shi H, Zhang H, Wang S, Wang Y, Li Q. Joint Progressive and Coarse-to-Fine Registration of Brain MRI via Deformation Field Integration and Non-Rigid Feature Fusion. IEEE TRANSACTIONS ON MEDICAL IMAGING 2022; 41:2788-2802. [PMID: 35482699 DOI: 10.1109/tmi.2022.3170879] [Citation(s) in RCA: 6] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/14/2023]
Abstract
Registration of brain MRI images requires to solve a deformation field, which is extremely difficult in aligning intricate brain tissues, e.g., subcortical nuclei, etc. Existing efforts resort to decomposing the target deformation field into intermediate sub-fields with either tiny motions, i.e., progressive registration stage by stage, or lower resolutions, i.e., coarse-to-fine estimation of the full-size deformation field. In this paper, we argue that those efforts are not mutually exclusive, and propose a unified framework for robust brain MRI registration in both progressive and coarse-to-fine manners simultaneously. Specifically, building on a dual-encoder U-Net, the fixed-moving MRI pair is encoded and decoded into multi-scale sub-fields from coarse to fine. Each decoding block contains two proposed novel modules: i) in Deformation Field Integration (DFI), a single integrated deformation sub-field is calculated, warping by which is equivalent to warping progressively by sub-fields from all previous decoding blocks, and ii) in Non-rigid Feature Fusion (NFF), features of the fixed-moving pair are aligned by DFI-integrated deformation field, and then fused to predict a finer sub-field. Leveraging both DFI and NFF, the target deformation field is factorized into multi-scale sub-fields, where the coarser fields alleviate the estimate of a finer one and the finer field learns to make up those misalignments insolvable by previous coarser ones. The extensive and comprehensive experimental results on both private and two public datasets demonstrate a superior registration performance of brain MRI images over progressive registration only and coarse-to-fine estimation only, with an increase by at most 8% in the average Dice.
Collapse
|
11
|
Mukherji D, Mukherji M, Mukherji N. Early detection of Alzheimer's disease using neuropsychological tests: a predict-diagnose approach using neural networks. Brain Inform 2022; 9:23. [PMID: 36166157 PMCID: PMC9515292 DOI: 10.1186/s40708-022-00169-1] [Citation(s) in RCA: 5] [Impact Index Per Article: 2.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/22/2021] [Accepted: 08/23/2022] [Indexed: 11/10/2022] Open
Abstract
Alzheimer’s disease (AD) is a slowly progressing disease for which there is no known therapeutic cure at present. Ongoing research around the world is actively engaged in the quest for identifying markers that can help predict the future cognitive state of individuals so that measures can be taken to prevent the onset or arrest the progression of the disease. Researchers are interested in both biological and neuropsychological markers that can serve as good predictors of the future cognitive state of individuals. The goal of this study is to identify non-invasive, inexpensive markers and develop neural network models that learn the relationship between those markers and the future cognitive state. To that end, we use the renowned Alzheimer’s Disease Neuroimaging Initiative (ADNI) data for a handful of neuropsychological tests to train Recurrent Neural Network (RNN) models to predict future neuropsychological test results and Multi-Level Perceptron (MLP) models to diagnose the future cognitive states of trial participants based on those predicted results. The results demonstrate that the predicted cognitive states match the actual cognitive states of ADNI test subjects with a high level of accuracy. Therefore, this novel two-step technique can serve as an effective tool for the prediction of Alzheimer’s disease progression. The reliance of the results on inexpensive, non-invasive tests implies that this technique can be used in countries around the world including those with limited financial resources.
Collapse
|
12
|
Zhang D, Han J, Cheng G, Yang MH. Weakly Supervised Object Localization and Detection: A Survey. IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE 2022; 44:5866-5885. [PMID: 33877967 DOI: 10.1109/tpami.2021.3074313] [Citation(s) in RCA: 31] [Impact Index Per Article: 15.5] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/12/2023]
Abstract
As an emerging and challenging problem in the computer vision community, weakly supervised object localization and detection plays an important role for developing new generation computer vision systems and has received significant attention in the past decade. As methods have been proposed, a comprehensive survey of these topics is of great importance. In this work, we review (1) classic models, (2) approaches with feature representations from off-the-shelf deep networks, (3) approaches solely based on deep learning, and (4) publicly available datasets and standard evaluation metrics that are widely used in this field. We also discuss the key challenges in this field, development history of this field, advantages/disadvantages of the methods in each category, the relationships between methods in different categories, applications of the weakly supervised object localization and detection methods, and potential future directions to further promote the development of this research field.
Collapse
|
13
|
End-to-End Deep Learning Architectures Using 3D Neuroimaging Biomarkers for Early Alzheimer’s Diagnosis. MATHEMATICS 2022. [DOI: 10.3390/math10152575] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 12/07/2022]
Abstract
This study uses magnetic resonance imaging (MRI) data to propose end-to-end learning implementing volumetric convolutional neural network (CNN) models for two binary classification tasks: Alzheimer’s disease (AD) vs. cognitively normal (CN) and stable mild cognitive impairment (sMCI) vs. AD. The baseline MP-RAGE T1 MR images of 245 AD patients and 229 with sMCI were obtained from the ADNI dataset, whereas 245 T1 MR images of CN people were obtained from the IXI dataset. All of the images were preprocessed in four steps: N4 bias field correction, denoising, brain extraction, and registration. End-to-end-learning-based deep CNNs were used to discern between different phases of AD. Eight CNN-based architectures were implemented and assessed. The DenseNet264 excelled in both types of classification, with 82.5% accuracy and 87.63% AUC for training and 81.03% accuracy for testing relating to the sMCI vs. AD and 100% accuracy and 100% AUC for training and 99.56% accuracy for testing relating to the AD vs. CN. Deep learning approaches based on CNN and end-to-end learning offer a strong tool for examining minute but complex properties in MR images which could aid in the early detection and prediction of Alzheimer’s disease in clinical settings.
Collapse
|
14
|
Subbaraya SH, Parthasarathy SS. Probabilistic principal component analysis and long short-term memory classifier for automatic detection of Alzheimer’s disease using MRI brain images. INTERNATIONAL JOURNAL OF KNOWLEDGE-BASED AND INTELLIGENT ENGINEERING SYSTEMS 2022. [DOI: 10.3233/kes-220007] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/15/2022]
Abstract
The automatic recognition and classification of Alzheimer disease utilizing magnetic resonance imaging is a hard task, due to the complexity and variability of the size, location, texture and shape of the lesions. The objective of this study is to propose a proper feature dimensional reduction and classification approach to improve the performance of Alzheimer disease recognition and classification. At first, the input brain images were acquired from Open Access Series of Imaging Studies (OASIS) and National Institute of Mental Health and Neuro Sciences (NIMHANS) databases. Then, the image pre-processing and feature extraction were attained by applying Contrast Limited Adaptive Histogram Equalization (CLAHE) and Discrete Wavelet Transform (DWT) approach to denoise and extract the feature vectors from the images. In addition, Probabilistic Principal Component Analysis (PPCA) was used to diminish the extracted features dimension that effectively lessen the “curse of dimensionality” concern. At last, Long Short-Term Memory (LSTM) classifier was employed to classify the brain images as Alzheimer disease, normal, and Mild Cognitive Impairment (MCI). From the simulation outcome, the proposed system attained better performance compared to the existing systems and showed 3–11% improvement in recognition accuracy.
Collapse
|
15
|
Vu MH, Norman G, Nyholm T, Lofstedt T. A Data-Adaptive Loss Function for Incomplete Data and Incremental Learning in Semantic Image Segmentation. IEEE TRANSACTIONS ON MEDICAL IMAGING 2022; 41:1320-1330. [PMID: 34965206 DOI: 10.1109/tmi.2021.3139161] [Citation(s) in RCA: 2] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/14/2023]
Abstract
In the last years, deep learning has dramatically improved the performances in a variety of medical image analysis applications. Among different types of deep learning models, convolutional neural networks have been among the most successful and they have been used in many applications in medical imaging. Training deep convolutional neural networks often requires large amounts of image data to generalize well to new unseen images. It is often time-consuming and expensive to collect large amounts of data in the medical image domain due to expensive imaging systems, and the need for experts to manually make ground truth annotations. A potential problem arises if new structures are added when a decision support system is already deployed and in use. Since the field of radiation therapy is constantly developing, the new structures would also have to be covered by the decision support system. In the present work, we propose a novel loss function to solve multiple problems: imbalanced datasets, partially-labeled data, and incremental learning. The proposed loss function adapts to the available data in order to utilize all available data, even when some have missing annotations. We demonstrate that the proposed loss function also works well in an incremental learning setting, where an existing model is easily adapted to semi-automatically incorporate delineations of new organs when they appear. Experiments on a large in-house dataset show that the proposed method performs on par with baseline models, while greatly reducing the training time and eliminating the hassle of maintaining multiple models in practice.
Collapse
|
16
|
Wang T, Bezerianos A, Cichocki A, Li J. Multikernel Capsule Network for Schizophrenia Identification. IEEE TRANSACTIONS ON CYBERNETICS 2022; 52:4741-4750. [PMID: 33259321 DOI: 10.1109/tcyb.2020.3035282] [Citation(s) in RCA: 19] [Impact Index Per Article: 9.5] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/12/2023]
Abstract
Schizophrenia seriously affects the quality of life. To date, both simple (e.g., linear discriminant analysis) and complex (e.g., deep neural network) machine-learning methods have been utilized to identify schizophrenia based on functional connectivity features. The existing simple methods need two separate steps (i.e., feature extraction and classification) to achieve the identification, which disables simultaneous tuning for the best feature extraction and classifier training. The complex methods integrate two steps and can be simultaneously tuned to achieve optimal performance, but these methods require a much larger amount of data for model training. To overcome the aforementioned drawbacks, we proposed a multikernel capsule network (MKCapsnet), which was developed by considering the brain anatomical structure. Kernels were set to match partition sizes of the brain anatomical structure in order to capture interregional connectivities at the varying scales. With the inspiration of the widely used dropout strategy in deep learning, we developed capsule dropout in the capsule layer to prevent overfitting of the model. The comparison results showed that the proposed method outperformed the state-of-the-art methods. Besides, we compared performances using different parameters and illustrated the routing process to reveal characteristics of the proposed method. MKCapsnet is promising for schizophrenia identification. Our study first utilized the capsule neural network for analyzing functional connectivity of magnetic resonance imaging (MRI) and proposed a novel multikernel capsule structure with the consideration of brain anatomical parcellation, which could be a new way to reveal brain mechanisms. In addition, we provided useful information in the parameter setting, which is informative for further studies using a capsule network for other neurophysiological signal classification.
Collapse
|
17
|
Chen L, Qiao H, Zhu F. Alzheimer's Disease Diagnosis With Brain Structural MRI Using Multiview-Slice Attention and 3D Convolution Neural Network. Front Aging Neurosci 2022; 14:871706. [PMID: 35557839 PMCID: PMC9088013 DOI: 10.3389/fnagi.2022.871706] [Citation(s) in RCA: 5] [Impact Index Per Article: 2.5] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/08/2022] [Accepted: 03/17/2022] [Indexed: 01/01/2023] Open
Abstract
Numerous artificial intelligence (AI) based approaches have been proposed for automatic Alzheimer's disease (AD) prediction with brain structural magnetic resonance imaging (sMRI). Previous studies extract features from the whole brain or individual slices separately, ignoring the properties of multi-view slices and feature complementarity. For this reason, we present a novel AD diagnosis model based on the multiview-slice attention and 3D convolution neural network (3D-CNN). Specifically, we begin by extracting the local slice-level characteristic in various dimensions using multiple sub-networks. Then we proposed a slice-level attention mechanism to emphasize specific 2D-slices to exclude the redundancy features. After that, a 3D-CNN was employed to capture the global subject-level structural changes. Finally, all these 2D and 3D features were fused to obtain more discriminative representations. We conduct the experiments on 1,451 subjects from ADNI-1 and ADNI-2 datasets. Experimental results showed the superiority of our model over the state-of-the-art approaches regarding dementia classification. Specifically, our model achieves accuracy values of 91.1 and 80.1% on ADNI-1 for AD diagnosis and mild cognitive impairment (MCI) convention prediction, respectively.
Collapse
Affiliation(s)
- Lin Chen
- Chongqing Key Laboratory of Big Data and Intelligent Computing, Chongqing Institute of Green and Intelligent Technology, Chinese Academy of Sciences, Chongqing, China
| | - Hezhe Qiao
- Chongqing Key Laboratory of Big Data and Intelligent Computing, Chongqing Institute of Green and Intelligent Technology, Chinese Academy of Sciences, Chongqing, China
- University of Chinese Academy of Sciences, Beijing, China
| | - Fan Zhu
- Chongqing Key Laboratory of Big Data and Intelligent Computing, Chongqing Institute of Green and Intelligent Technology, Chinese Academy of Sciences, Chongqing, China
| |
Collapse
|
18
|
Han K, He M, Yang F, Zhang Y. Multi-task multi-level feature adversarial network for joint Alzheimer’s disease diagnosis and atrophy localization using sMRI. Phys Med Biol 2022; 67. [DOI: 10.1088/1361-6560/ac5ed5] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/07/2021] [Accepted: 03/17/2022] [Indexed: 11/11/2022]
Abstract
Abstract
Capitalizing on structural magnetic resonance imaging (sMRI), existing deep learning methods (especially convolutional neural networks, CNNs) have been widely and successfully applied to computer-aided diagnosis of Alzheimer’s disease (AD) and its prodromal stage (i.e. mild cognitive impairment, MCI). But considering the generalization capability of the obtained model trained on limited number of samples, we construct a multi-task multi-level feature adversarial network (M2FAN) for joint diagnosis and atrophy localization using baseline sMRI. Specifically, the linear-aligned T1 MR images were first processed by a lightweight CNN backbone to capture the shared intermediate feature representations, which were then branched into a global subnet for preliminary dementia diagnosis and a multi instance learning network for brain atrophy localization in multi-task learning manner. As the global discriminative information captured by the global subnet might be unstable for disease diagnosis, we further designed a module of multi-level feature adversarial learning that accounts for regularization to make global features robust against the adversarial perturbation synthesized by the local/instance features to improve the diagnostic performance. Our proposed method was evaluated on three public datasets (i.e. ADNI-1, ADNI-2, and AIBL), demonstrating competitive performance compared with several state-of-the-art methods in both tasks of AD diagnosis and MCI conversion prediction.
Collapse
|
19
|
Lian C, Liu M, Pan Y, Shen D. Attention-Guided Hybrid Network for Dementia Diagnosis With Structural MR Images. IEEE TRANSACTIONS ON CYBERNETICS 2022; 52:1992-2003. [PMID: 32721906 PMCID: PMC7855081 DOI: 10.1109/tcyb.2020.3005859] [Citation(s) in RCA: 24] [Impact Index Per Article: 12.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 05/22/2023]
Abstract
Deep-learning methods (especially convolutional neural networks) using structural magnetic resonance imaging (sMRI) data have been successfully applied to computer-aided diagnosis (CAD) of Alzheimer's disease (AD) and its prodromal stage [i.e., mild cognitive impairment (MCI)]. As it is practically challenging to capture local and subtle disease-associated abnormalities directly from the whole-brain sMRI, most of those deep-learning approaches empirically preselect disease-associated sMRI brain regions for model construction. Considering that such isolated selection of potentially informative brain locations might be suboptimal, very few methods have been proposed to perform disease-associated discriminative region localization and disease diagnosis in a unified deep-learning framework. However, those methods based on task-oriented discriminative localization still suffer from two common limitations, that is: 1) identified brain locations are strictly consistent across all subjects, which ignores the unique anatomical characteristics of each brain and 2) only limited local regions/patches are used for model training, which does not fully utilize the global structural information provided by the whole-brain sMRI. In this article, we propose an attention-guided deep-learning framework to extract multilevel discriminative sMRI features for dementia diagnosis. Specifically, we first design a backbone fully convolutional network to automatically localize the discriminative brain regions in a weakly supervised manner. Using the identified disease-related regions as spatial attention guidance, we further develop a hybrid network to jointly learn and fuse multilevel sMRI features for CAD model construction. Our proposed method was evaluated on three public datasets (i.e., ADNI-1, ADNI-2, and AIBL), showing superior performance compared with several state-of-the-art methods in both tasks of AD diagnosis and MCI conversion prediction.
Collapse
|
20
|
Hojjati SH, Babajani-Feremi A. Prediction and Modeling of Neuropsychological Scores in Alzheimer's Disease Using Multimodal Neuroimaging Data and Artificial Neural Networks. Front Comput Neurosci 2022; 15:769982. [PMID: 35069161 PMCID: PMC8770936 DOI: 10.3389/fncom.2021.769982] [Citation(s) in RCA: 7] [Impact Index Per Article: 3.5] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/03/2021] [Accepted: 11/16/2021] [Indexed: 11/13/2022] Open
Abstract
Background: In recent years, predicting and modeling the progression of Alzheimer's disease (AD) based on neuropsychological tests has become increasingly appealing in AD research. Objective: In this study, we aimed to predict the neuropsychological scores and investigate the non-linear progression trend of the cognitive declines based on multimodal neuroimaging data. Methods: We utilized unimodal/bimodal neuroimaging measures and a non-linear regression method (based on artificial neural networks) to predict the neuropsychological scores in a large number of subjects (n = 1143), including healthy controls (HC) and patients with mild cognitive impairment non-converter (MCI-NC), mild cognitive impairment converter (MCI-C), and AD. We predicted two neuropsychological scores, i.e., the clinical dementia rating sum of boxes (CDRSB) and Alzheimer's disease assessment scale cognitive 13 (ADAS13), based on structural magnetic resonance imaging (sMRI) and positron emission tomography (PET) biomarkers. Results: Our results revealed that volumes of the entorhinal cortex and hippocampus and the average fluorodeoxyglucose (FDG)-PET of the angular gyrus, temporal gyrus, and posterior cingulate outperform other neuroimaging features in predicting ADAS13 and CDRSB scores. Compared to a unimodal approach, our results showed that a bimodal approach of integrating the top two neuroimaging features (i.e., the entorhinal volume and the average FDG of the angular gyrus, temporal gyrus, and posterior cingulate) increased the prediction performance of ADAS13 and CDRSB scores in the converting and stable stages of MCI and AD. Finally, a non-linear AD progression trend was modeled to describe the cognitive decline based on neuroimaging biomarkers in different stages of AD. Conclusion: Findings in this study show an association between neuropsychological scores and sMRI and FDG-PET biomarkers from normal aging to severe AD.
Collapse
Affiliation(s)
- Seyed Hani Hojjati
- Quantitative Neuroimaging Laboratory, Brain Health Imaging Institute, Department of Radiology, Weill Cornell Medicine, New York, NY, United States
| | - Abbas Babajani-Feremi
- Department of Neurology, Dell Medical School, The University of Texas at Austin, Austin, TX, United States
- Department of Neurosurgery, Dell Medical School, The University of Texas at Austin, Austin, TX, United States
- Magnetoencephalography Laboratory, Dell Children’s Medical Center, Austin, TX, United States
| | | |
Collapse
|
21
|
Swain S, Bhushan B, Dhiman G, Viriyasitavat W. Appositeness of Optimized and Reliable Machine Learning for Healthcare: A Survey. ARCHIVES OF COMPUTATIONAL METHODS IN ENGINEERING : STATE OF THE ART REVIEWS 2022; 29:3981-4003. [PMID: 35342282 PMCID: PMC8939887 DOI: 10.1007/s11831-022-09733-8] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Received: 08/05/2021] [Accepted: 02/09/2022] [Indexed: 05/04/2023]
Abstract
Machine Learning (ML) has been categorized as a branch of Artificial Intelligence (AI) under the Computer Science domain wherein programmable machines imitate human learning behavior with the help of statistical methods and data. The Healthcare industry is one of the largest and busiest sectors in the world, functioning with an extensive amount of manual moderation at every stage. Most of the clinical documents concerning patient care are hand-written by experts, selective reports are machine-generated. This process elevates the chances of misdiagnosis thereby, imposing a risk to a patient's life. Recent technological adoptions for automating manual operations have witnessed extensive use of ML in its applications. The paper surveys the applicability of ML approaches in automating medical systems. The paper discusses most of the optimized statistical ML frameworks that encourage better service delivery in clinical aspects. The universal adoption of various Deep Learning (DL) and ML techniques as the underlying systems for a variety of wellness applications, is delineated by challenges and elevated by myriads of security. This work tries to recognize a variety of vulnerabilities occurring in medical procurement, admitting the concerns over its predictive performance from a privacy point of view. Finally providing possible risk delimiting facts and directions for active challenges in the future.
Collapse
Affiliation(s)
- Subhasmita Swain
- Department of Computer Science and Engineering, School of Engineering and Technology, Sharda University, Greater Noida, India
| | - Bharat Bhushan
- Department of Computer Science and Engineering, School of Engineering and Technology, Sharda University, Greater Noida, India
| | - Gaurav Dhiman
- Department of Computer Science, Government Bikram College of Commerce, Patiala, India
- University Centre for Research and Development, Department of Computer Science and Engineering, Chandigarh University, Gharuan, Mohali, India
- Department of Computer Science and Engineering, Graphic Era Deemed to be University, Dehradun, India
| | - Wattana Viriyasitavat
- Department of Statistics, Faculty of Commerce and Accountancy, Chulalongkorn Business School, Bangkok, Thailand
| |
Collapse
|
22
|
Cao C, Liu Z, Liu G, Jin S, Xia S. Ability of weakly supervised learning to detect acute ischemic stroke and hemorrhagic infarction lesions with diffusion-weighted imaging. Quant Imaging Med Surg 2022; 12:321-332. [PMID: 34993081 DOI: 10.21037/qims-21-324] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/23/2021] [Accepted: 06/27/2021] [Indexed: 12/31/2022]
Abstract
BACKGROUND Gradient-recalled echo (GRE) sequence is time-consuming and not routinely performed. Herein, we aimed to investigate the ability of weakly supervised learning to identify acute ischemic stroke (AIS) and concurrent hemorrhagic infarction based on diffusion-weighted imaging (DWI). METHODS First, we proposed spatially locating small stroke lesions in different positions and hemorrhagic infarction lesions by residual neural and visual geometry group networks using weakly supervised learning. Next, we compared the sensitivity and specificity for identifying automatically concurrent hemorrhagic infarction in stroke patients with the sensitivity and specificity of human readings of diffusion and b0 images to evaluate the performance of the weakly supervised methods. Also, the labeling time of the weakly supervised approach was compared with that of the fully supervised approach. RESULTS Data from a total of 1,027 patients were analyzed. The residual neural network displayed a higher sensitivity than did the visual geometry group network in spatially locating the small stroke and hemorrhagic infarction lesions. The residual neural network had significantly greater patient-level sensitivity than did the human readers (98.4% versus 86.2%, P=0.008) in identifying concurrent hemorrhagic infarction with GRE as the reference standard; however, their specificities were comparable (95.4% versus 96.9%, P>0.99). Weak labeling of lesions required significantly less time than did full labeling of lesions (2.667 versus 10.115 minutes, P<0.001). CONCLUSIONS Weakly supervised learning was able to spatially locate small stroke lesions in different positions and showed more sensitivity than did human reading in identifying concurrent hemorrhagic infarction based on DWI. The proposed approach can reduce the labeling workload.
Collapse
Affiliation(s)
- Chen Cao
- Department of Radiology, First Central Clinical College, Tianjin Medical University, Tianjin, China.,Department of Radiology, Tianjin Huanhu Hospital, Tianjin, China
| | - Zhiyang Liu
- Tianjin Key Laboratory of Optoelectronic Sensor and Sensing Network Technology, College of Electronic Information and Optical Engineering, Nankai University, Tianjin, China
| | - Guohua Liu
- Tianjin Key Laboratory of Optoelectronic Sensor and Sensing Network Technology, College of Electronic Information and Optical Engineering, Nankai University, Tianjin, China
| | - Song Jin
- Department of Radiology, Tianjin Huanhu Hospital, Tianjin, China
| | - Shuang Xia
- Department of Radiology, Tianjin First Central Hospital, School of Medicine, Nankai University, Tianjin, China
| |
Collapse
|
23
|
Zhang Q, Ren X, Wei B. Segmentation of infected region in CT images of COVID-19 patients based on QC-HC U-net. Sci Rep 2021; 11:22854. [PMID: 34819524 PMCID: PMC8613253 DOI: 10.1038/s41598-021-01502-0] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/04/2021] [Accepted: 10/25/2021] [Indexed: 12/24/2022] Open
Abstract
Since the outbreak of COVID-19 in 2019, the rapid spread of the epidemic has brought huge challenges to medical institutions. If the pathological region in the COVID-19 CT image can be automatically segmented, it will help doctors quickly determine the patient's infection, thereby speeding up the diagnosis process. To be able to automatically segment the infected area, we proposed a new network structure and named QC-HC U-Net. First, we combine residual connection and dense connection to form a new connection method and apply it to the encoder and the decoder. Second, we choose to add Hypercolumns in the decoder section. Compared with the benchmark 3D U-Net, the improved network can effectively avoid vanishing gradient while extracting more features. To improve the situation of insufficient data, resampling and data enhancement methods are selected in this paper to expand the datasets. We used 63 cases of MSD lung tumor data for training and testing, continuously verified to ensure the training effect of this model, and then selected 20 cases of public COVID-19 data for training and testing. Experimental results showed that in the segmentation of COVID-19, the specificity and sensitivity were 85.3% and 83.6%, respectively, and in the segmentation of MSD lung tumors, the specificity and sensitivity were 81.45% and 80.93%, respectively, without any fitting.
Collapse
Affiliation(s)
- Qin Zhang
- School of Computer Science and Technology, Qilu University of Technology, Jinan, 250301, China
| | - Xiaoqiang Ren
- School of Computer Science and Technology, Qilu University of Technology, Jinan, 250301, China.
| | - Benzheng Wei
- Center for Medical Artificial Intelligence, Shandong University of Traditional Chinese Medicine, Jinan, China.
| |
Collapse
|
24
|
Sun J, Chen R, Tong Q, Ma J, Gao L, Fang J, Zhang D, Chan P, He H, Wu T. Convolutional neural network optimizes the application of diffusion kurtosis imaging in Parkinson's disease. Brain Inform 2021; 8:18. [PMID: 34585306 PMCID: PMC8479023 DOI: 10.1186/s40708-021-00139-z] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/17/2021] [Accepted: 08/11/2021] [Indexed: 11/25/2022] Open
Abstract
Objectives The literature regarding the use of diffusion-tensor imaging-derived metrics in the evaluation of Parkinson’s disease (PD) is controversial. This study attempted to assess the feasibility of a deep-learning-based method for detecting alterations in diffusion kurtosis measurements associated with PD. Methods A total of 68 patients with PD and 77 healthy controls were scanned using scanner-A (3 T Skyra) (DATASET-1). Meanwhile, an additional five healthy volunteers were scanned with both scanner-A and an additional scanner-B (3 T Prisma) (DATASET-2). Diffusion kurtosis imaging (DKI) of DATASET-2 had an extra b shell compared to DATASET-1. In addition, a 3D-convolutional neural network (CNN) was trained from DATASET-2 to harmonize the quality of scalar measures of scanner-A to a similar level as scanner-B. Whole-brain unpaired t test and Tract-Based Spatial Statistics (TBSS) were performed to validate the differences between the PD and control groups using the model-fitting method and CNN-based method, respectively. We further clarified the correlation between clinical assessments and DKI results. Results An increase in mean diffusivity (MD) was found in the left substantia nigra (SN) in the PD group. In the right SN, fractional anisotropy (FA) and mean kurtosis (MK) values were negatively correlated with Hoehn and Yahr (H&Y) scales. In the putamen (Put), FA values were positively correlated with the H&Y scales. It is worth noting that these findings were only observed with the deep learning method. There was neither a group difference nor a correlation with clinical assessments in the SN or striatum exceeding the significance level using the conventional model-fitting method. Conclusions The CNN-based method improves the robustness of DKI and can help to explore PD-associated imaging features. Supplementary Information The online version contains supplementary material available at 10.1186/s40708-021-00139-z.
Collapse
Affiliation(s)
- Junyan Sun
- Department of Neurobiology, Neurology and Geriatrics, Xuanwu Hospital of Capital Medical University, National Clinical Research Center for Geriatric Disease, Beijing, 100053, China
| | - Ruike Chen
- Center for Brain Imaging Science and Technology, College of Biomedical Engineering and Instrumental Science, Zhejiang University, Hangzhou, 310027, Zhejiang, China
| | - Qiqi Tong
- Center for Brain Imaging Science and Technology, College of Biomedical Engineering and Instrumental Science, Zhejiang University, Hangzhou, 310027, Zhejiang, China.,Research Center for Healthcare Data Science, Zhejiang Lab, Hangzhou, Zhejiang, China
| | - Jinghong Ma
- Department of Neurology, Xuanwu Hospital of Capital Medical University, Beijing, China
| | - Linlin Gao
- Department of Neurobiology, Neurology and Geriatrics, Xuanwu Hospital of Capital Medical University, National Clinical Research Center for Geriatric Disease, Beijing, 100053, China
| | - Jiliang Fang
- Department of Radiology, Guang'anmen Hospital, China Academy of Chinese Medical Sciences, Beijing, China
| | - Dongling Zhang
- Department of Neurobiology, Neurology and Geriatrics, Xuanwu Hospital of Capital Medical University, National Clinical Research Center for Geriatric Disease, Beijing, 100053, China
| | - Piu Chan
- Department of Neurobiology, Neurology and Geriatrics, Xuanwu Hospital of Capital Medical University, National Clinical Research Center for Geriatric Disease, Beijing, 100053, China.,Clinical Center for Parkinson's Disease, Capital Medical University, Beijing, China.,Key Laboratory for Neurodegenerative Disease of the Ministry of Education, Beijing Key Laboratory for Parkinson's Disease, Parkinson Disease Center of Beijing Institute for Brain Disorders, Beijing, China.,National Clinical Research Center for Geriatric Disorders, Beijing, China
| | - Hongjian He
- Center for Brain Imaging Science and Technology, College of Biomedical Engineering and Instrumental Science, Zhejiang University, Hangzhou, 310027, Zhejiang, China. .,Key Laboratory for Biomedical Engineering of Ministry of Education, Zhejiang University, Hangzhou, 310027, Zhejiang, China.
| | - Tao Wu
- Department of Neurobiology, Neurology and Geriatrics, Xuanwu Hospital of Capital Medical University, National Clinical Research Center for Geriatric Disease, Beijing, 100053, China. .,Clinical Center for Parkinson's Disease, Capital Medical University, Beijing, China. .,Key Laboratory for Neurodegenerative Disease of the Ministry of Education, Beijing Key Laboratory for Parkinson's Disease, Parkinson Disease Center of Beijing Institute for Brain Disorders, Beijing, China. .,National Clinical Research Center for Geriatric Disorders, Beijing, China.
| |
Collapse
|
25
|
Zhang J, Wu J, Li Q, Caselli RJ, Thompson PM, Ye J, Wang Y. Multi-Resemblance Multi-Target Low-Rank Coding for Prediction of Cognitive Decline With Longitudinal Brain Images. IEEE TRANSACTIONS ON MEDICAL IMAGING 2021; 40:2030-2041. [PMID: 33798076 PMCID: PMC8363167 DOI: 10.1109/tmi.2021.3070780] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/12/2023]
Abstract
An effective presymptomatic diagnosis and treatment of Alzheimer's disease (AD) would have enormous public health benefits. Sparse coding (SC) has shown strong potential for longitudinal brain image analysis in preclinical AD research. However, the traditional SC computation is time-consuming and does not explore the feature correlations that are consistent over the time. In addition, longitudinal brain image cohorts usually contain incomplete image data and clinical labels. To address these challenges, we propose a novel two-stage Multi-Resemblance Multi-Target Low-Rank Coding (MMLC) method, which encourages that sparse codes of neighboring longitudinal time points are resemblant to each other, favors sparse code low-rankness to reduce the computational cost and is resilient to both source and target data incompleteness. In stage one, we propose an online multi-resemblant low-rank SC method to utilize the common and task-specific dictionaries in different time points to immune to incomplete source data and capture the longitudinal correlation. In stage two, supported by a rigorous theoretical analysis, we develop a multi-target learning method to address the missing clinical label issue. To solve such a multi-task low-rank sparse optimization problem, we propose multi-task stochastic coordinate coding with a sequence of closed-form update steps which reduces the computational costs guaranteed by a theoretical convergence proof. We apply MMLC on a publicly available neuroimaging cohort to predict two clinical measures and compare it with six other methods. Our experimental results show our proposed method achieves superior results on both computational efficiency and predictive accuracy and has great potential to assist the AD prevention.
Collapse
|
26
|
Guan H, Liu Y, Yang E, Yap PT, Shen D, Liu M. Multi-site MRI harmonization via attention-guided deep domain adaptation for brain disorder identification. Med Image Anal 2021; 71:102076. [PMID: 33930828 PMCID: PMC8184627 DOI: 10.1016/j.media.2021.102076] [Citation(s) in RCA: 43] [Impact Index Per Article: 14.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/04/2020] [Revised: 12/21/2020] [Accepted: 04/03/2021] [Indexed: 01/18/2023]
Abstract
Structural magnetic resonance imaging (MRI) has shown great clinical and practical values in computer-aided brain disorder identification. Multi-site MRI data increase sample size and statistical power, but are susceptible to inter-site heterogeneity caused by different scanners, scanning protocols, and subject cohorts. Multi-site MRI harmonization (MMH) helps alleviate the inter-site difference for subsequent analysis. Some MMH methods performed at imaging level or feature extraction level are concise but lack robustness and flexibility to some extent. Even though several machine/deep learning-based methods have been proposed for MMH, some of them require a portion of labeled data in the to-be-analyzed target domain or ignore the potential contributions of different brain regions to the identification of brain disorders. In this work, we propose an attention-guided deep domain adaptation (AD2A) framework for MMH and apply it to automated brain disorder identification with multi-site MRIs. The proposed framework does not need any category label information of target data, and can also automatically identify discriminative regions in whole-brain MR images. Specifically, the proposed AD2A is composed of three key modules: (1) an MRI feature encoding module to extract representations of input MRIs, (2) an attention discovery module to automatically locate discriminative dementia-related regions in each whole-brain MRI scan, and (3) a domain transfer module trained with adversarial learning for knowledge transfer between the source and target domains. Experiments have been performed on 2572 subjects from four benchmark datasets with T1-weighted structural MRIs, with results demonstrating the effectiveness of the proposed method in both tasks of brain disorder identification and disease progression prediction.
Collapse
Affiliation(s)
- Hao Guan
- Department of Radiology and Biomedical Research Imaging Center, University of North Carolina at Chapel Hill, Chapel Hill, NC 27599, USA
| | - Yunbi Liu
- Department of Radiology and Biomedical Research Imaging Center, University of North Carolina at Chapel Hill, Chapel Hill, NC 27599, USA; School of Biomedical Engineering, Southern Medical University, Guangzhou 510515, China
| | - Erkun Yang
- Department of Radiology and Biomedical Research Imaging Center, University of North Carolina at Chapel Hill, Chapel Hill, NC 27599, USA
| | - Pew-Thian Yap
- Department of Radiology and Biomedical Research Imaging Center, University of North Carolina at Chapel Hill, Chapel Hill, NC 27599, USA
| | - Dinggang Shen
- Department of Radiology and Biomedical Research Imaging Center, University of North Carolina at Chapel Hill, Chapel Hill, NC 27599, USA
| | - Mingxia Liu
- Department of Radiology and Biomedical Research Imaging Center, University of North Carolina at Chapel Hill, Chapel Hill, NC 27599, USA.
| |
Collapse
|
27
|
Han K, Luo J, Xiao Q, Ning Z, Zhang Y. Light-weight cross-view hierarchical fusion network for joint localization and identification in Alzheimer's disease with adaptive instance-declined pruning. Phys Med Biol 2021; 66. [PMID: 33765665 DOI: 10.1088/1361-6560/abf200] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/17/2020] [Accepted: 03/25/2021] [Indexed: 11/11/2022]
Abstract
Magnetic resonance imaging (MRI) has been widely used in assessing development of Alzheimer's disease (AD) by providing structural information of disease-associated regions (e.g. atrophic regions). In this paper, we propose a light-weight cross-view hierarchical fusion network (CvHF-net), consisting of local patch and global subject subnets, for joint localization and identification of the discriminative local patches and regions in the whole brain MRI, upon which feature representations are then jointly learned and fused to construct hierarchical classification models for AD diagnosis. Firstly, based on the extracted class-discriminative 3D patches, we employ the local patch subnets to utilize multiple 2D views to represent 3D patches by using an attention-aware hierarchical fusion structure in a divide-and-conquer manner. Since different local patches are with various abilities in AD identification, the global subject subnet is developed to bias the allocation of available resources towards the most informative parts among these local patches to obtain global information for AD identification. Besides, an instance declined pruning algorithm is embedded in the CvHF-net for adaptively selecting most discriminant patches in a task-driven manner. The proposed method was evaluated on the AD Neuroimaging Initiative dataset and the experimental results show that our proposed method can achieve good performance on AD diagnosis.
Collapse
Affiliation(s)
- Kangfu Han
- School of Biomedical Engineering, Southern Medical University, Guangzhou, Guangdong, 510515, People's Republic of China.,Guangdong Provincial Key Laboratory of Medical Image Processing, Guangzhou, Guangdong, 510515, People's Republic of China
| | - Jiaxiu Luo
- School of Biomedical Engineering, Southern Medical University, Guangzhou, Guangdong, 510515, People's Republic of China.,Guangdong Provincial Key Laboratory of Medical Image Processing, Guangzhou, Guangdong, 510515, People's Republic of China
| | - Qing Xiao
- School of Biomedical Engineering, Southern Medical University, Guangzhou, Guangdong, 510515, People's Republic of China.,Guangdong Provincial Key Laboratory of Medical Image Processing, Guangzhou, Guangdong, 510515, People's Republic of China
| | - Zhenyuan Ning
- School of Biomedical Engineering, Southern Medical University, Guangzhou, Guangdong, 510515, People's Republic of China.,Guangdong Provincial Key Laboratory of Medical Image Processing, Guangzhou, Guangdong, 510515, People's Republic of China
| | - Yu Zhang
- School of Biomedical Engineering, Southern Medical University, Guangzhou, Guangdong, 510515, People's Republic of China.,Guangdong Provincial Key Laboratory of Medical Image Processing, Guangzhou, Guangdong, 510515, People's Republic of China
| |
Collapse
|
28
|
Xue J, He K, Nie D, Adeli E, Shi Z, Lee SW, Zheng Y, Liu X, Li D, Shen D. Cascaded MultiTask 3-D Fully Convolutional Networks for Pancreas Segmentation. IEEE TRANSACTIONS ON CYBERNETICS 2021; 51:2153-2165. [PMID: 31869812 DOI: 10.1109/tcyb.2019.2955178] [Citation(s) in RCA: 18] [Impact Index Per Article: 6.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/10/2023]
Abstract
Automatic pancreas segmentation is crucial to the diagnostic assessment of diabetes or pancreatic cancer. However, the relatively small size of the pancreas in the upper body, as well as large variations of its location and shape in retroperitoneum, make the segmentation task challenging. To alleviate these challenges, in this article, we propose a cascaded multitask 3-D fully convolution network (FCN) to automatically segment the pancreas. Our cascaded network is composed of two parts. The first part focuses on fast locating the region of the pancreas, and the second part uses a multitask FCN with dense connections to refine the segmentation map for fine voxel-wise segmentation. In particular, our multitask FCN with dense connections is implemented to simultaneously complete tasks of the voxel-wise segmentation and skeleton extraction from the pancreas. These two tasks are complementary, that is, the extracted skeleton provides rich information about the shape and size of the pancreas in retroperitoneum, which can boost the segmentation of pancreas. The multitask FCN is also designed to share the low- and mid-level features across the tasks. A feature consistency module is further introduced to enhance the connection and fusion of different levels of feature maps. Evaluations on two pancreas datasets demonstrate the robustness of our proposed method in correctly segmenting the pancreas in various settings. Our experimental results outperform both baseline and state-of-the-art methods. Moreover, the ablation study shows that our proposed parts/modules are critical for effective multitask learning.
Collapse
|
29
|
Wu H, Lu X, Lei B, Wen Z. Automated left ventricular segmentation from cardiac magnetic resonance images via adversarial learning with multi-stage pose estimation network and co-discriminator. Med Image Anal 2020; 68:101891. [PMID: 33260108 DOI: 10.1016/j.media.2020.101891] [Citation(s) in RCA: 11] [Impact Index Per Article: 2.8] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/21/2020] [Revised: 10/10/2020] [Accepted: 10/30/2020] [Indexed: 11/29/2022]
Abstract
Left ventricular (LV) segmentation is essential for the early diagnosis of cardiovascular diseases, which has been reported as the leading cause of death all over the world. However, automated LV segmentation from cardiac magnetic resonance images (CMRI) using the traditional convolutional neural networks (CNNs) is still a challenging task due to the limited labeled CMRI data and low tolerances to irregular scales, shapes and deformations of LV. In this paper, we propose an automated LV segmentation method based on adversarial learning by integrating a multi-stage pose estimation network (MSPN) and a co-discrimination network. Different from existing CNNs, we use a MSPN with multi-scale dilated convolution (MDC) modules to enhance the ranges of receptive field for deep feature extraction. To fully utilize both labeled and unlabeled CMRI data, we propose a novel generative adversarial network (GAN) framework for LV segmentation by combining MSPN with co-discrimination networks. Specifically, the labeled CMRI are first used to initialize our segmentation network (MSPN) and co-discrimination network. Our GAN training includes two different kinds of epochs fed with both labeled and unlabeled CMRI data alternatively, which are different from the traditional CNNs only relied on the limited labeled samples to train the segmentation networks. As both ground truth and unlabeled samples are involved in guiding training, our method not only can converge faster but also obtain a better performance in LV segmentation. Our method is evaluated using MICCAI 2009 and 2017 challenge databases. Experimental results show that our method has obtained promising performance in LV segmentation, which also outperforms the state-of-the-art methods in terms of LV segmentation accuracy from the comparison results.
Collapse
Affiliation(s)
- Huisi Wu
- College of Computer Science and Software Engineering, Shenzhen University, Shenzhen, China, 518060
| | - Xuheng Lu
- College of Computer Science and Software Engineering, Shenzhen University, Shenzhen, China, 518060
| | - Baiying Lei
- National-Regional Key Technology Engineering Laboratory for Medical Ultrasound, Guangdong Key Laboratory for Biomedical Measurements and Ultrasound Imaging, School of Biomedical Engineering, Health Science Center, Shenzhen University, Shenzhen, China, 518060.
| | - Zhenkun Wen
- College of Computer Science and Software Engineering, Shenzhen University, Shenzhen, China, 518060
| |
Collapse
|
30
|
Zhang X, Wei Y, Yang Y, Huang TS. SG-One: Similarity Guidance Network for One-Shot Semantic Segmentation. IEEE TRANSACTIONS ON CYBERNETICS 2020; 50:3855-3865. [PMID: 32497014 DOI: 10.1109/tcyb.2020.2992433] [Citation(s) in RCA: 36] [Impact Index Per Article: 9.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/11/2023]
Abstract
One-shot image semantic segmentation poses a challenging task of recognizing the object regions from unseen categories with only one annotated example as supervision. In this article, we propose a simple yet effective similarity guidance network to tackle the one-shot (SG-One) segmentation problem. We aim at predicting the segmentation mask of a query image with the reference to one densely labeled support image of the same category. To obtain the robust representative feature of the support image, we first adopt a masked average pooling strategy for producing the guidance features by only taking the pixels belonging to the support image into account. We then leverage the cosine similarity to build the relationship between the guidance features and features of pixels from the query image. In this way, the possibilities embedded in the produced similarity maps can be adopted to guide the process of segmenting objects. Furthermore, our SG-One is a unified framework that can efficiently process both support and query images within one network and be learned in an end-to-end manner. We conduct extensive experiments on Pascal VOC 2012. In particular, our SG-One achieves the mIoU score of 46.3%, surpassing the baseline methods.
Collapse
|
31
|
Li Q, Xing X, Feng Q. [Coupled convolutional and graph network-based diagnosis of Alzheimer's disease using MRI]. NAN FANG YI KE DA XUE XUE BAO = JOURNAL OF SOUTHERN MEDICAL UNIVERSITY 2020; 40:531-537. [PMID: 32895137 DOI: 10.12122/j.issn.1673-4254.2020.04.13] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [Subscribe] [Scholar Register] [Indexed: 11/24/2022]
Abstract
OBJECTIVE To propose a coupled convolutional and graph convolutional network (CCGCN) model for diagnosis of Alzheimer's disease (AD) and its prodromal stage. METHODS The disease-related brain regions generated by group-wise comparison were used as the input. The convolutional neural networks (CNNs) were used to extract disease-related features from different locations on brain magnetic resonance (MR) images. The generated features via the graph convolutional network (GCN) were processed, and graph pooling was performed to analyze the inherent relationship between the brain topology and the diagnosis task adaptively. Through ADNI dataset, we acquired the accuracy, sensitivity and specificity of the diagnosis tasks for AD and its prodromal stages, followed by an ablation study on the model structure. RESULTS The CCGCN model outperformed the current state-of-the-art methods and showed a classification accuracy of 92.5% for AD with a sensitivity of 88.1% and a specificity of 96.0%. CONCLUSIONS Based on the structural and topological features of the brain MR images, the proposed CCGCN model shows excellent performance in AD diagnosis and is expected to provide important assistance to physicians in disease diagnosis.
Collapse
Affiliation(s)
- Qingfeng Li
- School of Biomedical Engineering, Guangzhou 510515, China.,Guangdong Provincial Key Laboratory of Medical Image Processing, Southern Medical University, Guangzhou 510515, China
| | - Xiaodan Xing
- Medical Imaging Center, Shanghai Advanced Research Institute, Shanghai 201210, China
| | - Qianjin Feng
- School of Biomedical Engineering, Guangzhou 510515, China.,Guangdong Provincial Key Laboratory of Medical Image Processing, Southern Medical University, Guangzhou 510515, China
| |
Collapse
|
32
|
Oh K, Chung YC, Kim KW, Kim WS, Oh IS. Classification and Visualization of Alzheimer's Disease using Volumetric Convolutional Neural Network and Transfer Learning. Sci Rep 2019; 9:18150. [PMID: 31796817 PMCID: PMC6890708 DOI: 10.1038/s41598-019-54548-6] [Citation(s) in RCA: 83] [Impact Index Per Article: 16.6] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/11/2019] [Accepted: 11/12/2019] [Indexed: 12/21/2022] Open
Abstract
Recently, deep-learning-based approaches have been proposed for the classification of neuroimaging data related to Alzheimer's disease (AD), and significant progress has been made. However, end-to-end learning that is capable of maximizing the impact of deep learning has yet to receive much attention due to the endemic challenge of neuroimaging caused by the scarcity of data. Thus, this study presents an approach meant to encourage the end-to-end learning of a volumetric convolutional neural network (CNN) model for four binary classification tasks (AD vs. normal control (NC), progressive mild cognitive impairment (pMCI) vs. NC, stable mild cognitive impairment (sMCI) vs. NC and pMCI vs. sMCI) based on magnetic resonance imaging (MRI) and visualizes its outcomes in terms of the decision of the CNNs without any human intervention. In the proposed approach, we use convolutional autoencoder (CAE)-based unsupervised learning for the AD vs. NC classification task, and supervised transfer learning is applied to solve the pMCI vs. sMCI classification task. To detect the most important biomarkers related to AD and pMCI, a gradient-based visualization method that approximates the spatial influence of the CNN model's decision was applied. To validate the contributions of this study, we conducted experiments on the ADNI database, and the results demonstrated that the proposed approach achieved the accuracies of 86.60% and 73.95% for the AD and pMCI classification tasks respectively, outperforming other network models. In the visualization results, the temporal and parietal lobes were identified as key regions for classification.
Collapse
Affiliation(s)
- Kanghan Oh
- Jeonbuk National University, Department of Computer Science and Engineering, Jeonju, 54896, Korea
| | - Young-Chul Chung
- Research Institute of Clinical Medicine of Jeonbuk National University-Biomedical Research Institute of Jeonbuk National University Hospital, Jeonju, 54907, Korea
- Jeonbuk National University Medical School, Department of Psychiatry, Jeonju, 54907, Korea
| | - Ko Woon Kim
- Jeonbuk National University Medical School, Department of Neurology, Jeonju, 54907, Korea
| | - Woo-Sung Kim
- Research Institute of Clinical Medicine of Jeonbuk National University-Biomedical Research Institute of Jeonbuk National University Hospital, Jeonju, 54907, Korea
- Jeonbuk National University Medical School, Department of Psychiatry, Jeonju, 54907, Korea
| | - Il-Seok Oh
- Jeonbuk National University, Department of Computer Science and Engineering, Jeonju, 54896, Korea.
| |
Collapse
|
33
|
Lian C, Liu M, Wang L, Shen D. End-to-End Dementia Status Prediction from Brain MRI Using Multi-task Weakly-Supervised Attention Network. MEDICAL IMAGE COMPUTING AND COMPUTER-ASSISTED INTERVENTION : MICCAI ... INTERNATIONAL CONFERENCE ON MEDICAL IMAGE COMPUTING AND COMPUTER-ASSISTED INTERVENTION 2019; 11767:158-167. [PMID: 34355224 PMCID: PMC8336422] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Grants] [Subscribe] [Scholar Register] [Indexed: 06/13/2023]
Abstract
Computer-aided prediction of dementia status (e.g., clinical scores of cognitive tests) from brain MRI is of great clinical value, as it can help assess pathological stage and predict disease progression. Existing learning-based approaches typically preselect dementia-sensitive regions from the whole-brain MRI for feature extraction and prediction model construction, which might be sub-optimal due to potential heterogeneities between different steps. Also, based on anatomical prior knowledge (e.g., brain atlas) and time-consuming nonlinear registration, these preselected brain regions are usually the same across all subjects, ignoring their individual specificities in dementia progression. In this paper, we propose a multi-task weakly-supervised attention network (MWAN) to jointly predict multiple clinical scores from the baseline MRI data, by explicitly considering individual specificities of different subjects. Leveraging a fully-trainable dementia attention block, our MWAN method can automatically identify subject-specific discriminative locations from the whole-brain MRI for end-to-end feature learning and multi-task regression. We evaluated our MWAN method by cross-validation on two public datasets (i.e., ADNI-1 and ADNI-2). Experimental results demonstrate that the proposed method performs well in both the tasks of clinical score prediction and weakly-supervised discriminative localization in brain MR images.
Collapse
Affiliation(s)
- Chunfeng Lian
- Department of Radiology and BRIC, University of North Carolina at Chapel Hill, Chapel Hill, NC 27599, USA
| | - Mingxia Liu
- Department of Radiology and BRIC, University of North Carolina at Chapel Hill, Chapel Hill, NC 27599, USA
| | - Li Wang
- Department of Radiology and BRIC, University of North Carolina at Chapel Hill, Chapel Hill, NC 27599, USA
| | - Dinggang Shen
- Department of Radiology and BRIC, University of North Carolina at Chapel Hill, Chapel Hill, NC 27599, USA
| |
Collapse
|