1
|
Tinauer C, Heber S, Pirpamer L, Damulina A, Schmidt R, Stollberger R, Ropele S, Langkammer C. Interpretable brain disease classification and relevance-guided deep learning. Sci Rep 2022; 12:20254. [PMID: 36424437 PMCID: PMC9691637 DOI: 10.1038/s41598-022-24541-7] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/25/2022] [Accepted: 11/16/2022] [Indexed: 11/27/2022] Open
Abstract
Deep neural networks are increasingly used for neurological disease classification by MRI, but the networks' decisions are not easily interpretable by humans. Heat mapping by deep Taylor decomposition revealed that (potentially misleading) image features even outside of the brain tissue are crucial for the classifier's decision. We propose a regularization technique to train convolutional neural network (CNN) classifiers utilizing relevance-guided heat maps calculated online during training. The method was applied using T1-weighted MR images from 128 subjects with Alzheimer's disease (mean age = 71.9 ± 8.5 years) and 290 control subjects (mean age = 71.3 ± 6.4 years). The developed relevance-guided framework achieves higher classification accuracies than conventional CNNs but more importantly, it relies on less but more relevant and physiological plausible voxels within brain tissue. Additionally, preprocessing effects from skull stripping and registration are mitigated. With the interpretability of the decision mechanisms underlying CNNs, these results challenge the notion that unprocessed T1-weighted brain MR images in standard CNNs yield higher classification accuracy in Alzheimer's disease than solely atrophy.
Collapse
Affiliation(s)
- Christian Tinauer
- grid.11598.340000 0000 8988 2476Department of Neurology, Medical University of Graz, Graz, Austria
| | - Stefan Heber
- grid.11598.340000 0000 8988 2476Department of Neurology, Medical University of Graz, Graz, Austria
| | - Lukas Pirpamer
- grid.11598.340000 0000 8988 2476Department of Neurology, Medical University of Graz, Graz, Austria ,grid.6612.30000 0004 1937 0642Medical Image Analysis Center (MIAC) and Department of Biomedical Engineering, University of Basel, Basel, Switzerland
| | - Anna Damulina
- grid.11598.340000 0000 8988 2476Department of Neurology, Medical University of Graz, Graz, Austria
| | - Reinhold Schmidt
- grid.11598.340000 0000 8988 2476Department of Neurology, Medical University of Graz, Graz, Austria
| | - Rudolf Stollberger
- grid.410413.30000 0001 2294 748XInstitute of Biomedical Imaging, Graz University of Technology, Graz, Austria ,grid.452216.6BioTechMed-Graz, Graz, Austria
| | - Stefan Ropele
- grid.11598.340000 0000 8988 2476Department of Neurology, Medical University of Graz, Graz, Austria ,grid.452216.6BioTechMed-Graz, Graz, Austria
| | - Christian Langkammer
- grid.11598.340000 0000 8988 2476Department of Neurology, Medical University of Graz, Graz, Austria ,grid.452216.6BioTechMed-Graz, Graz, Austria
| |
Collapse
|
2
|
Quantifiable brain atrophy synthesis for benchmarking of cortical thickness estimation methods. Med Image Anal 2022; 82:102576. [DOI: 10.1016/j.media.2022.102576] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/13/2021] [Revised: 05/10/2022] [Accepted: 08/11/2022] [Indexed: 12/11/2022]
|
3
|
Hernandez M, Ramon-Julvez U, Sierra-Tome D. Partial Differential Equation-Constrained Diffeomorphic Registration from Sum of Squared Differences to Normalized Cross-Correlation, Normalized Gradient Fields, and Mutual Information: A Unifying Framework. SENSORS (BASEL, SWITZERLAND) 2022; 22:3735. [PMID: 35632143 PMCID: PMC9146848 DOI: 10.3390/s22103735] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 04/20/2022] [Revised: 05/10/2022] [Accepted: 05/11/2022] [Indexed: 06/15/2023]
Abstract
This work proposes a unifying framework for extending PDE-constrained Large Deformation Diffeomorphic Metric Mapping (PDE-LDDMM) with the sum of squared differences (SSD) to PDE-LDDMM with different image similarity metrics. We focused on the two best-performing variants of PDE-LDDMM with the spatial and band-limited parameterizations of diffeomorphisms. We derived the equations for gradient-descent and Gauss-Newton-Krylov (GNK) optimization with Normalized Cross-Correlation (NCC), its local version (lNCC), Normalized Gradient Fields (NGFs), and Mutual Information (MI). PDE-LDDMM with GNK was successfully implemented for NCC and lNCC, substantially improving the registration results of SSD. For these metrics, GNK optimization outperformed gradient-descent. However, for NGFs, GNK optimization was not able to overpass the performance of gradient-descent. For MI, GNK optimization involved the product of huge dense matrices, requesting an unaffordable memory load. The extensive evaluation reported the band-limited version of PDE-LDDMM based on the deformation state equation with NCC and lNCC image similarities among the best performing PDE-LDDMM methods. In comparison with benchmark deep learning-based methods, our proposal reached or surpassed the accuracy of the best-performing models. In NIREP16, several configurations of PDE-LDDMM outperformed ANTS-lNCC, the best benchmark method. Although NGFs and MI usually underperformed the other metrics in our evaluation, these metrics showed potentially competitive results in a multimodal deformable experiment. We believe that our proposed image similarity extension over PDE-LDDMM will promote the use of physically meaningful diffeomorphisms in a wide variety of clinical applications depending on deformable image registration.
Collapse
Affiliation(s)
- Monica Hernandez
- Aragon Institute of Engineering Research (I3A), 50018 Zaragoza, Spain
- Department of Computer Sciences, University of Zaragoza (UZ), 50018 Zaragoza, Spain; (U.R.-J.); (D.S.-T.)
| | - Ubaldo Ramon-Julvez
- Department of Computer Sciences, University of Zaragoza (UZ), 50018 Zaragoza, Spain; (U.R.-J.); (D.S.-T.)
| | - Daniel Sierra-Tome
- Department of Computer Sciences, University of Zaragoza (UZ), 50018 Zaragoza, Spain; (U.R.-J.); (D.S.-T.)
| |
Collapse
|
4
|
SimulAD: A dynamical model for personalized simulation and disease staging in Alzheimer’s disease. Neurobiol Aging 2022; 113:73-83. [DOI: 10.1016/j.neurobiolaging.2021.12.015] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/27/2021] [Revised: 12/13/2021] [Accepted: 12/30/2021] [Indexed: 11/17/2022]
|
5
|
Pilia M, Kullberg J, Ahlström H, Malmberg F, Ekström S, Strand R. Average volume reference space for large scale registration of whole-body magnetic resonance images. PLoS One 2019; 14:e0222700. [PMID: 31574093 PMCID: PMC6772040 DOI: 10.1371/journal.pone.0222700] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.4] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/10/2018] [Accepted: 09/05/2019] [Indexed: 01/12/2023] Open
Abstract
BACKGROUND AND OBJECTIVES The construction of whole-body magnetic resonance (MR) imaging atlases allows to perform statistical analysis with applications in anomaly detection, longitudinal, and correlation studies. Atlas-based methods require a common coordinate system to which all the subjects are mapped through image registration. Optimisation of the reference space is an important aspect that affects the subsequent analysis of the registered data, and having a reference space that is neutral with respect to local tissue volume is valuable in correlation studies. The purpose of this work is to generate a reference space for whole-body imaging that has zero voxel-wise average volume change when mapped to a cohort. METHODS This work proposes an approach to register multiple whole-body images to a common template using volume changes to generate a synthetic reference space, starting with an initial reference and refining it by warping it with a deformation that brings the voxel-wise average volume change associated to the mappings of all the images in the cohort to zero. RESULTS Experiments on fat/water separated whole-body MR images show how the method effectively generates a reference space neutral with respect to volume changes, without reducing the quality of the registration nor introducing artefacts in the anatomy, while providing better alignment when compared to an implicit reference groupwise approach. CONCLUSIONS The proposed method allows to quickly generate a reference space neutral with respect to local volume changes, that retains the registration quality of a sharp template, and that can be used for statistical analysis of voxel-wise correlations in large datasets of whole-body image data.
Collapse
Affiliation(s)
- Martino Pilia
- Department of Surgical Sciences, Uppsala University, Uppsala, Sweden
- Department of Information Technology, Uppsala University, Uppsala, Sweden
| | - Joel Kullberg
- Department of Surgical Sciences, Uppsala University, Uppsala, Sweden
- Antaros Medical, Uppsala, Sweden
| | - Håkan Ahlström
- Department of Surgical Sciences, Uppsala University, Uppsala, Sweden
- Antaros Medical, Uppsala, Sweden
| | - Filip Malmberg
- Department of Surgical Sciences, Uppsala University, Uppsala, Sweden
- Department of Information Technology, Uppsala University, Uppsala, Sweden
| | - Simon Ekström
- Department of Surgical Sciences, Uppsala University, Uppsala, Sweden
| | - Robin Strand
- Department of Surgical Sciences, Uppsala University, Uppsala, Sweden
- Department of Information Technology, Uppsala University, Uppsala, Sweden
| |
Collapse
|
6
|
A model of brain morphological changes related to aging and Alzheimer's disease from cross-sectional assessments. Neuroimage 2019; 198:255-270. [DOI: 10.1016/j.neuroimage.2019.05.040] [Citation(s) in RCA: 19] [Impact Index Per Article: 3.8] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/21/2018] [Revised: 05/14/2019] [Accepted: 05/15/2019] [Indexed: 01/06/2023] Open
|
7
|
Targeted Gold Nanoparticle⁻Oligonucleotide Contrast Agents in Combination with a New Local Voxel-Wise MRI Analysis Algorithm for In Vitro Imaging of Triple-Negative Breast Cancer. NANOMATERIALS 2019; 9:nano9050709. [PMID: 31067749 PMCID: PMC6566234 DOI: 10.3390/nano9050709] [Citation(s) in RCA: 14] [Impact Index Per Article: 2.8] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 03/05/2019] [Revised: 04/26/2019] [Accepted: 04/28/2019] [Indexed: 01/23/2023]
Abstract
Gold nanoparticles (GNPs) have tremendous potential as cancer-targeted contrast agents for diagnostic imaging. The ability to modify the particle surface with both disease-targeting molecules (such as the cancer-specific aptamer AS1411) and contrast agents (such as the gadolinium chelate Gd(III)-DO3A-SH) enables tailoring the particles for specific cancer-imaging and diagnosis. While the amount of image contrast generated by nanoparticle contrast agents is often low, it can be augmented with the assistance of computer image analysis algorithms. In this work, the ability of cancer-targeted gold nanoparticle–oligonucleotide conjugates to distinguish between malignant (MDA-MB-231) and healthy cells (MCF-10A) is tested using a T1-weighted image analysis algorithm based on three-dimensional, deformable model-based segmentation to extract the Volume of Interest (VOI). The gold nanoparticle/algorithm tandem was tested using contrast agent GNP-Gd(III)-DO3A-SH-AS1411) and nontargeted c-rich oligonucleotide (CRO) analogs and control (CTR) counterparts (GNP-Gd(III)-DO3A-SH-CRO/CTR) via in vitro studies. Remarkably, the cancer cells were notably distinguished from the nonmalignant cells, especially at nanomolar contrast agent concentrations. The T1-weighted image analysis algorithm provided similar results to the industry standard Varian software interface (VNMRJ) analysis of T1 maps at micromolar contrast agent concentrations, in which the VNMRJ produced a 19.5% better MRI contrast enhancement. However, our algorithm provided more sensitive and consistent results at nanomolar contrast agent concentrations, where our algorithm produced ~500% better MRI contrast enhancement.
Collapse
|
8
|
Abstract
The class of registration methods proposed in the framework of Stokes large deformation diffeomorphic metric mapping (LDDMM) is a particularly interesting family of physically meaningful diffeomorphic registration methods. Stokes-LDDMM methods are formulated as constrained variational problems, where the different physical models are imposed using the associated partial differential equations as hard constraints. The most significant limitation of Stokes-LDDMM framework is its huge computational complexity. The objective of this paper is to promote the use of Stokes-LDDMM in computational anatomy applications with an efficient approximation of the original variational problem. Thus, we propose a novel method for efficient Stokes-LDDMM diffeomorphic registration. Our method poses the constrained variational problem in the space of band-limited vector fields and it is implemented in the GPU. The performance of band-limited Stokes-LDDMM has been compared and evaluated with original Stokes-LDDMM, EPDiff-LDDMM, and band-limited EPDiff-LDDMM. The evaluation has been conducted in 3-D with the nonrigid image registration evaluation project database. Since the update equation in Stokes-LDDMM involves the action of low-pass filters, the computational complexity has been greatly alleviated with a modest accuracy lose. We have obtained a competitive performance for some method configurations. Overall, our proposed method may make feasible the extensive use of novel physically meaningful Stokes-LDDMM methods in different computational anatomy applications. In addition, our results reinforce the usefulness of band-limited vector fields in diffeomorphic registration methods involving the action of low-pass filters in the optimization, even in algorithmically challenging environments such as Stokes-LDDMM.
Collapse
|
9
|
Khanal B, Ayache N, Pennec X. Simulating Longitudinal Brain MRIs with Known Volume Changes and Realistic Variations in Image Intensity. Front Neurosci 2017; 11:132. [PMID: 28381986 PMCID: PMC5360759 DOI: 10.3389/fnins.2017.00132] [Citation(s) in RCA: 9] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/28/2016] [Accepted: 03/06/2017] [Indexed: 12/17/2022] Open
Abstract
This paper presents a simulator tool that can simulate large databases of visually realistic longitudinal MRIs with known volume changes. The simulator is based on a previously proposed biophysical model of brain deformation due to atrophy in AD. In this work, we propose a novel way of reproducing realistic intensity variation in longitudinal brain MRIs, which is inspired by an approach used for the generation of synthetic cardiac sequence images. This approach combines a deformation field obtained from the biophysical model with a deformation field obtained by a non-rigid registration of two images. The combined deformation field is then used to simulate a new image with specified atrophy from the first image, but with the intensity characteristics of the second image. This allows to generate the realistic variations present in real longitudinal time-series of images, such as the independence of noise between two acquisitions and the potential presence of variable acquisition artifacts. Various options available in the simulator software are briefly explained in this paper. In addition, the software is released as an open-source repository. The availability of the software allows researchers to produce tailored databases of images with ground truth volume changes; we believe this will help developing more robust brain morphometry tools. Additionally, we believe that the scientific community can also use the software to further experiment with the proposed model, and add more complex models of brain deformation and atrophy generation.
Collapse
Affiliation(s)
- Bishesh Khanal
- Asclepios, INRIA Sophia Antipolis MediterranéSophia Antipolis, France
| | | | | |
Collapse
|
10
|
Alberdi A, Aztiria A, Basarab A. On the early diagnosis of Alzheimer's Disease from multimodal signals: A survey. Artif Intell Med 2016; 71:1-29. [PMID: 27506128 DOI: 10.1016/j.artmed.2016.06.003] [Citation(s) in RCA: 83] [Impact Index Per Article: 10.4] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/29/2016] [Revised: 05/23/2016] [Accepted: 06/07/2016] [Indexed: 11/15/2022]
Abstract
INTRODUCTION The number of Alzheimer's Disease (AD) patients is increasing with increased life expectancy and 115.4 million people are expected to be affected in 2050. Unfortunately, AD is commonly diagnosed too late, when irreversible damages have been caused in the patient. OBJECTIVE An automatic, continuous and unobtrusive early AD detection method would be required to improve patients' life quality and avoid big healthcare costs. Thus, the objective of this survey is to review the multimodal signals that could be used in the development of such a system, emphasizing on the accuracy that they have shown up to date for AD detection. Some useful tools and specific issues towards this goal will also have to be reviewed. METHODS An extensive literature review was performed following a specific search strategy, inclusion criteria, data extraction and quality assessment in the Inspec, Compendex and PubMed databases. RESULTS This work reviews the extensive list of psychological, physiological, behavioural and cognitive measurements that could be used for AD detection. The most promising measurements seem to be magnetic resonance imaging (MRI) for AD vs control (CTL) discrimination with an 98.95% accuracy, while electroencephalogram (EEG) shows the best results for mild cognitive impairment (MCI) vs CTL (97.88%) and MCI vs AD distinction (94.05%). Available physiological and behavioural AD datasets are listed, as well as medical imaging analysis steps and neuroimaging processing toolboxes. Some issues such as "label noise" and multi-site data are discussed. CONCLUSIONS The development of an unobtrusive and transparent AD detection system should be based on a multimodal system in order to take full advantage of all kinds of symptoms, detect even the smallest changes and combine them, so as to detect AD as early as possible. Such a multimodal system might probably be based on physiological monitoring of MRI or EEG, as well as behavioural measurements like the ones proposed along the article. The mentioned AD datasets and image processing toolboxes are available for their use towards this goal. Issues like "label noise" and multi-site neuroimaging incompatibilities may also have to be overcome, but methods for this purpose are already available.
Collapse
Affiliation(s)
- Ane Alberdi
- Mondragon University, Electronics and Computing Department, Goiru Kalea, 2, Arrasate 20500, Spain.
| | - Asier Aztiria
- Mondragon University, Electronics and Computing Department, Goiru Kalea, 2, Arrasate 20500, Spain.
| | - Adrian Basarab
- Université de Toulouse, Institut de Recherche en Informatique de Toulouse, Centre National de la Recherche Scientifique, Unité Mixte de Recherche 5505, Université Paul Sabatier, 118 Route de Narbonne, 31062 Toulouse, France.
| |
Collapse
|