1
|
Ribaldi F, Altomare D, Jovicich J, Ferrari C, Picco A, Pizzini FB, Soricelli A, Mega A, Ferretti A, Drevelegas A, Bosch B, Müller BW, Marra C, Cavaliere C, Bartrés-Faz D, Nobili F, Alessandrini F, Barkhof F, Gros-Dagnac H, Ranjeva JP, Wiltfang J, Kuijer J, Sein J, Hoffmann KT, Roccatagliata L, Parnetti L, Tsolaki M, Constantinidis M, Aiello M, Salvatore M, Montalti M, Caulo M, Didic M, Bargallo N, Blin O, Rossini PM, Schonknecht P, Floridi P, Payoux P, Visser PJ, Bordet R, Lopes R, Tarducci R, Bombois S, Hensch T, Fiedler U, Richardson JC, Frisoni GB, Marizzoni M. Accuracy and reproducibility of automated white matter hyperintensities segmentation with lesion segmentation tool: A European multi-site 3T study. Magn Reson Imaging 2020; 76:108-115. [PMID: 33220450 DOI: 10.1016/j.mri.2020.11.008] [Citation(s) in RCA: 20] [Impact Index Per Article: 5.0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/22/2020] [Revised: 10/02/2020] [Accepted: 11/14/2020] [Indexed: 01/18/2023]
Abstract
Brain vascular damage accumulate in aging and often manifest as white matter hyperintensities (WMHs) on MRI. Despite increased interest in automated methods to segment WMHs, a gold standard has not been achieved and their longitudinal reproducibility has been poorly investigated. The aim of present work is to evaluate accuracy and reproducibility of two freely available segmentation algorithms. A harmonized MRI protocol was implemented in 3T-scanners across 13 European sites, each scanning five volunteers twice (test-retest) using 2D-FLAIR. Automated segmentation was performed using Lesion segmentation tool algorithms (LST): the Lesion growth algorithm (LGA) in SPM8 and 12 and the Lesion prediction algorithm (LPA). To assess reproducibility, we applied the LST longitudinal pipeline to the LGA and LPA outputs for both the test and retest scans. We evaluated volumetric and spatial accuracy comparing LGA and LPA with manual tracing, and for reproducibility the test versus retest. Median volume difference between automated WMH and manual segmentations (mL) was -0.22[IQR = 0.50] for LGA-SPM8, -0.12[0.57] for LGA-SPM12, -0.09[0.53] for LPA, while the spatial accuracy (Dice Coefficient) was 0.29[0.31], 0.33[0.26] and 0.41[0.23], respectively. The reproducibility analysis showed a median reproducibility error of 20%[IQR = 41] for LGA-SPM8, 14% [31] for LGA-SPM12 and 10% [27] with the LPA cross-sectional pipeline. Applying the LST longitudinal pipeline, the reproducibility errors were considerably reduced (LGA: 0%[IQR = 0], p < 0.001; LPA: 0% [3], p < 0.001) compared to those derived using the cross-sectional algorithms. The DC using the longitudinal pipeline was excellent (median = 1) for LGA [IQR = 0] and LPA [0.02]. LST algorithms showed moderate accuracy and good reproducibility. Therefore, it can be used as a reliable cross-sectional and longitudinal tool in multi-site studies.
Collapse
Affiliation(s)
- Federica Ribaldi
- Laboratory of Alzheimer's Neuroimaging and Alzheimer's Epidemiology, IRCCS Istituto Centro San Giovanni di Dio Fatebenefratelli, Brescia, Italy; Department of Molecular and Translational Medicine, University of Brescia, Brescia, Italy; Laboratory of Neuroimaging of Aging (LANVIE), University of Geneva, Geneva, Switzerland; Memory Clinic, Geneva University Hospitals, Geneva, Switzerland.
| | - Daniele Altomare
- Laboratory of Neuroimaging of Aging (LANVIE), University of Geneva, Geneva, Switzerland; Memory Clinic, Geneva University Hospitals, Geneva, Switzerland
| | - Jorge Jovicich
- Center for Mind/Brain Sciences (CIMEC), University of Trento, Rovereto, Italy
| | - Clarissa Ferrari
- Unit of Statistics, IRCCS Istituto Centro San Giovanni di Dio Fatebenefratelli, Brescia, Italy
| | - Agnese Picco
- Department of Neuroscience, Ophthalmology, Genetics and Mother-Child Health (DINOGMI), University of Genoa, Genoa, Italy
| | | | | | - Anna Mega
- Laboratory of Alzheimer's Neuroimaging and Alzheimer's Epidemiology, IRCCS Istituto Centro San Giovanni di Dio Fatebenefratelli, Brescia, Italy
| | - Antonio Ferretti
- Department of Neuroscience Imaging and Clinical Sciences, University "G. d'Annunzio" of Chieti, Italy; Institute for Advanced Biomedical Technologies (ITAB), University "G. d'Annunzio" of Chieti, Italy
| | - Antonios Drevelegas
- Interbalkan Medical Center of Thessaloniki, Thessaloniki, Greece; Department of Radiology, Aristotle University of Thessaloniki, Thessaloniki, Greece
| | - Beatriz Bosch
- Department of Psychiatry and Clinical Psychobiology, Universitat de Barcelona and IDIBAPS, Barcelona, Spain
| | - Bernhard W Müller
- LVR-Clinic for Psychiatry and Psychotherapy, Institutes and Clinics of the University Duisburg-Essen, Essen, Germany
| | - Camillo Marra
- Center for Neuropsychological Research, Catholic University, Rome, Italy
| | | | - David Bartrés-Faz
- Department of Psychiatry and Clinical Psychobiology, Universitat de Barcelona and IDIBAPS, Barcelona, Spain
| | - Flavio Nobili
- Dept. of Neuroscience (DINOGMI), University of Genoa, Italy; IRCCS Ospedale Policlinico San Martino Genova, Italy
| | - Franco Alessandrini
- Radiology, Dept. of Diagnostic and Public Health, Verona University, Verona, Italy
| | - Frederik Barkhof
- Centre for Medical Image Computing, Department of Medical Physics and Biomedical Engineering, University College London, London, UK; Department of Radiology and Nuclear Medicine, Amsterdam University Medical Centers, Amsterdam, the Netherlands
| | - Helene Gros-Dagnac
- ToNIC, Toulouse NeuroImaging Center, Université de Toulouse, Inserm, UPS, France; Université Toulouse 3 Paul Sabatier, UMR 825 Imagerie Cérébrale et Handicaps Neurologiques, F-31024 Toulouse, France
| | - Jean-Philippe Ranjeva
- Institut de Neurosciences de la Timone (INT), Aix-Marseille Université, CNRS, UMR 7289, 13005 Marseille, France
| | - Jens Wiltfang
- Department of Psychiatry and Psychotherapy, University Medical Center (UMG), Georg-August University, Göttingen, Germany
| | - Joost Kuijer
- Department of Radiology and Nuclear Medicine, Amsterdam University Medical Centers, Amsterdam, the Netherlands
| | - Julien Sein
- Institut de Neurosciences de la Timone (INT), Aix-Marseille Université, CNRS, UMR 7289, 13005 Marseille, France
| | | | - Luca Roccatagliata
- IRCCS Ospedale Policlinico San Martino Genova, Italy; Dept. of Health Sciences (DISSAL), University of Genoa, Italy
| | - Lucilla Parnetti
- Section of Neurology, Centre for Memory Disturbances, University of Perugia, Perugia, Italy
| | - Magda Tsolaki
- 1st Department of Neurology, Aristotle University of Thessaloniki, Makedonia, Greece
| | | | | | | | - Martina Montalti
- Laboratory of Alzheimer's Neuroimaging and Alzheimer's Epidemiology, IRCCS Istituto Centro San Giovanni di Dio Fatebenefratelli, Brescia, Italy
| | - Massimo Caulo
- Department of Neuroscience Imaging and Clinical Sciences, University "G. d'Annunzio" of Chieti, Italy; Institute for Advanced Biomedical Technologies (ITAB), University "G. d'Annunzio" of Chieti, Italy
| | - Mira Didic
- APHM, Timone, Service de Neurologie et Neuropsychologie, APHM Hôpital Timone Adultes, Marseille, France; Aix Marseille Univ, INSERM, INS, Inst Neurosci Syst, Marseille, France
| | - Núria Bargallo
- Department of Neuroradiology and Magnetic Resonance Image Core Facility, Hospital Clínic de Barcelona, IDIBAPS, Barcelona, Spain
| | - Olivier Blin
- Aix Marseille University, UMR-INSERM 1106, Service de Pharmacologie Clinique, AP-HM, Marseille, France
| | - Paolo M Rossini
- Dept. Neuroscience & Neurorehabilitation, IRCCS-San Raffaele-Pisana, Rome, Italy
| | - Peter Schonknecht
- Department of Psychiatry and Psychotherapy, University of Leipzig Medical Center, Leipzig, Germany
| | - Piero Floridi
- Neuroradiology Unit, Perugia General Hospital, Perugia, Italy
| | - Pierre Payoux
- ToNIC, Toulouse NeuroImaging Center, Université de Toulouse, Inserm, UPS, France
| | - Pieter Jelle Visser
- Alzheimer Center Amsterdam, Department of Neurology, Amsterdam Neuroscience, Vrije Universiteit Amsterdam, Amsterdam UMC, Amsterdam, The Netherlands
| | - Régis Bordet
- Univ. Lille, INSERM, CHU Lille, Lille Neuroscience & Cognition - Degenerative and Vascular Cognitive Disorders-U1172. F-59000 Lille, France
| | - Renaud Lopes
- Univ. Lille, INSERM, CHU Lille, Lille Neuroscience & Cognition - Degenerative and Vascular Cognitive Disorders-U1172. F-59000 Lille, France
| | | | - Stephanie Bombois
- Univ. Lille, INSERM, CHU Lille, Lille Neuroscience & Cognition - Degenerative and Vascular Cognitive Disorders-U1172. F-59000 Lille, France
| | - Tilman Hensch
- Department of Psychiatry and Psychotherapy, University of Leipzig Medical Center, Leipzig, Germany
| | - Ute Fiedler
- LVR-Clinic for Psychiatry and Psychotherapy, Institutes and Clinics of the University Duisburg-Essen, Essen, Germany
| | - Jill C Richardson
- Neurosciences Therapeutic Area, GlaxoSmithKline R&D, Gunnels Wood Road, Stevenage, United Kingdom
| | - Giovanni B Frisoni
- Laboratory of Neuroimaging of Aging (LANVIE), University of Geneva, Geneva, Switzerland; Memory Clinic, Geneva University Hospitals, Geneva, Switzerland
| | - Moira Marizzoni
- Laboratory of Alzheimer's Neuroimaging and Alzheimer's Epidemiology, IRCCS Istituto Centro San Giovanni di Dio Fatebenefratelli, Brescia, Italy
| |
Collapse
|
2
|
Boccardi M, Bocchetta M, Morency FC, Collins DL, Nishikawa M, Ganzola R, Grothe MJ, Wolf D, Redolfi A, Pievani M, Antelmi L, Fellgiebel A, Matsuda H, Teipel S, Duchesne S, Jack CR, Frisoni GB. Training labels for hippocampal segmentation based on the EADC-ADNI harmonized hippocampal protocol. Alzheimers Dement 2015; 11:175-83. [PMID: 25616957 DOI: 10.1016/j.jalz.2014.12.002] [Citation(s) in RCA: 68] [Impact Index Per Article: 7.6] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/07/2013] [Revised: 03/25/2014] [Accepted: 12/12/2014] [Indexed: 11/26/2022]
Abstract
BACKGROUND The European Alzheimer's Disease Consortium and Alzheimer's Disease Neuroimaging Initiative (ADNI) Harmonized Protocol (HarP) is a Delphi definition of manual hippocampal segmentation from magnetic resonance imaging (MRI) that can be used as the standard of truth to train new tracers, and to validate automated segmentation algorithms. Training requires large and representative data sets of segmented hippocampi. This work aims to produce a set of HarP labels for the proper training and certification of tracers and algorithms. METHODS Sixty-eight 1.5 T and 67 3 T volumetric structural ADNI scans from different subjects, balanced by age, medial temporal atrophy, and scanner manufacturer, were segmented by five qualified HarP tracers whose absolute interrater intraclass correlation coefficients were 0.953 and 0.975 (left and right). Labels were validated as HarP compliant through centralized quality check and correction. RESULTS Hippocampal volumes (mm(3)) were as follows: controls: left = 3060 (standard deviation [SD], 502), right = 3120 (SD, 897); mild cognitive impairment (MCI): left = 2596 (SD, 447), right = 2686 (SD, 473); and Alzheimer's disease (AD): left = 2301 (SD, 492), right = 2445 (SD, 525). Volumes significantly correlated with atrophy severity at Scheltens' scale (Spearman's ρ = <-0.468, P = <.0005). Cerebrospinal fluid spaces (mm(3)) were as follows: controls: left = 23 (32), right = 25 (25); MCI: left = 15 (13), right = 22 (16); and AD: left = 11 (13), right = 20 (25). Five subjects (3.7%) presented with unusual anatomy. CONCLUSIONS This work provides reference hippocampal labels for the training and certification of automated segmentation algorithms. The publicly released labels will allow the widespread implementation of the standard segmentation protocol.
Collapse
Affiliation(s)
- Marina Boccardi
- LENITEM (Laboratory of Epidemiology, Neuroimaging and Telemedicine) IRCCS - Centro S. Giovanni di Dio - Fatebenefratelli, Brescia, Italy.
| | - Martina Bocchetta
- LENITEM (Laboratory of Epidemiology, Neuroimaging and Telemedicine) IRCCS - Centro S. Giovanni di Dio - Fatebenefratelli, Brescia, Italy; Department of Molecular and Translational Medicine, University of Brescia, Brescia, Italy
| | - Félix C Morency
- Department of Radiology, Université Laval and Centre de Recherche de l'Institut universitaire de santé mentale de Québec, Quebec City, Canada; Imeka, Sherbrooke, Québec, Canada
| | - D Louis Collins
- McConnell Brain Imaging Center, Montreal Neurological Institute, McGill University, Montreal, Canada
| | | | - Rossana Ganzola
- Department of Radiology, Université Laval and Centre de Recherche de l'Institut universitaire de santé mentale de Québec, Quebec City, Canada
| | - Michel J Grothe
- German Center for Neurodegenerative Diseases (DZNE), Rostock, Germany
| | - Dominik Wolf
- Klinik für Psychiatrie und Psychotherapie, Johannes Gutenberg-Universität, Mainz, Germany
| | - Alberto Redolfi
- LENITEM (Laboratory of Epidemiology, Neuroimaging and Telemedicine) IRCCS - Centro S. Giovanni di Dio - Fatebenefratelli, Brescia, Italy
| | - Michela Pievani
- LENITEM (Laboratory of Epidemiology, Neuroimaging and Telemedicine) IRCCS - Centro S. Giovanni di Dio - Fatebenefratelli, Brescia, Italy
| | - Luigi Antelmi
- LENITEM (Laboratory of Epidemiology, Neuroimaging and Telemedicine) IRCCS - Centro S. Giovanni di Dio - Fatebenefratelli, Brescia, Italy; Laboratory of Neuroimaging of Aging, Department of Psychiatry, HUG Belle-Idée, Geneva, Switzerland
| | - Andreas Fellgiebel
- Klinik für Psychiatrie und Psychotherapie, Johannes Gutenberg-Universität, Mainz, Germany
| | - Hiroshi Matsuda
- Integrative Brain Imaging Center, National Center of Neurology and Psychiatry, Tokyo, Japan
| | - Stefan Teipel
- German Center for Neurodegenerative Diseases (DZNE), Rostock, Germany; Integrative Brain Imaging Center, National Center of Neurology and Psychiatry, Tokyo, Japan
| | - Simon Duchesne
- Department of Radiology, Université Laval and Centre de Recherche de l'Institut universitaire de santé mentale de Québec, Quebec City, Canada
| | - Clifford R Jack
- Department of Diagnostic Radiology, Mayo Clinic and Foundation, Rochester, MN, USA
| | - Giovanni B Frisoni
- LENITEM (Laboratory of Epidemiology, Neuroimaging and Telemedicine) IRCCS - Centro S. Giovanni di Dio - Fatebenefratelli, Brescia, Italy; Laboratory of Neuroimaging of Aging, Department of Psychiatry, HUG Belle-Idée, Geneva, Switzerland; University of Geneva, Faculty of Medicine, Geneva, Switzerland
| | | |
Collapse
|