1
|
Liu Z, Kainth K, Zhou A, Deyer TW, Fayad ZA, Greenspan H, Mei X. A review of self-supervised, generative, and few-shot deep learning methods for data-limited magnetic resonance imaging segmentation. NMR IN BIOMEDICINE 2024; 37:e5143. [PMID: 38523402 DOI: 10.1002/nbm.5143] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 06/01/2023] [Revised: 02/15/2024] [Accepted: 02/16/2024] [Indexed: 03/26/2024]
Abstract
Magnetic resonance imaging (MRI) is a ubiquitous medical imaging technology with applications in disease diagnostics, intervention, and treatment planning. Accurate MRI segmentation is critical for diagnosing abnormalities, monitoring diseases, and deciding on a course of treatment. With the advent of advanced deep learning frameworks, fully automated and accurate MRI segmentation is advancing. Traditional supervised deep learning techniques have advanced tremendously, reaching clinical-level accuracy in the field of segmentation. However, these algorithms still require a large amount of annotated data, which is oftentimes unavailable or impractical. One way to circumvent this issue is to utilize algorithms that exploit a limited amount of labeled data. This paper aims to review such state-of-the-art algorithms that use a limited number of annotated samples. We explain the fundamental principles of self-supervised learning, generative models, few-shot learning, and semi-supervised learning and summarize their applications in cardiac, abdomen, and brain MRI segmentation. Throughout this review, we highlight algorithms that can be employed based on the quantity of annotated data available. We also present a comprehensive list of notable publicly available MRI segmentation datasets. To conclude, we discuss possible future directions of the field-including emerging algorithms, such as contrastive language-image pretraining, and potential combinations across the methods discussed-that can further increase the efficacy of image segmentation with limited labels.
Collapse
Affiliation(s)
- Zelong Liu
- BioMedical Engineering and Imaging Institute, Icahn School of Medicine at Mount Sinai, New York, New York, USA
| | - Komal Kainth
- BioMedical Engineering and Imaging Institute, Icahn School of Medicine at Mount Sinai, New York, New York, USA
| | - Alexander Zhou
- BioMedical Engineering and Imaging Institute, Icahn School of Medicine at Mount Sinai, New York, New York, USA
| | - Timothy W Deyer
- East River Medical Imaging, New York, New York, USA
- Department of Radiology, Cornell Medicine, New York, New York, USA
| | - Zahi A Fayad
- BioMedical Engineering and Imaging Institute, Icahn School of Medicine at Mount Sinai, New York, New York, USA
- Department of Diagnostic, Molecular, and Interventional Radiology, Icahn School of Medicine at Mount Sinai, New York, New York, USA
| | - Hayit Greenspan
- BioMedical Engineering and Imaging Institute, Icahn School of Medicine at Mount Sinai, New York, New York, USA
- Department of Diagnostic, Molecular, and Interventional Radiology, Icahn School of Medicine at Mount Sinai, New York, New York, USA
| | - Xueyan Mei
- BioMedical Engineering and Imaging Institute, Icahn School of Medicine at Mount Sinai, New York, New York, USA
- Department of Diagnostic, Molecular, and Interventional Radiology, Icahn School of Medicine at Mount Sinai, New York, New York, USA
| |
Collapse
|
2
|
Zhang L, Ning G, Liang H, Han B, Liao H. One-shot neuroanatomy segmentation through online data augmentation and confidence aware pseudo label. Med Image Anal 2024; 95:103182. [PMID: 38688039 DOI: 10.1016/j.media.2024.103182] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/23/2022] [Revised: 11/26/2023] [Accepted: 04/18/2024] [Indexed: 05/02/2024]
Abstract
Recently, deep learning-based brain segmentation methods have achieved great success. However, most approaches focus on supervised segmentation, which requires many high-quality labeled images. In this paper, we pay attention to one-shot segmentation, aiming to learn from one labeled image and a few unlabeled images. We propose an end-to-end unified network that joints deformation modeling and segmentation tasks. Our network consists of a shared encoder, a deformation modeling head, and a segmentation head. In the training phase, the atlas and unlabeled images are input to the encoder to get multi-scale features. The features are then fed to the multi-scale deformation modeling module to estimate the atlas-to-image deformation field. The deformation modeling module implements the estimation at the feature level in a coarse-to-fine manner. Then, we employ the field to generate the augmented image pair through online data augmentation. We do not apply any appearance transformations cause the shared encoder could capture appearance variations. Finally, we adopt supervised segmentation loss for the augmented image. Considering that the unlabeled images still contain rich information, we introduce confidence aware pseudo label for them to further boost the segmentation performance. We validate our network on three benchmark datasets. Experimental results demonstrate that our network significantly outperforms other deep single-atlas-based and traditional multi-atlas-based segmentation methods. Notably, the second dataset is collected from multi-center, and our network still achieves promising segmentation performance on both the seen and unseen test sets, revealing its robustness. The source code will be available at https://github.com/zhangliutong/brainseg.
Collapse
Affiliation(s)
- Liutong Zhang
- Department of Biomedical Engineering, School of Medicine, Tsinghua University, Beijing, China
| | - Guochen Ning
- School of Clinical Medicine, Tsinghua University, Beijing, China
| | - Hanying Liang
- Department of Biomedical Engineering, School of Medicine, Tsinghua University, Beijing, China
| | - Boxuan Han
- Department of Biomedical Engineering, School of Medicine, Tsinghua University, Beijing, China
| | - Hongen Liao
- Department of Biomedical Engineering, School of Medicine, Tsinghua University, Beijing, China; School of Biomedical Engineering, Institute of Medical Robotics, Shanghai Jiao Tong University, Shanghai, China.
| |
Collapse
|
3
|
Torres-Simon L, Del Cerro-León A, Yus M, Bruña R, Gil-Martinez L, Dolado AM, Maestú F, Arrazola-Garcia J, Cuesta P. Decoding the best automated segmentation tools for vascular white matter hyperintensities in the aging brain: a clinician's guide to precision and purpose. GeroScience 2024:10.1007/s11357-024-01238-5. [PMID: 38869712 DOI: 10.1007/s11357-024-01238-5] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/09/2024] [Accepted: 06/04/2024] [Indexed: 06/14/2024] Open
Abstract
White matter hyperintensities of vascular origin (WMH) are commonly found in individuals over 60 and increase in prevalence with age. The significance of WMH is well-documented, with strong associations with cognitive impairment, risk of stroke, mental health, and brain structure deterioration. Consequently, careful monitoring is crucial for the early identification and management of individuals at risk. Luckily, WMH are detectable and quantifiable on standard MRI through visual assessment scales, but it is time-consuming and has high rater variability. Addressing this issue, the main aim of our study is to decipher the utility of quantitative measures of WMH, assessed with automatic tools, in establishing risk profiles for cerebrovascular deterioration. For this purpose, first, we work to determine the most precise WMH segmentation open access tool compared to clinician manual segmentations (LST-LPA, LST-LGA, SAMSEG, and BIANCA), offering insights into methodology and usability to balance clinical precision with practical application. The results indicated that supervised algorithms (LST-LPA and BIANCA) were superior, particularly in detecting small WMH, and can improve their consistency when used in parallel with unsupervised tools (LST-LGA and SAMSEG). Additionally, to investigate the behavior and real clinical utility of these tools, we tested them in a real-world scenario (N = 300; age > 50 y.o. and MMSE > 26), proposing an imaging biomarker for moderate vascular damage. The results confirmed its capacity to effectively identify individuals at risk comparing the cognitive and brain structural profiles of cognitively healthy adults above and below the resulted threshold.
Collapse
Affiliation(s)
- Lucia Torres-Simon
- Center of Cognitive and Computational Neuroscience, Universidad Complutense de Madrid (UCM), Madrid, Spain
- Department of Experimental Psychology, Cognitive Processes and Speech Therapy, Universidad Complutense de Madrid (UCM), Madrid, Spain
| | - Alberto Del Cerro-León
- Center of Cognitive and Computational Neuroscience, Universidad Complutense de Madrid (UCM), Madrid, Spain.
- Department of Experimental Psychology, Cognitive Processes and Speech Therapy, Universidad Complutense de Madrid (UCM), Madrid, Spain.
- Facultad de Psicología, Campus de Somosaguas, 28223, Pozuelo de Alarcón, Spain.
| | - Miguel Yus
- Health Research Institute of the Hospital Clínico San Carlos (IdISSC), 28040, Madrid, Spain
- Department of Diagnostic Imaging, Hospital Clínico San Carlos, 28040, Madrid, Spain
| | - Ricardo Bruña
- Center of Cognitive and Computational Neuroscience, Universidad Complutense de Madrid (UCM), Madrid, Spain
- Health Research Institute of the Hospital Clínico San Carlos (IdISSC), 28040, Madrid, Spain
- Department of Radiology, Complutense University of Madrid, 28040, Madrid, Spain
| | - Lidia Gil-Martinez
- Foundation for Biomedical Research at Hospital Clínico San Carlos (FIBHCSC), Hospital Clínico San Carlos, 28040, Madrid, Spain
| | - Alberto Marcos Dolado
- Health Research Institute of the Hospital Clínico San Carlos (IdISSC), 28040, Madrid, Spain
- Department of Medicine, School of Medicine, Complutense University of Madrid, 28040, Madrid, Spain
- Department of Neurology, Hospital Clínico San Carlos, 28040, Madrid, Spain
| | - Fernando Maestú
- Center of Cognitive and Computational Neuroscience, Universidad Complutense de Madrid (UCM), Madrid, Spain
- Department of Experimental Psychology, Cognitive Processes and Speech Therapy, Universidad Complutense de Madrid (UCM), Madrid, Spain
- Health Research Institute of the Hospital Clínico San Carlos (IdISSC), 28040, Madrid, Spain
| | - Juan Arrazola-Garcia
- Health Research Institute of the Hospital Clínico San Carlos (IdISSC), 28040, Madrid, Spain
- Department of Diagnostic Imaging, Hospital Clínico San Carlos, 28040, Madrid, Spain
- Department of Radiology, Rehabilitation and Radiation Therapy, School of Medicine, Complutense University of Madrid, 28040, Madrid, Spain
| | - Pablo Cuesta
- Center of Cognitive and Computational Neuroscience, Universidad Complutense de Madrid (UCM), Madrid, Spain
- Health Research Institute of the Hospital Clínico San Carlos (IdISSC), 28040, Madrid, Spain
- Department of Radiology, Complutense University of Madrid, 28040, Madrid, Spain
| |
Collapse
|
4
|
Bai L, Wang D, Wang H, Barnett M, Cabezas M, Cai W, Calamante F, Kyle K, Liu D, Ly L, Nguyen A, Shieh CC, Sullivan R, Zhan G, Ouyang W, Wang C. Improving multiple sclerosis lesion segmentation across clinical sites: A federated learning approach with noise-resilient training. Artif Intell Med 2024; 152:102872. [PMID: 38701636 DOI: 10.1016/j.artmed.2024.102872] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/26/2023] [Revised: 03/28/2024] [Accepted: 04/15/2024] [Indexed: 05/05/2024]
Abstract
Accurately measuring the evolution of Multiple Sclerosis (MS) with magnetic resonance imaging (MRI) critically informs understanding of disease progression and helps to direct therapeutic strategy. Deep learning models have shown promise for automatically segmenting MS lesions, but the scarcity of accurately annotated data hinders progress in this area. Obtaining sufficient data from a single clinical site is challenging and does not address the heterogeneous need for model robustness. Conversely, the collection of data from multiple sites introduces data privacy concerns and potential label noise due to varying annotation standards. To address this dilemma, we explore the use of the federated learning framework while considering label noise. Our approach enables collaboration among multiple clinical sites without compromising data privacy under a federated learning paradigm that incorporates a noise-robust training strategy based on label correction. Specifically, we introduce a Decoupled Hard Label Correction (DHLC) strategy that considers the imbalanced distribution and fuzzy boundaries of MS lesions, enabling the correction of false annotations based on prediction confidence. We also introduce a Centrally Enhanced Label Correction (CELC) strategy, which leverages the aggregated central model as a correction teacher for all sites, enhancing the reliability of the correction process. Extensive experiments conducted on two multi-site datasets demonstrate the effectiveness and robustness of our proposed methods, indicating their potential for clinical applications in multi-site collaborations to train better deep learning models with lower cost in data collection and annotation.
Collapse
Affiliation(s)
- Lei Bai
- Brain and Mind Centre, The University of Sydney, NSW 2050, Australia; School of Electrical and Information Engineering, The University of Sydney, NSW 2006, Australia
| | - Dongang Wang
- Brain and Mind Centre, The University of Sydney, NSW 2050, Australia; Sydney Neuroimaging Analysis Centre, 94 Mallett Street, NSW 2050, Australia.
| | - Hengrui Wang
- Sydney Neuroimaging Analysis Centre, 94 Mallett Street, NSW 2050, Australia
| | - Michael Barnett
- Brain and Mind Centre, The University of Sydney, NSW 2050, Australia; Sydney Neuroimaging Analysis Centre, 94 Mallett Street, NSW 2050, Australia; Royal Prince Alfred Hospital, NSW, 2050, Australia
| | - Mariano Cabezas
- Brain and Mind Centre, The University of Sydney, NSW 2050, Australia
| | - Weidong Cai
- Brain and Mind Centre, The University of Sydney, NSW 2050, Australia; School of Computer Science, The University of Sydney, NSW 2006, Australia
| | - Fernando Calamante
- Brain and Mind Centre, The University of Sydney, NSW 2050, Australia; School of Biomedical Engineering, The University of Sydney, NSW 2006, Australia; Sydney Imaging, The University of Sydney, NSW 2006, Australia
| | - Kain Kyle
- Brain and Mind Centre, The University of Sydney, NSW 2050, Australia; Sydney Neuroimaging Analysis Centre, 94 Mallett Street, NSW 2050, Australia
| | - Dongnan Liu
- Brain and Mind Centre, The University of Sydney, NSW 2050, Australia; School of Computer Science, The University of Sydney, NSW 2006, Australia
| | - Linda Ly
- Sydney Neuroimaging Analysis Centre, 94 Mallett Street, NSW 2050, Australia
| | - Aria Nguyen
- Sydney Neuroimaging Analysis Centre, 94 Mallett Street, NSW 2050, Australia
| | - Chun-Chien Shieh
- Sydney Neuroimaging Analysis Centre, 94 Mallett Street, NSW 2050, Australia
| | - Ryan Sullivan
- School of Biomedical Engineering, The University of Sydney, NSW 2006, Australia; Australian Imaging Service, NSW 2006, Australia
| | - Geng Zhan
- Brain and Mind Centre, The University of Sydney, NSW 2050, Australia; Sydney Neuroimaging Analysis Centre, 94 Mallett Street, NSW 2050, Australia
| | - Wanli Ouyang
- School of Electrical and Information Engineering, The University of Sydney, NSW 2006, Australia
| | - Chenyu Wang
- Brain and Mind Centre, The University of Sydney, NSW 2050, Australia; Sydney Neuroimaging Analysis Centre, 94 Mallett Street, NSW 2050, Australia.
| |
Collapse
|
5
|
Torres-Simon L, Del Cerro-León A, Yus M, Bruña R, Gil-Martinez L, Marcos Dolado A, Maestú F, Arrazola-Garcia J, Cuesta P. Decoding the Best Automated Segmentation Tools for Vascular White Matter Hyperintensities in the Aging Brain: A Clinician's Guide to Precision and Purpose. MEDRXIV : THE PREPRINT SERVER FOR HEALTH SCIENCES 2024:2023.03.30.23287946. [PMID: 38798616 PMCID: PMC11118558 DOI: 10.1101/2023.03.30.23287946] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 05/29/2024]
Abstract
Cerebrovascular damage from small vessel disease (SVD) occurs in healthy and pathological aging. SVD markers, such as white matter hyperintensities (WMH), are commonly found in individuals over 60 and increase in prevalence with age. WMHs are detectable on standard MRI by adhering to the STRIVE criteria. Currently, visual assessment scales are used in clinical and research scenarios but is time-consuming and has rater variability, limiting its practicality. Addressing this issue, our study aimed to determine the most precise WMH segmentation software, offering insights into methodology and usability to balance clinical precision with practical application. This study employed a dataset comprising T1, FLAIR, and DWI images from 300 cognitively healthy older adults. WMHs in this cohort were evaluated using four automated neuroimaging tools: Lesion Prediction Algorithm (LPA) and Lesion Growth Algorithm (LGA) from Lesion Segmentation Tool (LST), Sequence Adaptive Multimodal Segmentation (SAMSEG), and Brain Intensity Abnormalities Classification Algorithm (BIANCA). Additionally, clinicians manually segmented WMHs in a subsample of 45 participants to establish a gold standard. The study assessed correlations with the Fazekas scale, algorithm performance, and the influence of WMH volume on reliability. Results indicated that supervised algorithms were superior, particularly in detecting small WMHs, and can improve their consistency when used in parallel with unsupervised tools. The research also proposed a biomarker for moderate vascular damage, derived from the top 95th percentile of WMH volume in healthy individuals aged 50 to 60. This biomarker effectively differentiated subgroups within the cohort, correlating with variations in brain structure and behavior.
Collapse
|
6
|
Chaves H, Serra MM, Shalom DE, Ananía P, Rueda F, Osa Sanz E, Stefanoff NI, Rodríguez Murúa S, Costa ME, Kitamura FC, Yañez P, Cejas C, Correale J, Ferrante E, Fernández Slezak D, Farez MF. Assessing robustness and generalization of a deep neural network for brain MS lesion segmentation on real-world data. Eur Radiol 2024; 34:2024-2035. [PMID: 37650967 DOI: 10.1007/s00330-023-10093-5] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/03/2023] [Revised: 07/01/2023] [Accepted: 07/12/2023] [Indexed: 09/01/2023]
Abstract
OBJECTIVES Evaluate the performance of a deep learning (DL)-based model for multiple sclerosis (MS) lesion segmentation and compare it to other DL and non-DL algorithms. METHODS This ambispective, multicenter study assessed the performance of a DL-based model for MS lesion segmentation and compared it to alternative DL- and non-DL-based methods. Models were tested on internal (n = 20) and external (n = 18) datasets from Latin America, and on an external dataset from Europe (n = 49). We also examined robustness by rescanning six patients (n = 6) from our MS clinical cohort. Moreover, we studied inter-human annotator agreement and discussed our findings in light of these results. Performance and robustness were assessed using intraclass correlation coefficient (ICC), Dice coefficient (DC), and coefficient of variation (CV). RESULTS Inter-human ICC ranged from 0.89 to 0.95, while spatial agreement among annotators showed a median DC of 0.63. Using expert manual segmentations as ground truth, our DL model achieved a median DC of 0.73 on the internal, 0.66 on the external, and 0.70 on the challenge datasets. The performance of our DL model exceeded that of the alternative algorithms on all datasets. In the robustness experiment, our DL model also achieved higher DC (ranging from 0.82 to 0.90) and lower CV (ranging from 0.7 to 7.9%) when compared to the alternative methods. CONCLUSION Our DL-based model outperformed alternative methods for brain MS lesion segmentation. The model also proved to generalize well on unseen data and has a robust performance and low processing times both on real-world and challenge-based data. CLINICAL RELEVANCE STATEMENT Our DL-based model demonstrated superior performance in accurately segmenting brain MS lesions compared to alternative methods, indicating its potential for clinical application with improved accuracy, robustness, and efficiency. KEY POINTS • Automated lesion load quantification in MS patients is valuable; however, more accurate methods are still necessary. • A novel deep learning model outperformed alternative MS lesion segmentation methods on multisite datasets. • Deep learning models are particularly suitable for MS lesion segmentation in clinical scenarios.
Collapse
Affiliation(s)
- Hernán Chaves
- Diagnostic Imaging Department, Fleni, Montañeses, 2325 (C1428AQK), Ciudad de Buenos Aires, Argentina.
| | - María M Serra
- Diagnostic Imaging Department, Fleni, Montañeses, 2325 (C1428AQK), Ciudad de Buenos Aires, Argentina
| | - Diego E Shalom
- Department of Physics, University of Buenos Aires (UBA), Buenos Aires, Argentina
- Physics Institute of Buenos Aires (IFIBA) CONICET, Buenos Aires, Argentina
- Laboratorio de Neurociencia, Universidad Torcuato Di Tella, Buenos Aires, Argentina
| | | | - Fernanda Rueda
- Radiology Department, Diagnósticos da América SA (Dasa), Rio de Janeiro, Brazil
| | - Emilia Osa Sanz
- Diagnostic Imaging Department, Fleni, Montañeses, 2325 (C1428AQK), Ciudad de Buenos Aires, Argentina
| | - Nadia I Stefanoff
- Diagnostic Imaging Department, Fleni, Montañeses, 2325 (C1428AQK), Ciudad de Buenos Aires, Argentina
| | - Sofía Rodríguez Murúa
- Center for Research On Neuroimmunological Diseases (CIEN), Fleni, Buenos Aires, Argentina
| | | | - Felipe C Kitamura
- DasaInova, Diagnósticos da América SA (Dasa), São Paulo, São Paulo, Brazil
| | - Paulina Yañez
- Diagnostic Imaging Department, Fleni, Montañeses, 2325 (C1428AQK), Ciudad de Buenos Aires, Argentina
| | - Claudia Cejas
- Diagnostic Imaging Department, Fleni, Montañeses, 2325 (C1428AQK), Ciudad de Buenos Aires, Argentina
| | | | - Enzo Ferrante
- Instituto de Investigación en Señales, Sistemas e Inteligencia Computacional, sinc(i) CONICET-UNL, Santa Fe, Argentina
| | - Diego Fernández Slezak
- Center for Research On Neuroimmunological Diseases (CIEN), Fleni, Buenos Aires, Argentina
- Departamento de Computación, Facultad de Ciencias Exactas y Naturales, Universidad de Buenos Aires (UBA), Buenos Aires, Argentina
- Instituto de Investigación en Ciencias de la Computación (ICC), CONICET-UBA, Buenos Aires, Argentina
| | - Mauricio F Farez
- Radiology Department, Diagnósticos da América SA (Dasa), Rio de Janeiro, Brazil
- Center for Research On Neuroimmunological Diseases (CIEN), Fleni, Buenos Aires, Argentina
- Center for Biostatistics, Epidemiology and Public Health (CEBES), Fleni, Buenos Aires, Argentina
| |
Collapse
|
7
|
Calvi A, Mendelsohn Z, Hamed W, Chard D, Tur C, Stutters J, MacManus D, Kanber B, Wheeler-Kingshott CAMG, Barkhof F, Prados F. Treatment reduces the incidence of newly appearing multiple sclerosis lesions evolving into chronic active, slowly expanding lesions: A retrospective analysis. Eur J Neurol 2024; 31:e16092. [PMID: 37823722 DOI: 10.1111/ene.16092] [Citation(s) in RCA: 4] [Impact Index Per Article: 4.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/31/2023] [Revised: 09/05/2023] [Accepted: 09/21/2023] [Indexed: 10/13/2023]
Abstract
BACKGROUND AND PURPOSE Newly appearing lesions in multiple sclerosis (MS) may evolve into chronically active, slowly expanding lesions (SELs), leading to sustained disability progression. The aim of this study was to evaluate the incidence of newly appearing lesions developing into SELs, and their correlation to clinical evolution and treatment. METHODS A retrospective analysis of a fingolimod trial in primary progressive MS (PPMS; INFORMS, NCT00731692) was undertaken. Data were available from 324 patients with magnetic resonance imaging scans up to 3 years after screening. New lesions at year 1 were identified with convolutional neural networks, and SELs obtained through a deformation-based method. Clinical disability was assessed annually by Expanded Disability Status Scale (EDSS), Nine-Hole Peg Test, Timed 25-Foot Walk, and Paced Auditory Serial Addition Test. Linear, logistic, and mixed-effect models were used to assess the relationship between the Jacobian expansion in new lesions and SELs, disability scores, and treatment status. RESULTS One hundred seventy patients had ≥1 new lesions at year 1 and had a higher lesion count at screening compared to patients with no new lesions (median = 27 vs. 22, p = 0.007). Among the new lesions (median = 2 per patient), 37% evolved into definite or possible SELs. Higher SEL volume and count were associated with EDSS worsening and confirmed disability progression. Treated patients had lower volume and count of definite SELs (β = -0.04, 95% confidence interval [CI] = -0.07 to -0.01, p = 0.015; β = -0.36, 95% CI = -0.67 to -0.06, p = 0.019, respectively). CONCLUSIONS Incident chronic active lesions are common in PPMS, and fingolimod treatment can reduce their number.
Collapse
Affiliation(s)
- Alberto Calvi
- NMR Research Unit, Institute of Neurology, University College London, London, UK
- Laboratory of Advanced Imaging in Neuroimmunological Diseases, Hospital Clinic Barcelona, Fundació Clinic per a la Recerca Biomèdica, Barcelona, Spain
| | - Zoe Mendelsohn
- NMR Research Unit, Institute of Neurology, University College London, London, UK
- Department of Radiology, Charité School of Medicine and University Hospital Berlin, Berlin, Germany
| | - Weaam Hamed
- NMR Research Unit, Institute of Neurology, University College London, London, UK
- Department of Radiology, Mansoura University Hospital, Mansoura, Egypt
| | - Declan Chard
- NMR Research Unit, Institute of Neurology, University College London, London, UK
- National Institute for Health Research, Biomedical Research Centre, University College London Hospitals, London, UK
| | - Carmen Tur
- NMR Research Unit, Institute of Neurology, University College London, London, UK
- Neurology-Neuroimmunology Department, Multiple Sclerosis Centre of Catalonia, Vall d'Hebron Barcelona Hospital Campus, Barcelona, Spain
| | - Jon Stutters
- NMR Research Unit, Institute of Neurology, University College London, London, UK
| | - David MacManus
- NMR Research Unit, Institute of Neurology, University College London, London, UK
| | - Baris Kanber
- National Institute for Health Research, Biomedical Research Centre, University College London Hospitals, London, UK
- Department of Medical Physics and Biomedical Engineering, Centre for Medical Image Computing, University College London, London, UK
| | - Claudia A M Gandini Wheeler-Kingshott
- NMR Research Unit, Institute of Neurology, University College London, London, UK
- Department of Brain and Behavioral Sciences, University of Pavia, Pavia, Italy
| | - Frederik Barkhof
- NMR Research Unit, Institute of Neurology, University College London, London, UK
- Department of Medical Physics and Biomedical Engineering, Centre for Medical Image Computing, University College London, London, UK
- Radiology and Nuclear Medicine, Amsterdam University Medical Centers (UMC), Vrije Universiteit, Amsterdam, the Netherlands
| | - Ferran Prados
- NMR Research Unit, Institute of Neurology, University College London, London, UK
- Department of Medical Physics and Biomedical Engineering, Centre for Medical Image Computing, University College London, London, UK
- e-Health Centre, Universitat Oberta de Catalunya, Barcelona, Spain
| |
Collapse
|
8
|
Gentile G, Jenkinson M, Griffanti L, Luchetti L, Leoncini M, Inderyas M, Mortilla M, Cortese R, De Stefano N, Battaglini M. BIANCA-MS: An optimized tool for automated multiple sclerosis lesion segmentation. Hum Brain Mapp 2023; 44:4893-4913. [PMID: 37530598 PMCID: PMC10472913 DOI: 10.1002/hbm.26424] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/26/2022] [Revised: 05/20/2023] [Accepted: 07/06/2023] [Indexed: 08/03/2023] Open
Abstract
In this work we present BIANCA-MS, a novel tool for brain white matter lesion segmentation in multiple sclerosis (MS), able to generalize across both the wide spectrum of MRI acquisition protocols and the heterogeneity of manually labeled data. BIANCA-MS is based on the original version of BIANCA and implements two innovative elements: a harmonized setting, tested under different MRI protocols, which avoids the need to further tune algorithm parameters to each dataset; and a cleaning step developed to improve consistency in automated and manual segmentations, thus reducing unwanted variability in output segmentations and validation data. BIANCA-MS was tested on three datasets, acquired with different MRI protocols. First, we compared BIANCA-MS to other widely used tools. Second, we tested how BIANCA-MS performs in separate datasets. Finally, we evaluated BIANCA-MS performance on a pooled dataset where all MRI data were merged. We calculated the overlap using the DICE spatial similarity index (SI) as well as the number of false positive/negative clusters (nFPC/nFNC) in comparison to the manual masks processed with the cleaning step. BIANCA-MS clearly outperformed other available tools in both high- and low-resolution images and provided comparable performance across different scanning protocols, sets of modalities and image resolutions. BIANCA-MS performance on the pooled dataset (SI: 0.72 ± 0.25, nFPC: 13 ± 11, nFNC: 4 ± 8) were comparable to those achieved on each individual dataset (median across datasets SI: 0.72 ± 0.28, nFPC: 14 ± 11, nFNC: 4 ± 8). Our findings suggest that BIANCA-MS is a robust and accurate approach for automated MS lesion segmentation.
Collapse
Affiliation(s)
- Giordano Gentile
- Department of Medicine, Surgery and NeuroscienceUniversity of SienaSienaItaly
- SIENA Imaging SRLSienaItaly
| | - Mark Jenkinson
- Welcome Centre for Integrative Neuroimaging (WIN), FMRIB, Nuffield Department of Clinical NeurosciencesUniversity of Oxford, John Radcliffe HospitalOxfordUK
- Australian Institute of Machine Learning (AIML), School of Computer and Mathematical SciencesUniversity of AdelaideAdelaideSouth AustraliaAustralia
- South Australian Health and Medical Research Institute (SAHMRI)AdelaideSouth AustraliaAustralia
| | - Ludovica Griffanti
- Welcome Centre for Integrative Neuroimaging (WIN), FMRIB, Nuffield Department of Clinical NeurosciencesUniversity of Oxford, John Radcliffe HospitalOxfordUK
- Welcome Centre for Integrative Neuroimaging (WIN), OHBA, Department of PsychiatryUniversity of Oxford, Warneford HospitalOxfordUK
| | - Ludovico Luchetti
- Department of Medicine, Surgery and NeuroscienceUniversity of SienaSienaItaly
| | - Matteo Leoncini
- Department of Medicine, Surgery and NeuroscienceUniversity of SienaSienaItaly
- SIENA Imaging SRLSienaItaly
| | - Maira Inderyas
- Department of Medicine, Surgery and NeuroscienceUniversity of SienaSienaItaly
- SIENA Imaging SRLSienaItaly
| | | | - Rosa Cortese
- Department of Medicine, Surgery and NeuroscienceUniversity of SienaSienaItaly
| | - Nicola De Stefano
- Department of Medicine, Surgery and NeuroscienceUniversity of SienaSienaItaly
| | - Marco Battaglini
- Department of Medicine, Surgery and NeuroscienceUniversity of SienaSienaItaly
- SIENA Imaging SRLSienaItaly
| |
Collapse
|
9
|
Wenger KJ, Hoelter MC, Yalachkov Y, Hendrik Schäfer J, Özkan D, Steffen F, Bittner S, Hattingen E, Foerch C, Schaller-Paule MA. Serum neurofilament light chain is more strongly associated with T2 lesion volume than with number of T2 lesions in patients with multiple sclerosis. Eur J Radiol 2023; 166:111019. [PMID: 37549559 DOI: 10.1016/j.ejrad.2023.111019] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/11/2023] [Revised: 03/24/2023] [Accepted: 07/28/2023] [Indexed: 08/09/2023]
Abstract
BACKGROUND AND PURPOSE MR imaging provides information on the number and extend of focal lesions in multiple sclerosis (MS) patients. This study explores whether total brain T2 lesion volume or lesion number shows a better correlation with serum and cerebrospinal fluid (CSF) biomarkers of disease activity. MATERIALS AND METHODS In total, 52 patients suffering from clinically isolated syndrome (CIS)/relapsing-remitting multiple sclerosis (RRMS) were assessed including MRI markers (total brain T2 lesion volume semi-automatically outlined on 3D DIR/FLAIR sequences, number of lesions), serum and CSF biomarkers at the time of neuroimaging (neurofilament light chain (NfL), glial fibrillary acidic protein (GFAP)), and clinical parameters. After log-transformation and partial correlations adjusted for the covariates patients' age, BMI, EDSS-score and diagnosis, the Fisher's r-to-Z transformation was used to compare different correlation coefficients. RESULTS The correlation between lesion volume and serum NfL (r = 0.6, p < 0.001) was stronger compared to the association between the number of T2 lesions and serum NfL (r = 0.4, p < 0.01) (z = -2.0, p < 0.05). With regard to CSF NfL, there was a moderate, positive relationship for both number of T2 lesions and lesion volume (r = 0.5 respectively, p < 0.01). We found no significant association between MRI markers and GFAP levels. CONCLUSION Our findings suggest that there is a stronger association between serum NfL and T2 lesion volume, than there is between serum NfL and T2 lesion number. Improving robustness and accuracy of fully-automated lesion volume segmentation tools can expedite implementation into clinical routine and trials.
Collapse
Affiliation(s)
- Katharina J Wenger
- Goethe University Frankfurt, University Hospital, Institute of Neuroradiology, Germany.
| | - Maya C Hoelter
- Goethe University Frankfurt, University Hospital, Institute of Neuroradiology, Germany
| | - Yavor Yalachkov
- Goethe University Frankfurt, University Hospital, Department of Neurology, Germany
| | - Jan Hendrik Schäfer
- Goethe University Frankfurt, University Hospital, Department of Neurology, Germany
| | - Dilek Özkan
- Goethe University Frankfurt, University Hospital, Institute of Neuroradiology, Germany
| | - Falk Steffen
- Department of Neurology, University Medical Center of the Johannes Gutenberg-University Mainz, Mainz, Germany
| | - Stefan Bittner
- Department of Neurology, University Medical Center of the Johannes Gutenberg-University Mainz, Mainz, Germany
| | - Elke Hattingen
- Goethe University Frankfurt, University Hospital, Institute of Neuroradiology, Germany
| | - Christian Foerch
- Goethe University Frankfurt, University Hospital, Department of Neurology, Germany
| | - Martin A Schaller-Paule
- Goethe University Frankfurt, University Hospital, Department of Neurology, Germany; Department of Psychiatry and Psychotherapy, University Medical Center of the Johannes Gutenberg-University Mainz, Mainz, Germany
| |
Collapse
|
10
|
Chacko A, Schoeman S, Venkatakrishna SSB, Bolton S, Shearn AIU, Andronikou S. Caution: shortcomings of traditional segmentation methods from magnetic resonance imaging brain scans intended for 3-dimensional surface modelling in children with pathology. Pediatr Radiol 2023; 53:1854-1862. [PMID: 37249622 PMCID: PMC10421760 DOI: 10.1007/s00247-023-05692-9] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 12/20/2022] [Revised: 05/08/2023] [Accepted: 05/09/2023] [Indexed: 05/31/2023]
Abstract
This technical innovation assesses the adaptability of some common automated segmentation tools on abnormal pediatric magnetic resonance (MR) brain scans. We categorized 35 MR scans by pathologic features: (1) "normal"; (2) "atrophy"; (3) "cavity"; (4) "other." The following three tools, (1) Computational Anatomy Toolbox version 12 (CAT12); (2) Statistical Parametic Mapping version 12 (SPM12); and (3) MRTool, were tested on each scan-with default and adjusted settings. Success was determined by radiologist consensus on the surface accuracy. Automated segmentation failed in scans demonstrating severe surface brain pathology. Segmentation of the "cavity" group was ineffective, with success rates of 23.1% (CAT12), 69.2% (SPM12) and 46.2% (MRTool), even with refined settings and manual edits. Further investigation is required to improve this workflow and automated segmentation methodology for complex surface pathology.
Collapse
Affiliation(s)
- Anith Chacko
- CRICBristol, 60 St Michael's Hill, Faculty of Health Sciences, University of Bristol, Bristol, BS28DX, UK.
| | - Sean Schoeman
- Children's Hospital of Philadelphia, Philadelphia, PA, USA
| | | | - Samuel Bolton
- CRICBristol, 60 St Michael's Hill, Faculty of Health Sciences, University of Bristol, Bristol, BS28DX, UK
| | - Andrew I U Shearn
- CRICBristol, 60 St Michael's Hill, Faculty of Health Sciences, University of Bristol, Bristol, BS28DX, UK
| | - Savvas Andronikou
- Children's Hospital of Philadelphia, Philadelphia, PA, USA
- Perelman School of Medicine, University of Pennsylvania, Philadelphia, PA, USA
| |
Collapse
|
11
|
Lv JJ, Chen HY, Li JW, Lin KH, Chen RJ, Wang LJ, Zeng XX, Ren JC, Zhao HM. Contour extraction of medical images using an attention-based network. Biomed Signal Process Control 2023. [DOI: 10.1016/j.bspc.2023.104828] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 03/17/2023]
|
12
|
Zhang X, Liu C, Ou N, Zeng X, Zhuo Z, Duan Y, Xiong X, Yu Y, Liu Z, Liu Y, Ye C. CarveMix: A simple data augmentation method for brain lesion segmentation. Neuroimage 2023; 271:120041. [PMID: 36933626 DOI: 10.1016/j.neuroimage.2023.120041] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/20/2023] [Revised: 03/01/2023] [Accepted: 03/15/2023] [Indexed: 03/18/2023] Open
Abstract
Brain lesion segmentation provides a valuable tool for clinical diagnosis and research, and convolutional neural networks (CNNs) have achieved unprecedented success in the segmentation task. Data augmentation is a widely used strategy to improve the training of CNNs. In particular, data augmentation approaches that mix pairs of annotated training images have been developed. These methods are easy to implement and have achieved promising results in various image processing tasks. However, existing data augmentation approaches based on image mixing are not designed for brain lesions and may not perform well for brain lesion segmentation. Thus, the design of this type of simple data augmentation method for brain lesion segmentation is still an open problem. In this work, we propose a simple yet effective data augmentation approach, dubbed as CarveMix, for CNN-based brain lesion segmentation. Like other mixing-based methods, CarveMix stochastically combines two existing annotated images (annotated for brain lesions only) to obtain new labeled samples. To make our method more suitable for brain lesion segmentation, CarveMix is lesion-aware, where the image combination is performed with a focus on the lesions and preserves the lesion information. Specifically, from one annotated image we carve a region of interest (ROI) according to the lesion location and geometry with a variable ROI size. The carved ROI then replaces the corresponding voxels in a second annotated image to synthesize new labeled images for network training, and additional harmonization steps are applied for heterogeneous data where the two annotated images can originate from different sources. Besides, we further propose to model the mass effect that is unique to whole brain tumor segmentation during image mixing. To evaluate the proposed method, experiments were performed on multiple publicly available or private datasets, and the results show that our method improves the accuracy of brain lesion segmentation. The code of the proposed method is available at https://github.com/ZhangxinruBIT/CarveMix.git.
Collapse
Affiliation(s)
- Xinru Zhang
- School of Integrated Circuits and Electronics, Beijing Institute of Technology, Beijing, China
| | - Chenghao Liu
- School of Integrated Circuits and Electronics, Beijing Institute of Technology, Beijing, China
| | - Ni Ou
- School of Automation, Beijing Institute of Technology, Beijing, China
| | - Xiangzhu Zeng
- Department of Radiology, Peking University Third Hospital, Beijing, China
| | - Zhizheng Zhuo
- Department of Radiology, Beijing Tiantan Hospital, Capital Medical University, Beijing, China
| | - Yunyun Duan
- Department of Radiology, Beijing Tiantan Hospital, Capital Medical University, Beijing, China
| | | | | | - Zhiwen Liu
- School of Integrated Circuits and Electronics, Beijing Institute of Technology, Beijing, China.
| | - Yaou Liu
- Department of Radiology, Beijing Tiantan Hospital, Capital Medical University, Beijing, China.
| | - Chuyang Ye
- School of Integrated Circuits and Electronics, Beijing Institute of Technology, Beijing, China.
| |
Collapse
|
13
|
Commowick O, Combès B, Cervenansky F, Dojat M. Editorial: Automatic methods for multiple sclerosis new lesions detection and segmentation. Front Neurosci 2023; 17:1176625. [PMID: 36998735 PMCID: PMC10043498 DOI: 10.3389/fnins.2023.1176625] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/28/2023] [Accepted: 03/01/2023] [Indexed: 03/15/2023] Open
Affiliation(s)
- Olivier Commowick
- Empenn INSERM U1228, CNRS UMR6074, Inria, University of Rennes I, Rennes, France
| | - Benoît Combès
- Empenn INSERM U1228, CNRS UMR6074, Inria, University of Rennes I, Rennes, France
| | - Frédéric Cervenansky
- Univ Lyon, INSA-Lyon, Université Claude Bernard Lyon 1, UJM-Saint Etienne, CNRS, Inserm, CREATIS UMR 5220, U1206, Lyon, France
| | - Michel Dojat
- Univ Grenoble Alpes, Inserm, U1216, CHU Grenoble Alpes, Grenoble Institut Neurosciences, GIN, Grenoble, France
- *Correspondence: Michel Dojat
| |
Collapse
|
14
|
Mendelsohn Z, Pemberton HG, Gray J, Goodkin O, Carrasco FP, Scheel M, Nawabi J, Barkhof F. Commercial volumetric MRI reporting tools in multiple sclerosis: a systematic review of the evidence. Neuroradiology 2023; 65:5-24. [PMID: 36331588 PMCID: PMC9816195 DOI: 10.1007/s00234-022-03074-w] [Citation(s) in RCA: 5] [Impact Index Per Article: 5.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/13/2022] [Accepted: 09/29/2022] [Indexed: 11/06/2022]
Abstract
PURPOSE MRI is integral to the diagnosis of multiple sclerosis (MS) and is important for clinical prognostication. Quantitative volumetric reporting tools (QReports) can improve the accuracy and objectivity of MRI-based assessments. Several QReports are commercially available; however, validation can be difficult to establish and does not currently follow a common pathway. To aid evidence-based clinical decision-making, we performed a systematic review of commercial QReports for use in MS including technical details and published reports of validation and in-use evaluation. METHODS We categorized studies into three types of testing: technical validation, for example, comparison to manual segmentation, clinical validation by clinicians or interpretation of results alongside clinician-rated variables, and in-use evaluation, such as health economic assessment. RESULTS We identified 10 companies, which provide MS lesion and brain segmentation and volume quantification, and 38 relevant publications. Tools received regulatory approval between 2006 and 2020, contextualize results to normative reference populations, ranging from 620 to 8000 subjects, and require T1- and T2-FLAIR-weighted input sequences for longitudinal assessment of whole-brain volume and lesions. In MS, six QReports provided evidence of technical validation, four companies have conducted clinical validation by correlating results with clinical variables, only one has tested their QReport by clinician end-users, and one has performed a simulated in-use socioeconomic evaluation. CONCLUSION We conclude that there is limited evidence in the literature regarding clinical validation and in-use evaluation of commercial MS QReports with a particular lack of clinician end-user testing. Our systematic review provides clinicians and institutions with the available evidence when considering adopting a quantitative reporting tool for MS.
Collapse
Affiliation(s)
- Zoe Mendelsohn
- grid.83440.3b0000000121901201Neuroradiological Academic Unit, UCL Queen Square Institute of Neurology, University College London, London, UK ,grid.83440.3b0000000121901201Department of Medical Physics and Bioengineering, Centre for Medical Image Computing (CMIC), University College London, London, UK ,grid.83440.3b0000000121901201Department of Neuroinflammation, Queen Square Multiple Sclerosis Centre, UCL Institute of Neurology, University College London, London, UK ,grid.6363.00000 0001 2218 4662Department of Neuroradiology, Charité School of Medicine and University Hospital Berlin, Berlin, Germany ,grid.6363.00000 0001 2218 4662Department of Radiology, Charité School of Medicine and University Hospital Berlin, Berlin, Germany
| | - Hugh G. Pemberton
- grid.83440.3b0000000121901201Neuroradiological Academic Unit, UCL Queen Square Institute of Neurology, University College London, London, UK ,grid.83440.3b0000000121901201Department of Medical Physics and Bioengineering, Centre for Medical Image Computing (CMIC), University College London, London, UK ,grid.420685.d0000 0001 1940 6527GE Healthcare, Amersham, UK
| | - James Gray
- grid.416626.10000 0004 0391 2793Stepping Hill Hospital, NHS Foundation Trust, Stockport, UK
| | - Olivia Goodkin
- grid.83440.3b0000000121901201Neuroradiological Academic Unit, UCL Queen Square Institute of Neurology, University College London, London, UK ,grid.83440.3b0000000121901201Department of Medical Physics and Bioengineering, Centre for Medical Image Computing (CMIC), University College London, London, UK ,grid.83440.3b0000000121901201Department of Neuroinflammation, Queen Square Multiple Sclerosis Centre, UCL Institute of Neurology, University College London, London, UK
| | - Ferran Prados Carrasco
- grid.83440.3b0000000121901201Department of Medical Physics and Bioengineering, Centre for Medical Image Computing (CMIC), University College London, London, UK ,grid.83440.3b0000000121901201Department of Neuroinflammation, Queen Square Multiple Sclerosis Centre, UCL Institute of Neurology, University College London, London, UK ,grid.36083.3e0000 0001 2171 6620E-Health Centre, Universitat Oberta de Catalunya, Barcelona, Spain
| | - Michael Scheel
- grid.6363.00000 0001 2218 4662Department of Neuroradiology, Charité School of Medicine and University Hospital Berlin, Berlin, Germany
| | - Jawed Nawabi
- grid.6363.00000 0001 2218 4662Department of Radiology, Charité School of Medicine and University Hospital Berlin, Berlin, Germany ,grid.484013.a0000 0004 6879 971XBerlin Institute of Health at Charité – Universitätsmedizin Berlin, BIH Biomedical Innovation Academy, BIH Charité Digital Clinician Scientist Program, Berlin, Germany
| | - Frederik Barkhof
- grid.83440.3b0000000121901201Neuroradiological Academic Unit, UCL Queen Square Institute of Neurology, University College London, London, UK ,grid.83440.3b0000000121901201Department of Medical Physics and Bioengineering, Centre for Medical Image Computing (CMIC), University College London, London, UK ,grid.83440.3b0000000121901201Department of Neuroinflammation, Queen Square Multiple Sclerosis Centre, UCL Institute of Neurology, University College London, London, UK ,grid.12380.380000 0004 1754 9227Radiology & Nuclear Medicine, Amsterdam University Medical Centers, Vrije Universiteit, Amsterdam, The Netherlands
| |
Collapse
|
15
|
Hitziger S, Ling WX, Fritz T, D'Albis T, Lemke A, Grilo J. Triplanar U-Net with lesion-wise voting for the segmentation of new lesions on longitudinal MRI studies. Front Neurosci 2022; 16:964250. [PMID: 36033604 PMCID: PMC9412001 DOI: 10.3389/fnins.2022.964250] [Citation(s) in RCA: 8] [Impact Index Per Article: 4.0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/08/2022] [Accepted: 07/22/2022] [Indexed: 11/13/2022] Open
Abstract
We present a deep learning method for the segmentation of new lesions in longitudinal FLAIR MRI sequences acquired at two different time points. In our approach, the 3D volumes are processed slice-wise across the coronal, axial, and sagittal planes and the predictions from the three orientations are merged using an optimized voting strategy. Our method achieved best F1 score (0.541) among all participating methods in the MICCAI 2021 challenge Multiple sclerosis new lesions segmentation (MSSEG-2). Moreover, we show that our method is on par with the challenge's expert neuroradiologists: on an unbiased ground truth, our method achieves results comparable to those of the four experts in terms of detection (F1 score) and segmentation accuracy (Dice score).
Collapse
|
16
|
Zhu W, Huang H, Zhou Y, Shi F, Shen H, Chen R, Hua R, Wang W, Xu S, Luo X. Automatic segmentation of white matter hyperintensities in routine clinical brain MRI by 2D VB-Net: A large-scale study. Front Aging Neurosci 2022; 14:915009. [PMID: 35966772 PMCID: PMC9372352 DOI: 10.3389/fnagi.2022.915009] [Citation(s) in RCA: 9] [Impact Index Per Article: 4.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/07/2022] [Accepted: 07/14/2022] [Indexed: 11/13/2022] Open
Abstract
White matter hyperintensities (WMH) are imaging manifestations frequently observed in various neurological disorders, yet the clinical application of WMH quantification is limited. In this study, we designed a series of dedicated WMH labeling protocols and proposed a convolutional neural network named 2D VB-Net for the segmentation of WMH and other coexisting intracranial lesions based on a large dataset of 1,045 subjects across various demographics and multiple scanners using 2D thick-slice protocols that are more commonly applied in clinical practice. Using our labeling pipeline, the Dice consistency of the WMH regions manually depicted by two observers was 0.878, which formed a solid basis for the development and evaluation of the automatic segmentation system. The proposed algorithm outperformed other state-of-the-art methods (uResNet, 3D V-Net and Visual Geometry Group network) in the segmentation of WMH and other coexisting intracranial lesions and was well validated on datasets with thick-slice magnetic resonance (MR) images and the 2017 medical image computing and computer assisted intervention WMH Segmentation Challenge dataset (with thin-slice MR images), all showing excellent effectiveness. Furthermore, our method can subclassify WMH to display the WMH distributions and is very lightweight. Additionally, in terms of correlation to visual rating scores, our algorithm showed excellent consistency with the manual delineations and was overall better than those from other competing methods. In conclusion, we developed an automatic WMH quantification framework for multiple application scenarios, exhibiting a promising future in clinical practice.
Collapse
Affiliation(s)
- Wenhao Zhu
- Department of Neurology, Tongji Hospital, Tongji Medical College, Huazhong University of Science and Technology, Wuhan, China
| | - Hao Huang
- Department of Neurology, Tongji Hospital, Tongji Medical College, Huazhong University of Science and Technology, Wuhan, China
| | - Yaqi Zhou
- Shanghai United Imaging Intelligence, Wuhan, China
| | - Feng Shi
- Shanghai United Imaging Intelligence, Shanghai, China
| | - Hong Shen
- Shanghai United Imaging Intelligence, Wuhan, China
| | - Ran Chen
- Shanghai United Imaging Intelligence, Wuhan, China
| | - Rui Hua
- Shanghai United Imaging Intelligence, Shanghai, China
| | - Wei Wang
- Department of Neurology, Tongji Hospital, Tongji Medical College, Huazhong University of Science and Technology, Wuhan, China
| | - Shabei Xu
- Department of Neurology, Tongji Hospital, Tongji Medical College, Huazhong University of Science and Technology, Wuhan, China
| | - Xiang Luo
- Department of Neurology, Tongji Hospital, Tongji Medical College, Huazhong University of Science and Technology, Wuhan, China
| |
Collapse
|
17
|
Role of artificial intelligence in MS clinical practice. Neuroimage Clin 2022; 35:103065. [PMID: 35661470 PMCID: PMC9163993 DOI: 10.1016/j.nicl.2022.103065] [Citation(s) in RCA: 22] [Impact Index Per Article: 11.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/23/2021] [Revised: 05/04/2022] [Accepted: 05/26/2022] [Indexed: 11/24/2022]
Abstract
For medical applications, machine learning (including deep learning) are the most commonly used artificial intelligence (AI) approaches. It can improve multiple sclerosis (MS) diagnosis, prognostication and treatment monitoring. Thanks to AI, MRI and cognitive phenotypes of MS patients were identified. AI can shorten MRI protocols for MS, allowing the application of advanced techniques. It can reduce the human effort for MRI analysis, especially for lesion segmentation.
Machine learning (ML) and its subset, deep learning (DL), are branches of artificial intelligence (AI) showing promising findings in the medical field, especially when applied to imaging data. Given the substantial role of MRI in the diagnosis and management of patients with multiple sclerosis (MS), this disease is an ideal candidate for the application of AI techniques. In this narrative review, we are going to discuss the potential applications of AI for MS clinical practice, together with their limitations. Among their several advantages, ML algorithms are able to automate repetitive tasks, to analyze more data in less time and to achieve higher accuracy and reproducibility than the human counterpart. To date, these algorithms have been applied to MS diagnosis, prognosis, disease and treatment monitoring. Other fields of application have been improvement of MRI protocols as well as automated lesion and tissue segmentation. However, several challenges remain, including a better understanding of the information selected by AI algorithms, appropriate multicenter and longitudinal validations of results and practical aspects regarding hardware and software integration. Finally, one cannot overemphasize the paramount importance of human supervision, in order to optimize the use and take full advantage of the potential of AI approaches.
Collapse
|
18
|
Ong K, Young DM, Sulaiman S, Shamsuddin SM, Mohd Zain NR, Hashim H, Yuen K, Sanders SJ, Yu W, Hang S. Detection of subtle white matter lesions in MRI through texture feature extraction and boundary delineation using an embedded clustering strategy. Sci Rep 2022; 12:4433. [PMID: 35292654 PMCID: PMC8924181 DOI: 10.1038/s41598-022-07843-8] [Citation(s) in RCA: 2] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/29/2021] [Accepted: 02/24/2022] [Indexed: 11/29/2022] Open
Abstract
White matter lesions (WML) underlie multiple brain disorders, and automatic WML segmentation is crucial to evaluate the natural disease course and effectiveness of clinical interventions, including drug discovery. Although recent research has achieved tremendous progress in WML segmentation, accurate detection of subtle WML present early in the disease course remains particularly challenging. Here we propose an approach to automatic WML segmentation of mild WML loads using an intensity standardisation technique, gray level co-occurrence matrix (GLCM) embedded clustering technique, and random forest (RF) classifier to extract texture features and identify morphology specific to true WML. We precisely define their boundaries through a local outlier factor (LOF) algorithm that identifies edge pixels by local density deviation relative to its neighbors. The automated approach was validated on 32 human subjects, demonstrating strong agreement and correlation (excluding one outlier) with manual delineation by a neuroradiologist through Intra-Class Correlation (ICC = 0.881, 95% CI 0.769, 0.941) and Pearson correlation (r = 0.895, p-value < 0.001), respectively, and outperforming three leading algorithms (Trimmed Mean Outlier Detection, Lesion Prediction Algorithm, and SALEM-LS) in five of the six established key metrics defined in the MICCAI Grand Challenge. By facilitating more accurate segmentation of subtle WML, this approach may enable earlier diagnosis and intervention.
Collapse
Affiliation(s)
- Kokhaur Ong
- Bioinformatics Institute, A*STAR, Singapore, Singapore.,Institute of Molecular and Cell Biology, A*STAR, Singapore, Singapore
| | - David M Young
- Institute of Molecular and Cell Biology, A*STAR, Singapore, Singapore.,Department of Psychiatry and Behavioral Sciences, UCSF Weill Institute for Neurosciences, University of California, San Francisco, USA
| | - Sarina Sulaiman
- School of Computing, Faculty of Engineering, Universiti Teknologi Malaysia, Johor, Malaysia
| | | | | | - Hilwati Hashim
- Department of Radiology, Faculty of Medicine, Universiti Teknologi MARA, Sungai Buloh, Malaysia
| | - Kahhay Yuen
- School of Pharmaceutical Sciences, Universiti Sains Malaysia, Penang, Malaysia
| | - Stephan J Sanders
- Department of Psychiatry and Behavioral Sciences, UCSF Weill Institute for Neurosciences, University of California, San Francisco, USA
| | - Weimiao Yu
- Bioinformatics Institute, A*STAR, Singapore, Singapore. .,Institute of Molecular and Cell Biology, A*STAR, Singapore, Singapore. .,Computational Digital Pathology Laboratory, Bioinformatics Institute (BII), 30 Biopolis Street, #07-46 Matrix, Singapore, 138671, Singapore.
| | - Seepheng Hang
- Department of Mathematical Sciences, Faculty of Science, Universiti Teknologi Malaysia, UTM Skudai, 81310, Johor, Malaysia.
| |
Collapse
|
19
|
Tran P, Thoprakarn U, Gourieux E, Dos Santos CL, Cavedo E, Guizard N, Cotton F, Krolak-Salmon P, Delmaire C, Heidelberg D, Pyatigorskaya N, Ströer S, Dormont D, Martini JB, Chupin M. Automatic segmentation of white matter hyperintensities: validation and comparison with state-of-the-art methods on both Multiple Sclerosis and elderly subjects. Neuroimage Clin 2022; 33:102940. [PMID: 35051744 PMCID: PMC8896108 DOI: 10.1016/j.nicl.2022.102940] [Citation(s) in RCA: 11] [Impact Index Per Article: 5.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/28/2021] [Revised: 12/15/2021] [Accepted: 01/06/2022] [Indexed: 11/27/2022]
Abstract
Automatic segmentation of MS lesions and age-related WMH from 3D T1 and T2-FLAIR. Comparison to consensus show improved performance of WHASA-3D compared to WHASA. WHASA-3D outperforms available state-of-the-art methods with their default settings. WHASA-3D could be a useful tool for clinical practice and clinical trials.
Different types of white matter hyperintensities (WMH) can be observed through MRI in the brain and spinal cord, especially Multiple Sclerosis (MS) lesions for patients suffering from MS and age-related WMH for subjects with cognitive disorders and/or elderly people. To better diagnose and monitor the disease progression, the quantitative evaluation of WMH load has proven to be useful for clinical routine and trials. Since manual delineation for WMH segmentation is highly time-consuming and suffers from intra and inter observer variability, several methods have been proposed to automatically segment either MS lesions or age-related WMH, but none is validated on both WMH types. Here, we aim at proposing the White matter Hyperintensities Automatic Segmentation Algorithm adapted to 3D T2-FLAIR datasets (WHASA-3D), a fast and robust automatic segmentation tool designed to be implemented in clinical practice for the detection of both MS lesions and age-related WMH in the brain, using both 3D T1-weighted and T2-FLAIR images. In order to increase its robustness for MS lesions, WHASA-3D expands the original WHASA method, which relies on the coupling of non-linear diffusion framework and watershed parcellation, where regions considered as WMH are selected based on intensity and location characteristics, and finally refined with geodesic dilation. The previous validation was performed on 2D T2-FLAIR and subjects with cognitive disorders and elderly subjects. 60 subjects from a heterogeneous database of dementia patients, multiple sclerosis patients and elderly subjects with multiple MRI scanners and a wide range of lesion loads were used to evaluate WHASA and WHASA-3D through volume and spatial agreement in comparison with consensus reference segmentations. In addition, a direct comparison on the MS database with six available supervised and unsupervised state-of-the-art WMH segmentation methods (LST-LGA and LPA, Lesion-TOADS, lesionBrain, BIANCA and nicMSlesions) with default and optimised settings (when feasible) was conducted. WHASA-3D confirmed an improved performance with respect to WHASA, achieving a better spatial overlap (Dice) (0.67 vs 0.63), a reduced absolute volume error (AVE) (3.11 vs 6.2 mL) and an increased volume agreement (intraclass correlation coefficient, ICC) (0.96 vs 0.78). Compared to available state-of-the-art algorithms on the MS database, WHASA-3D outperformed both unsupervised and supervised methods when used with their default settings, showing the highest volume agreement (ICC = 0.95) as well as the highest average Dice (0.58). Optimising and/or retraining LST-LGA, BIANCA and nicMSlesions, using a subset of the MS database as training set, resulted in improved performances on the remaining testing set (average Dice: LST-LGA default/optimized = 0.41/0.51, BIANCA default/optimized = 0.22/0.39, nicMSlesions default/optimized = 0.17/0.63, WHASA-3D = 0.58). Evaluation and comparison results suggest that WHASA-3D is a reliable and easy-to-use method for the automated segmentation of white matter hyperintensities, for both MS lesions and age-related WMH. Further validation on larger datasets would be useful to confirm these first findings.
Collapse
Affiliation(s)
- Philippe Tran
- Qynapse, Paris, France; Equipe-projet ARAMIS, ICM, CNRS UMR 7225, Inserm U1117, Sorbonne Université UMR_S 1127, Centre Inria de Paris, Groupe Hospitalier Pitié-Salpêtrière Charles Foix, Faculté de Médecine Sorbonne Université, Paris, France.
| | | | - Emmanuelle Gourieux
- CATI, ICM, CNRS UMR 7225, Inserm U1117, Sorbonne Université UMR_S 1127, Paris, France; NeuroSpin, CEA, Saclay, France
| | | | | | | | - François Cotton
- Service de Radiologie, Centre Hospitalier Lyon-Sud, Hospices Civils de Lyon, Pierre-Bénite, France; Univ Lyon, INSA-Lyon, Université Claude Bernard Lyon 1, UJM-Saint Etienne, CNRS, Inserm, CREATIS UMR 5220, U1206, F-69495, Pierre-Bénite, France
| | - Pierre Krolak-Salmon
- Univ Lyon, INSA-Lyon, Université Claude Bernard Lyon 1, UJM-Saint Etienne, CNRS, Inserm, CREATIS UMR 5220, U1206, F-69495, Pierre-Bénite, France; Clinical and Research Memory Centre of Lyon, Hospices Civils de Lyon, Lyon, France; INSERM, U1028, UMR CNRS 5292, Lyon Neuroscience Research Center, Lyon, France
| | | | - Damien Heidelberg
- Service de Radiologie, Centre Hospitalier Lyon-Sud, Hospices Civils de Lyon, Pierre-Bénite, France
| | - Nadya Pyatigorskaya
- Department of Neuroradiology, Groupe Hospitalier Pitié-Salpêtrière, AP-HP, Sorbonne Université UMR_S 1127, Paris, France
| | - Sébastian Ströer
- Department of Neuroradiology, Groupe Hospitalier Pitié-Salpêtrière, AP-HP, Sorbonne Université UMR_S 1127, Paris, France
| | - Didier Dormont
- Equipe-projet ARAMIS, ICM, CNRS UMR 7225, Inserm U1117, Sorbonne Université UMR_S 1127, Centre Inria de Paris, Groupe Hospitalier Pitié-Salpêtrière Charles Foix, Faculté de Médecine Sorbonne Université, Paris, France; Department of Neuroradiology, Groupe Hospitalier Pitié-Salpêtrière, AP-HP, Sorbonne Université UMR_S 1127, Paris, France
| | | | - Marie Chupin
- CATI, ICM, CNRS UMR 7225, Inserm U1117, Sorbonne Université UMR_S 1127, Paris, France
| | | |
Collapse
|
20
|
Ma Y, Zhang C, Cabezas M, Song Y, Tang Z, Liu D, Cai W, Barnett M, Wang C. Multiple Sclerosis Lesion Analysis in Brain Magnetic Resonance Images: Techniques and Clinical Applications. IEEE J Biomed Health Inform 2022; 26:2680-2692. [PMID: 35171783 DOI: 10.1109/jbhi.2022.3151741] [Citation(s) in RCA: 9] [Impact Index Per Article: 4.5] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/10/2022]
Abstract
Multiple sclerosis (MS) is a chronic inflammatory and degenerative disease of the central nervous system, characterized by the appearance of focal lesions in the white and gray matter that topographically correlate with an individual patients neurological symptoms and signs. Magnetic resonance imaging (MRI) provides detailed in-vivo structural information, permitting the quantification and categorization of MS lesions that critically inform disease management. Traditionally, MS lesions have been manually annotated on 2D MRI slices, a process that is inefficient and prone to inter-/intra-observer errors. Recently, automated statistical imaging analysis techniques have been proposed to detect and segment MS lesions based on MRI voxel intensity. However, their effectiveness is limited by the heterogeneity of both MRI data acquisition techniques and the appearance of MS lesions. By learning complex lesion representations directly from images, deep learning techniques have achieved remarkable breakthroughs in the MS lesion segmentation task. Here, we provide a comprehensive review of state-of-the-art automatic statistical and deep-learning MS segmentation methods and discuss current and future clinical applications. Further, we review technical strategies, such as domain adaptation, to enhance MS lesion segmentation in real-world clinical settings.
Collapse
|
21
|
Nabizadeh F, Masrouri S, Ramezannezhad E, Ghaderi A, Sharafi AM, Soraneh S, Moghadasi AN. Artificial intelligence in the diagnosis of Multiple Sclerosis: a systematic review. Mult Scler Relat Disord 2022; 59:103673. [DOI: 10.1016/j.msard.2022.103673] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/03/2022] [Revised: 01/24/2022] [Accepted: 02/05/2022] [Indexed: 10/19/2022]
|
22
|
Multiple sclerosis lesions segmentation from multiple experts: The MICCAI 2016 challenge dataset. Neuroimage 2021; 244:118589. [PMID: 34563682 DOI: 10.1016/j.neuroimage.2021.118589] [Citation(s) in RCA: 11] [Impact Index Per Article: 3.7] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/27/2021] [Revised: 09/03/2021] [Accepted: 09/16/2021] [Indexed: 11/23/2022] Open
Abstract
MRI plays a crucial role in multiple sclerosis diagnostic and patient follow-up. In particular, the delineation of T2-FLAIR hyperintense lesions is crucial although mostly performed manually - a tedious task. Many methods have thus been proposed to automate this task. However, sufficiently large datasets with a thorough expert manual segmentation are still lacking to evaluate these methods. We present a unique dataset for MS lesions segmentation evaluation. It consists of 53 patients acquired on 4 different scanners with a harmonized protocol. Hyperintense lesions on FLAIR were manually delineated on each patient by 7 experts with control on T2 sequence, and gathered in a consensus segmentation for evaluation. We provide raw and preprocessed data and a split of the dataset into training and testing data, the latter including data from a scanner not present in the training dataset. We strongly believe that this dataset will become a reference in MS lesions segmentation evaluation, allowing to evaluate many aspects: evaluation of performance on unseen scanner, comparison to individual experts performance, comparison to other challengers who already used this dataset, etc.
Collapse
|
23
|
Kamraoui RA, Ta VT, Tourdias T, Mansencal B, Manjon JV, Coupé P. DeepLesionBrain: Towards a broader deep-learning generalization for multiple sclerosis lesion segmentation. Med Image Anal 2021; 76:102312. [PMID: 34894571 DOI: 10.1016/j.media.2021.102312] [Citation(s) in RCA: 17] [Impact Index Per Article: 5.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/14/2020] [Revised: 10/18/2021] [Accepted: 11/22/2021] [Indexed: 11/29/2022]
Abstract
Recently, segmentation methods based on Convolutional Neural Networks (CNNs) showed promising performance in automatic Multiple Sclerosis (MS) lesions segmentation. These techniques have even outperformed human experts in controlled evaluation conditions such as Longitudinal MS Lesion Segmentation Challenge (ISBI Challenge). However, state-of-the-art approaches trained to perform well on highly-controlled datasets fail to generalize on clinical data from unseen datasets. Instead of proposing another improvement of the segmentation accuracy, we propose a novel method robust to domain shift and performing well on unseen datasets, called DeepLesionBrain (DLB). This generalization property results from three main contributions. First, DLB is based on a large group of compact 3D CNNs. This spatially distributed strategy aims to produce a robust prediction despite the risk of generalization failure of some individual networks. Second, we propose a hierarchical specialization learning (HSL) by pre-training a generic network over the whole brain, before using its weights as initialization to locally specialized networks. By this end, DLB learns both generic features extracted at global image level and specific features extracted at local image level. Finally, DLB includes a new image quality data augmentation to reduce dependency to training data specificity (e.g., acquisition protocol). DLB generalization was validated in cross-dataset experiments on MSSEG'16, ISBI challenge, and in-house datasets. During experiments, DLB showed higher segmentation accuracy, better segmentation consistency and greater generalization performance compared to state-of-the-art methods. Therefore, DLB offers a robust framework well-suited for clinical practice.
Collapse
Affiliation(s)
| | - Vinh-Thong Ta
- Univ. Bordeaux, Bordeaux INP, CNRS, LaBRI, UMR5800, PICTURA, Talence F-33400, France
| | - Thomas Tourdias
- Service de Neuroimagerie Diagnostique et Thérapeutique, Univ. Bordeaux, Bordeaux F-33000, France; Univ. Bordeaux, INSERM, Neurocentre Magendie, U1215, Bordeaux F-3300, France
| | - Boris Mansencal
- Univ. Bordeaux, Bordeaux INP, CNRS, LaBRI, UMR5800, PICTURA, Talence F-33400, France
| | - José V Manjon
- ITACA, Universitat Politécnica de Valéncia, Valencia 46022, Spain
| | - Pierrick Coupé
- Univ. Bordeaux, Bordeaux INP, CNRS, LaBRI, UMR5800, PICTURA, Talence F-33400, France
| |
Collapse
|
24
|
Al-Mubarak H, Vallatos A, Gallagher L, Birch J, Chalmers AJ, Holmes WM. Evaluating potential of multi-parametric MRI using co-registered histology: Application to a mouse model of glioblastoma. Magn Reson Imaging 2021; 85:121-127. [PMID: 34687852 DOI: 10.1016/j.mri.2021.10.030] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/05/2020] [Revised: 04/23/2021] [Accepted: 10/17/2021] [Indexed: 11/30/2022]
Abstract
BACKGROUND Conventional MRI fails to detect regions of glioblastoma cell infiltration beyond the contrast-enhanced T1 solid tumor region, with infiltrating tumor cells often migrating along host blood vessels. PURPOSE MRI is capable of generating a range of image contrasts which are commonly assessed individually by qualitative visual inspection. It has long been hypothesized that better diagnoses could be achieved by combining these multiple images, so called multi-parametric or multi-spectral MRI. However, the lack of clinical histology and the difficulties of co-registration, has meant this hypothesis has never been rigorously tested. Here we test this hypothesis, using a previously published multi-dimensional dataset consisting of registered MR images and histology. STUDY TYPE Animal Model. SUBJECTS Mice bearing orthotopic glioblastoma xenografts generated from a patient-derived glioblastoma cell line. FIELD STRENGTH/SEQUENCES 7 Tesla, T1/T2 weighted, T2 mapping, contrast enhance T1, diffusion-weighted, diffusion tensor imaging. ASSESSMENT Immunohistochemistry sections were stained for Human Leukocyte Antigen (probing human-derived tumor cells). To achieve quantitative MRI-tissue comparison, multiple histological slices cut in the MRI plane were stacked to produce tumor cell density maps acting as 'ground truth'. STATISTICAL TESTS Sensitivity, specificity, accuracy and Dice similarity indices were calculated. ANOVA, t-test, Bonferroni correction and Pearson coefficients were used for statistical analysis. RESULTS Correlation coefficient analysis with co-registered 'ground truth' histology showed interactive regression maps had higher correlation coefficients and sensitivity values than T2W, ADC, FA, and T2map. Further, the interaction regression maps showed statistical improved detection of tumor volume. DATA CONCLUSION Voxel-by-voxel analysis provided quantitative evidence confirming the hypothesis that mpMRI can, potentially, better distinguish between the tumor region and normal tissue.
Collapse
Affiliation(s)
- H Al-Mubarak
- Glasgow Experimental MRI centre, Institute of Neuroscience and Psychology, University of Glasgow, G61 1QH, UK; Department of Physics, College of Science, University of Misan, Iraq.
| | - A Vallatos
- Centre for Clinical Brain Sciences, University of Edinburgh, EH16 4SB,UK.
| | - L Gallagher
- Glasgow Experimental MRI centre, Institute of Neuroscience and Psychology, University of Glasgow, G61 1QH, UK.
| | - J Birch
- Beatson Institute for Cancer Research, UK.
| | - A J Chalmers
- Wolfson Wohl Translational Cancer Research Centre, Institute of Cancer Sciences University of Glasgow, G61 1QH, UK.
| | - W M Holmes
- Glasgow Experimental MRI centre, Institute of Neuroscience and Psychology, University of Glasgow, G61 1QH, UK.
| |
Collapse
|
25
|
Zhang Y, Duan Y, Wang X, Zhuo Z, Haller S, Barkhof F, Liu Y. A deep learning algorithm for white matter hyperintensity lesion detection and segmentation. Neuroradiology 2021; 64:727-734. [PMID: 34599377 DOI: 10.1007/s00234-021-02820-w] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/14/2021] [Accepted: 09/24/2021] [Indexed: 11/26/2022]
Abstract
PURPOSE White matter hyperintensity (WMHI) lesions on MR images are an important indication of various types of brain diseases that involve inflammation and blood vessel abnormalities. Automated quantification of the WMHI can be valuable for the clinical management of patients, but existing automated software is often developed for a single type of disease and may not be applicable for clinical scans with thick slices and different scanning protocols. The purpose of the study is to develop and validate an algorithm for automatic quantification of white matter hyperintensity suitable for heterogeneous MRI data with different disease types. METHODS We developed and evaluated "DeepWML", a deep learning method for fully automated white matter lesion (WML) segmentation of multicentre FLAIR images. We used MRI from 507 patients, including three distinct white matter diseases, obtained in 9 centres, with a wide range of scanners and acquisition protocols. The automated delineation tool was evaluated through quantitative parameters of Dice similarity, sensitivity and precision compared to manual delineation (gold standard). RESULTS The overall median Dice similarity coefficient was 0.78 (range 0.64 ~ 0.86) across the three disease types and multiple centres. The median sensitivity and precision were 0.84 (range 0.67 ~ 0.94) and 0.81 (range 0.64 ~ 0.92), respectively. The tool's performance increased with larger lesion volumes. CONCLUSION DeepWML was successfully applied to a wide spectrum of MRI data in the three white matter disease types, which has the potential to improve the practical workflow of white matter lesion delineation.
Collapse
Affiliation(s)
- Yajing Zhang
- MR Clinical Science, Philips Healthcare, 258 Zhongyuan Rd, Suzhou, SIP, China
| | - Yunyun Duan
- Department of Radiology, Beijing Tiantan Hospital, Fengtai District, Capital Medical University, No. 119 the West Southern 4th Ring Road, Beijing, 100070, China
| | - Xiaoyang Wang
- MR Clinical Science, Philips Healthcare, 258 Zhongyuan Rd, Suzhou, SIP, China
| | - Zhizheng Zhuo
- Department of Radiology, Beijing Tiantan Hospital, Fengtai District, Capital Medical University, No. 119 the West Southern 4th Ring Road, Beijing, 100070, China
| | - Sven Haller
- Department of Imaging and Medical Informatics, University of Geneva, Geneva, Switzerland
- Faculty of Medicine, University of Geneva, Geneva, Switzerland
| | - Frederik Barkhof
- Department of Radiology and Nuclear Medicine, Amsterdam University Medical Center, Amsterdam, The Netherlands
- Queen Square Institute of Neurology and Center for Medical Image Computing, University College, London, UK
| | - Yaou Liu
- Department of Radiology, Beijing Tiantan Hospital, Fengtai District, Capital Medical University, No. 119 the West Southern 4th Ring Road, Beijing, 100070, China.
| |
Collapse
|
26
|
Hindsholm AM, Cramer SP, Simonsen HJ, Frederiksen JL, Andersen F, Højgaard L, Ladefoged CN, Lindberg U. Assessment of Artificial Intelligence Automatic Multiple Sclerosis Lesion Delineation Tool for Clinical Use. Clin Neuroradiol 2021; 32:643-653. [PMID: 34542644 PMCID: PMC9424132 DOI: 10.1007/s00062-021-01089-z] [Citation(s) in RCA: 4] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/02/2021] [Accepted: 08/16/2021] [Indexed: 11/24/2022]
Abstract
PURPOSE To implement and validate an existing algorithm for automatic delineation of white matter lesions on magnetic resonance imaging (MRI) in patients with multiple sclerosis (MS) on a local single-center dataset. METHODS We implemented a white matter hyperintensity segmentation model, based on a 2D convolutional neural network, using the conventional T2-weighted fluid attenuated inversion recovery (FLAIR) MRI sequence as input. The model was adapted for delineation of MS lesions by further training on a local dataset of 93 MS patients with a total of 3040 lesions. A quantitative evaluation was performed on ten test patients, in which model-generated masks were compared to manually delineated masks from two expert delineators. A subsequent qualitative evaluation of the implemented model was performed by two expert delineators, in which generated delineation masks on a clinical dataset of 53 patients were rated acceptable (< 10% errors) or unacceptable (> 10% errors) based on the total number of true lesions. RESULTS The quantitative evaluation resulted in an average accuracy score (F1) of 0.71, recall of 0.77 and dice similarity coefficient of 0.62. Our implemented model obtained the highest scores in all three metrics, when compared to three out of the box lesion segmentation models. In the clinical evaluation an average of 94% of our 53 model-generated masks were rated acceptable. CONCLUSION After adaptation to our local dataset, the implemented segmentation model was able to delineate MS lesions with a high clinical value as rated by delineation experts while outperforming popular out of the box applications. This serves as a promising step towards implementation of automatic lesion delineation in our MS clinic.
Collapse
Affiliation(s)
- Amalie Monberg Hindsholm
- Department of Clinical Physiology, Nuclear Medicine and PET, Rigshospitalet, University of Copenhagen, Blegdamsvej 9, 2100, Copenhagen east, Denmark.
| | - Stig Præstekjær Cramer
- Department of Clinical Physiology, Nuclear Medicine and PET, Rigshospitalet, University of Copenhagen, Blegdamsvej 9, 2100, Copenhagen east, Denmark
| | - Helle Juhl Simonsen
- Department of Clinical Physiology, Nuclear Medicine and PET, Rigshospitalet, University of Copenhagen, Blegdamsvej 9, 2100, Copenhagen east, Denmark
| | - Jette Lautrup Frederiksen
- Danish Multiple Sclerosis Center, Department of Neurology, Rigshospitalet, University of Copenhagen, Valdemar Hansens Vej 13, 2600, Glostrup, Denmark
| | - Flemming Andersen
- Department of Clinical Physiology, Nuclear Medicine and PET, Rigshospitalet, University of Copenhagen, Blegdamsvej 9, 2100, Copenhagen east, Denmark
| | - Liselotte Højgaard
- Department of Clinical Physiology, Nuclear Medicine and PET, Rigshospitalet, University of Copenhagen, Blegdamsvej 9, 2100, Copenhagen east, Denmark
| | - Claes Nøhr Ladefoged
- Department of Clinical Physiology, Nuclear Medicine and PET, Rigshospitalet, University of Copenhagen, Blegdamsvej 9, 2100, Copenhagen east, Denmark
| | - Ulrich Lindberg
- Department of Clinical Physiology, Nuclear Medicine and PET, Rigshospitalet, University of Copenhagen, Blegdamsvej 9, 2100, Copenhagen east, Denmark
| |
Collapse
|
27
|
Adaptive segmentation model for liver CT images based on neural network and level set method. Neurocomputing 2021. [DOI: 10.1016/j.neucom.2021.01.081] [Citation(s) in RCA: 18] [Impact Index Per Article: 6.0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/23/2022]
|
28
|
Ansari SU, Javed K, Qaisar SM, Jillani R, Haider U. Multiple Sclerosis Lesion Segmentation in Brain MRI Using Inception Modules Embedded in a Convolutional Neural Network. JOURNAL OF HEALTHCARE ENGINEERING 2021; 2021:4138137. [PMID: 34484652 PMCID: PMC8410443 DOI: 10.1155/2021/4138137] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 06/11/2021] [Accepted: 07/27/2021] [Indexed: 11/17/2022]
Abstract
Multiple sclerosis (MS) is a chronic and autoimmune disease that forms lesions in the central nervous system. Quantitative analysis of these lesions has proved to be very useful in clinical trials for therapies and assessing disease prognosis. However, the efficacy of these quantitative analyses greatly depends on how accurately the MS lesions have been identified and segmented in brain MRI. This is usually carried out by radiologists who label 3D MR images slice by slice using commonly available segmentation tools. However, such manual practices are time consuming and error prone. To circumvent this problem, several automatic segmentation techniques have been investigated in recent years. In this paper, we propose a new framework for automatic brain lesion segmentation that employs a novel convolutional neural network (CNN) architecture. In order to segment lesions of different sizes, we have to pick a specific filter or size 3 × 3 or 5 × 5. Sometimes, it is hard to decide which filter will work better to get the best results. Google Net has solved this problem by introducing an inception module. An inception module uses 3 × 3, 5 × 5, 1 × 1 and max pooling filters in parallel fashion. Results show that incorporating inception modules in a CNN has improved the performance of the network in the segmentation of MS lesions. We compared the results of the proposed CNN architecture for two loss functions: binary cross entropy (BCE) and structural similarity index measure (SSIM) using the publicly available ISBI-2015 challenge dataset. A score of 93.81 which is higher than the human rater with BCE loss function is achieved.
Collapse
Affiliation(s)
- Shahab U. Ansari
- Faculty of Computer Science and Engineering, Ghulam Ishaq Khan Institute of Engineering Sciences and Technology, Topi, Pakistan
| | - Kamran Javed
- Faculty of Computer Science and Engineering, Ghulam Ishaq Khan Institute of Engineering Sciences and Technology, Topi, Pakistan
- National Centre of Artificial Intelligence (NCAI), Saudi Data and Artificial Intelligence Authority (SDAIA), Riyadh, Saudi Arabia
| | - Saeed Mian Qaisar
- Electrical and Computer Engineering Department, Effat University, Jeddah 22332, Saudi Arabia
- Communication and Signal Processing Lab, Energy and Technology Research Center, Effat University, Jeddah 22332, Saudi Arabia
| | - Rashad Jillani
- Faculty of Computer Science and Engineering, Ghulam Ishaq Khan Institute of Engineering Sciences and Technology, Topi, Pakistan
| | - Usman Haider
- Faculty of Computer Science and Engineering, Ghulam Ishaq Khan Institute of Engineering Sciences and Technology, Topi, Pakistan
| |
Collapse
|
29
|
Shoeibi A, Khodatars M, Jafari M, Moridian P, Rezaei M, Alizadehsani R, Khozeimeh F, Gorriz JM, Heras J, Panahiazar M, Nahavandi S, Acharya UR. Applications of deep learning techniques for automated multiple sclerosis detection using magnetic resonance imaging: A review. Comput Biol Med 2021; 136:104697. [PMID: 34358994 DOI: 10.1016/j.compbiomed.2021.104697] [Citation(s) in RCA: 50] [Impact Index Per Article: 16.7] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/11/2021] [Revised: 07/22/2021] [Accepted: 07/25/2021] [Indexed: 11/18/2022]
Abstract
Multiple Sclerosis (MS) is a type of brain disease which causes visual, sensory, and motor problems for people with a detrimental effect on the functioning of the nervous system. In order to diagnose MS, multiple screening methods have been proposed so far; among them, magnetic resonance imaging (MRI) has received considerable attention among physicians. MRI modalities provide physicians with fundamental information about the structure and function of the brain, which is crucial for the rapid diagnosis of MS lesions. Diagnosing MS using MRI is time-consuming, tedious, and prone to manual errors. Research on the implementation of computer aided diagnosis system (CADS) based on artificial intelligence (AI) to diagnose MS involves conventional machine learning and deep learning (DL) methods. In conventional machine learning, feature extraction, feature selection, and classification steps are carried out by using trial and error; on the contrary, these steps in DL are based on deep layers whose values are automatically learn. In this paper, a complete review of automated MS diagnosis methods performed using DL techniques with MRI neuroimaging modalities is provided. Initially, the steps involved in various CADS proposed using MRI modalities and DL techniques for MS diagnosis are investigated. The important preprocessing techniques employed in various works are analyzed. Most of the published papers on MS diagnosis using MRI modalities and DL are presented. The most significant challenges facing and future direction of automated diagnosis of MS using MRI modalities and DL techniques are also provided.
Collapse
Affiliation(s)
- Afshin Shoeibi
- Faculty of Electrical Engineering, Biomedical Data Acquisition Lab (BDAL), K. N. Toosi University of Technology, Tehran, Iran.
| | - Marjane Khodatars
- Faculty of Engineering, Mashhad Branch, Islamic Azad University, Mashhad, Iran
| | - Mahboobeh Jafari
- Electrical and Computer Engineering Faculty, Semnan University, Semnan, Iran
| | - Parisa Moridian
- Faculty of Engineering, Science and Research Branch, Islamic Azad University, Tehran, Iran
| | - Mitra Rezaei
- Electrical and Computer Engineering Dept., Tarbiat Modares University, Tehran, Iran
| | - Roohallah Alizadehsani
- Institute for Intelligent Systems Research and Innovation (IISRI), Deakin University, Geelong, Australia
| | - Fahime Khozeimeh
- Institute for Intelligent Systems Research and Innovation (IISRI), Deakin University, Geelong, Australia
| | - Juan Manuel Gorriz
- Department of Signal Theory, Networking and Communications, Universidad de Granada, Spain; Department of Psychiatry. University of Cambridge, UK
| | - Jónathan Heras
- Department of Mathematics and Computer Science, University of La Rioja, La Rioja, Spain
| | | | - Saeid Nahavandi
- Institute for Intelligent Systems Research and Innovation (IISRI), Deakin University, Geelong, Australia
| | - U Rajendra Acharya
- Department of Biomedical Engineering, School of Science and Technology, Singapore University of Social Sciences, Singapore; Dept. of Electronics and Computer Engineering, Ngee Ann Polytechnic, 599489, Singapore; Department of Bioinformatics and Medical Engineering, Asia University, Taiwan
| |
Collapse
|
30
|
Hamwood J, Schmutz B, Collins MJ, Allenby MC, Alonso-Caneiro D. A deep learning method for automatic segmentation of the bony orbit in MRI and CT images. Sci Rep 2021; 11:13693. [PMID: 34211081 PMCID: PMC8249400 DOI: 10.1038/s41598-021-93227-3] [Citation(s) in RCA: 20] [Impact Index Per Article: 6.7] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/31/2020] [Accepted: 06/15/2021] [Indexed: 12/23/2022] Open
Abstract
This paper proposes a fully automatic method to segment the inner boundary of the bony orbit in two different image modalities: magnetic resonance imaging (MRI) and computed tomography (CT). The method, based on a deep learning architecture, uses two fully convolutional neural networks in series followed by a graph-search method to generate a boundary for the orbit. When compared to human performance for segmentation of both CT and MRI data, the proposed method achieves high Dice coefficients on both orbit and background, with scores of 0.813 and 0.975 in CT images and 0.930 and 0.995 in MRI images, showing a high degree of agreement with a manual segmentation by a human expert. Given the volumetric characteristics of these imaging modalities and the complexity and time-consuming nature of the segmentation of the orbital region in the human skull, it is often impractical to manually segment these images. Thus, the proposed method provides a valid clinical and research tool that performs similarly to the human observer.
Collapse
Affiliation(s)
- Jared Hamwood
- Contact Lens and Visual Optics Laboratory, Centre for Vision and Eye Research, School of Optometry and Vision Science, Queensland University of Technology (QUT), Kelvin Grove, Qld, 4059, Australia
| | - Beat Schmutz
- Centre in Regenerative Medicine, Institute of Health and Biomedical Innovation, Queensland University of Technology, Kelvin Grove, QLD, 4059, Australia
- Metro North Hospital and Health Service, Jamieson Trauma Institute, Herston, QLD, 4029, Australia
| | - Michael J Collins
- Contact Lens and Visual Optics Laboratory, Centre for Vision and Eye Research, School of Optometry and Vision Science, Queensland University of Technology (QUT), Kelvin Grove, Qld, 4059, Australia
| | - Mark C Allenby
- Biofabrication and Tissue Morphology Laboratory, Centre for Biomedical Technologies, School of Mechanical Medical and Process Engineering, Queensland University of Technology (QUT), Herston, Qld, 4000, Australia
| | - David Alonso-Caneiro
- Contact Lens and Visual Optics Laboratory, Centre for Vision and Eye Research, School of Optometry and Vision Science, Queensland University of Technology (QUT), Kelvin Grove, Qld, 4059, Australia.
| |
Collapse
|
31
|
Cui D, Hui ES, Cao P. A multi-inversion-recovery magnetic resonance fingerprinting for multi-compartment water mapping. Magn Reson Imaging 2021; 81:82-87. [PMID: 34146651 DOI: 10.1016/j.mri.2021.06.005] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/20/2021] [Revised: 06/11/2021] [Accepted: 06/15/2021] [Indexed: 11/25/2022]
Abstract
PURPOSE This study aimed at introducing short-T1/T2 compartment to MR fingerprinting (MRF) at 3 T. Water that is bound to myelin macromolecules have significantly shorter T1 and T2 than free water and can be distinguished from free water by multi-compartment analysis. METHODS We developed a new multi-inversion-recovery (mIR) water mapping-MRF based on an unbalanced steady-state coherent sequence (FISP). mIR pulses with an interval of 400 or 500 repetition times (TRs) were inserted into the conventional FISP MRF sequence. Data from our proposed mIR MRF was used to quantify different compartments, including myelin water, gray matter free water, and white matter free water, of brain water by virtue of the iterative non-negative least square (NNLS) with reweighting. Three healthy volunteers were scanned with mIR MRF on a clinical 3 T MRI. RESULTS Using an extended phase graph simulation, we found that our proposed mIR scheme with four IR pulses allowed differentiation between short and long T1/T2 components. For in vivo experiments, we achieved the quantification of myelin water, gray matter water, and white matter water at an image resolution of 1.17 × 1.17 × 5 mm3/pixel. As compared to the conventional MRF technique with single IR, our proposed mIR improved the detection of myelin water content. In addition, mIR MRF using spiral-in/out trajectory provided a higher signal level compared with that with spiral-out trajectory. Myelin water quantification using mIR MRF with 4 IR and 5 IR pulses were qualitatively similar. Meanwhile, 5 IR MRF showed fewer artifacts in myelin water detection. CONCLUSION We developed a new mIR MRF sequence for the rapid quantification of brain water compartments.
Collapse
Affiliation(s)
- Di Cui
- Department of Diagnostic Radiology, The University of Hong Kong, HKSAR, China
| | - Edward S Hui
- Department of Rehabilitation Science, The Hong Kong Polytechnic University, Hong Kong, HKSAR, China
| | - Peng Cao
- Department of Diagnostic Radiology, The University of Hong Kong, HKSAR, China.
| |
Collapse
|
32
|
Rakić M, Vercruyssen S, Van Eyndhoven S, de la Rosa E, Jain S, Van Huffel S, Maes F, Smeets D, Sima DM. icobrain ms 5.1: Combining unsupervised and supervised approaches for improving the detection of multiple sclerosis lesions. Neuroimage Clin 2021; 31:102707. [PMID: 34111718 PMCID: PMC8193144 DOI: 10.1016/j.nicl.2021.102707] [Citation(s) in RCA: 10] [Impact Index Per Article: 3.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/19/2021] [Revised: 05/20/2021] [Accepted: 05/21/2021] [Indexed: 01/03/2023]
Abstract
Multiple sclerosis (MS) is a chronic autoimmune, inflammatory neurological disease of the central nervous system. Its diagnosis nowadays commonly includes performing an MRI scan, as it is the most sensitive imaging test for MS. MS plaques are commonly identified from fluid-attenuated inversion recovery (FLAIR) images as hyperintense regions that are highly varying in terms of their shapes, sizes and locations, and are routinely classified in accordance to the McDonald criteria. Recent years have seen an increase in works that aimed at development of various semi-automatic and automatic methods for detection, segmentation and classification of MS plaques. In this paper, we present an automatic combined method, based on two pipelines: a traditional unsupervised machine learning technique and a deep-learning attention-gate 3D U-net network. The deep-learning network is specifically trained to address the weaker points of the traditional approach, namely difficulties in segmenting infratentorial and juxtacortical plaques in real-world clinical MRIs. It was trained and validated on a multi-center multi-scanner dataset that contains 159 cases, each with T1 weighted (T1w) and FLAIR images, as well as manual delineations of the MS plaques, segmented and validated by a panel of raters. The detection rate was quantified using lesion-wise Dice score. A simple label fusion is implemented to combine the output segmentations of the two pipelines. This combined method improves the detection of infratentorial and juxtacortical lesions by 14% and 31% respectively, in comparison to the unsupervised machine learning pipeline that was used as a performance assessment baseline.
Collapse
Affiliation(s)
- Mladen Rakić
- icometrix, Leuven, Belgium; KU Leuven, Department of Electrical Engineering (ESAT), Processing Speech and Images (PSI) and Medical Imaging Research Center, 3001 Leuven, Belgium.
| | | | | | - Ezequiel de la Rosa
- icometrix, Leuven, Belgium; Technical University of Munich, Department of Computer Science, Munich, Germany
| | | | - Sabine Van Huffel
- KU Leuven, Department of Electrical Engineering (ESAT), STADIUS Center for Dynamical Systems, Signal Processing and Data Analytics, 3001 Leuven, Belgium
| | - Frederik Maes
- KU Leuven, Department of Electrical Engineering (ESAT), Processing Speech and Images (PSI) and Medical Imaging Research Center, 3001 Leuven, Belgium
| | | | | |
Collapse
|
33
|
Kushibar K, Salem M, Valverde S, Rovira À, Salvi J, Oliver A, Lladó X. Transductive Transfer Learning for Domain Adaptation in Brain Magnetic Resonance Image Segmentation. Front Neurosci 2021; 15:608808. [PMID: 33994917 PMCID: PMC8116893 DOI: 10.3389/fnins.2021.608808] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/21/2020] [Accepted: 03/26/2021] [Indexed: 11/13/2022] Open
Abstract
Segmentation of brain images from Magnetic Resonance Images (MRI) is an indispensable step in clinical practice. Morphological changes of sub-cortical brain structures and quantification of brain lesions are considered biomarkers of neurological and neurodegenerative disorders and used for diagnosis, treatment planning, and monitoring disease progression. In recent years, deep learning methods showed an outstanding performance in medical image segmentation. However, these methods suffer from generalisability problem due to inter-centre and inter-scanner variabilities of the MRI images. The main objective of the study is to develop an automated deep learning segmentation approach that is accurate and robust to the variabilities in scanner and acquisition protocols. In this paper, we propose a transductive transfer learning approach for domain adaptation to reduce the domain-shift effect in brain MRI segmentation. The transductive scenario assumes that there are sets of images from two different domains: (1) source-images with manually annotated labels; and (2) target-images without expert annotations. Then, the network is jointly optimised integrating both source and target images into the transductive training process to segment the regions of interest and to minimise the domain-shift effect. We proposed to use a histogram loss in the feature level to carry out the latter optimisation problem. In order to demonstrate the benefit of the proposed approach, the method has been tested in two different brain MRI image segmentation problems using multi-centre and multi-scanner databases for: (1) sub-cortical brain structure segmentation; and (2) white matter hyperintensities segmentation. The experiments showed that the segmentation performance of a pre-trained model could be significantly improved by up to 10%. For the first segmentation problem it was possible to achieve a maximum improvement from 0.680 to 0.799 in average Dice Similarity Coefficient (DSC) metric and for the second problem the average DSC improved from 0.504 to 0.602. Moreover, the improvements after domain adaptation were on par or showed better performance compared to the commonly used traditional unsupervised segmentation methods (FIRST and LST), also achieving faster execution time. Taking this into account, this work presents one more step toward the practical implementation of deep learning algorithms into the clinical routine.
Collapse
Affiliation(s)
- Kaisar Kushibar
- Institute of Computer Vision and Robotics, University of Girona, Girona, Spain
| | - Mostafa Salem
- Institute of Computer Vision and Robotics, University of Girona, Girona, Spain.,Computer Science Department, Faculty of Computers and Information, Assiut University, Asyut, Egypt
| | - Sergi Valverde
- Institute of Computer Vision and Robotics, University of Girona, Girona, Spain
| | - Àlex Rovira
- Magnetic Resonance Unit, Department of Radiology, Vall d'Hebron University Hospital, Barcelona, Spain
| | - Joaquim Salvi
- Institute of Computer Vision and Robotics, University of Girona, Girona, Spain
| | - Arnau Oliver
- Institute of Computer Vision and Robotics, University of Girona, Girona, Spain
| | - Xavier Lladó
- Institute of Computer Vision and Robotics, University of Girona, Girona, Spain
| |
Collapse
|
34
|
Pridham G, Hossain S, Rawji KS, Zhang Y. A metric learning method for estimating myelin content based on T2-weighted MRI from a de- and re-myelination model of multiple sclerosis. PLoS One 2021; 16:e0249460. [PMID: 33819278 PMCID: PMC8021181 DOI: 10.1371/journal.pone.0249460] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/21/2020] [Accepted: 03/18/2021] [Indexed: 11/19/2022] Open
Abstract
Myelin plays a critical role in the pathogenesis of neurological disorders but is difficult to characterize in vivo using standard analysis methods. Our goal was to develop a novel analytical framework for estimating myelin content using T2-weighted magnetic resonance imaging (MRI) based on a de- and re-myelination model of multiple sclerosis. We examined 18 mice with lysolecithin induced demyelination and spontaneous remyelination in the ventral white matter of thoracic spinal cord. Cohorts of 6 mice underwent 9.4T MRI at days 7 (peak demyelination), 14 (ongoing recovery), and 28 (near complete recovery), as well as histological analysis of myelin and the associated cellularity at corresponding timepoints. Our MRI framework took an unsupervised learning approach, including tissue segmentation using a Gaussian Markov random field (GMRF), and myelin and cellularity feature estimation based on the Mahalanobis distance. For comparison, we also investigated 2 regression-based supervised learning approaches, one using our GMRF results, and another using a freely available generalized additive model (GAM). Results showed that GMRF segmentation was 73.2% accurate, and our unsupervised learning method achieved a correlation coefficient of 0.67 (top quartile: 0.78) with histological myelin, similar to 0.70 (top quartile: 0.78) obtained using supervised analyses. Further, the area under the receiver operator characteristic curve of our unsupervised myelin feature (0.883, 95% CI: 0.874–0.891) was significantly better than any of the supervised models in detecting white matter myelin as compared to histology. Collectively, metric learning using standard MRI may prove to be a new alternative method for estimating myelin content, which ultimately can improve our disease monitoring ability in a clinical setting.
Collapse
Affiliation(s)
- Glen Pridham
- Department of Clinical Neurosciences, University of Calgary, Calgary, Alberta, Canada
- Hotchkiss Brain Institute, University of Calgary, Calgary, Alberta, Canada
| | - Shahnewaz Hossain
- Department of Medical Sciences, University of Calgary, Calgary, Alberta, Canada
| | - Khalil S. Rawji
- Department of Clinical Neurosciences, University of Calgary, Calgary, Alberta, Canada
| | - Yunyan Zhang
- Department of Clinical Neurosciences, University of Calgary, Calgary, Alberta, Canada
- Hotchkiss Brain Institute, University of Calgary, Calgary, Alberta, Canada
- Department of Radiology, University of Calgary, Calgary, Alberta, Canada
- * E-mail:
| |
Collapse
|
35
|
DiGregorio J, Arezza G, Gibicar A, Moody AR, Tyrrell PN, Khademi A. Intracranial volume segmentation for neurodegenerative populations using multicentre FLAIR MRI. NEUROIMAGE: REPORTS 2021. [DOI: 10.1016/j.ynirp.2021.100006] [Citation(s) in RCA: 4] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 01/18/2023]
|
36
|
Hayat H, Sun A, Hayat H, Liu S, Talebloo N, Pinger C, Bishop JO, Gudi M, Dwan BF, Ma X, Zhao Y, Moore A, Wang P. Artificial Intelligence Analysis of Magnetic Particle Imaging for Islet Transplantation in a Mouse Model. Mol Imaging Biol 2021; 23:18-29. [PMID: 32833112 PMCID: PMC7785569 DOI: 10.1007/s11307-020-01533-5] [Citation(s) in RCA: 6] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/07/2020] [Revised: 08/06/2020] [Accepted: 08/12/2020] [Indexed: 12/13/2022]
Abstract
PURPOSE Current approaches to quantification of magnetic particle imaging (MPI) for cell-based therapy are thwarted by the lack of reliable, standardized methods of segmenting the signal from background in images. This calls for the development of artificial intelligence (AI) systems for MPI analysis. PROCEDURES We utilize a canonical algorithm in the domain of unsupervised machine learning, known as K-means++, to segment the regions of interest (ROI) of images and perform iron quantification analysis using a standard curve model. We generated in vitro, in vivo, and ex vivo data using islets and mouse models and applied the AI algorithm to gain insight into segmentation and iron prediction on these MPI data. In vitro models included imaging the VivoTrax-labeled islets in varying numbers. In vivo mouse models were generated through transplantation of increasing numbers of the labeled islets under the kidney capsule of mice. Ex vivo data were obtained from the MPI images of excised kidney grafts. RESULTS The K-means++ algorithms segmented the ROI of in vitro phantoms with minimal noise. A linear correlation between the islet numbers and the increasing prediction of total iron value (TIV) in the islets was observed. Segmentation results of the ROI of the in vivo MPI scans showed that with increasing number of transplanted islets, the signal intensity increased with linear trend. Upon segmenting the ROI of ex vivo data, a linear trend was observed in which increasing intensity of the ROI yielded increasing TIV of the islets. Through statistical evaluation of the algorithm performance via intraclass correlation coefficient validation, we observed excellent performance of K-means++-based model on segmentation and quantification analysis of MPI data. CONCLUSIONS We have demonstrated the ability of the K-means++-based model to provide a standardized method of segmentation and quantification of MPI scans in an islet transplantation mouse model.
Collapse
Affiliation(s)
- Hasaan Hayat
- Precision Health Program, Michigan State University, 766 Service Road, Rm. 2020, East Lansing, MI, 48823, USA
- Lyman Briggs College, Michigan State University, East Lansing, MI, USA
| | - Aixia Sun
- Precision Health Program, Michigan State University, 766 Service Road, Rm. 2020, East Lansing, MI, 48823, USA
- Department of Radiology, College of Human Medicine, Michigan State University, East Lansing, MI, USA
| | - Hanaan Hayat
- Lyman Briggs College, Michigan State University, East Lansing, MI, USA
- Institute for Quantitative Health Science and Engineering, Department of Biomedical Engineering, Michigan State University, East Lansing, MI, USA
| | - Sihai Liu
- Precision Health Program, Michigan State University, 766 Service Road, Rm. 2020, East Lansing, MI, 48823, USA
- Department of Radiology, College of Human Medicine, Michigan State University, East Lansing, MI, USA
- Department of Orthopedics, Beijing Charity Hospital, Capital Medical University, Beijing, China
| | - Nazanin Talebloo
- Precision Health Program, Michigan State University, 766 Service Road, Rm. 2020, East Lansing, MI, 48823, USA
- Department of Chemistry, College of Natural Science, Michigan State University, East Lansing, MI, USA
| | - Cody Pinger
- Institute for Quantitative Health Science and Engineering, Department of Biomedical Engineering, Michigan State University, East Lansing, MI, USA
| | - Jack Owen Bishop
- Precision Health Program, Michigan State University, 766 Service Road, Rm. 2020, East Lansing, MI, 48823, USA
- Department of Neuroscience, College of Natural Science, Michigan State University, East Lansing, MI, USA
| | - Mithil Gudi
- Precision Health Program, Michigan State University, 766 Service Road, Rm. 2020, East Lansing, MI, 48823, USA
- Lyman Briggs College, Michigan State University, East Lansing, MI, USA
| | - Bennett Francis Dwan
- Precision Health Program, Michigan State University, 766 Service Road, Rm. 2020, East Lansing, MI, 48823, USA
- College of Natural Science, Michigan State University, East Lansing, MI, USA
| | - Xiaohong Ma
- Precision Health Program, Michigan State University, 766 Service Road, Rm. 2020, East Lansing, MI, 48823, USA
- Department of Radiology, College of Human Medicine, Michigan State University, East Lansing, MI, USA
- Department of Radiology, National Cancer Center/Cancer Hospital, Chinese Academy of Medical Sciences and Peking Union Medical College, Beijing, China
| | - Yanfeng Zhao
- Precision Health Program, Michigan State University, 766 Service Road, Rm. 2020, East Lansing, MI, 48823, USA
- Department of Radiology, College of Human Medicine, Michigan State University, East Lansing, MI, USA
- Department of Radiology, National Cancer Center/Cancer Hospital, Chinese Academy of Medical Sciences and Peking Union Medical College, Beijing, China
| | - Anna Moore
- Precision Health Program, Michigan State University, 766 Service Road, Rm. 2020, East Lansing, MI, 48823, USA
- Department of Radiology, College of Human Medicine, Michigan State University, East Lansing, MI, USA
| | - Ping Wang
- Precision Health Program, Michigan State University, 766 Service Road, Rm. 2020, East Lansing, MI, 48823, USA.
- Department of Radiology, College of Human Medicine, Michigan State University, East Lansing, MI, USA.
| |
Collapse
|
37
|
Gryska E, Schneiderman J, Björkman-Burtscher I, Heckemann RA. Automatic brain lesion segmentation on standard magnetic resonance images: a scoping review. BMJ Open 2021; 11:e042660. [PMID: 33514580 PMCID: PMC7849889 DOI: 10.1136/bmjopen-2020-042660] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 07/11/2020] [Revised: 01/09/2021] [Accepted: 01/12/2021] [Indexed: 12/11/2022] Open
Abstract
OBJECTIVES Medical image analysis practices face challenges that can potentially be addressed with algorithm-based segmentation tools. In this study, we map the field of automatic MR brain lesion segmentation to understand the clinical applicability of prevalent methods and study designs, as well as challenges and limitations in the field. DESIGN Scoping review. SETTING Three databases (PubMed, IEEE Xplore and Scopus) were searched with tailored queries. Studies were included based on predefined criteria. Emerging themes during consecutive title, abstract, methods and whole-text screening were identified. The full-text analysis focused on materials, preprocessing, performance evaluation and comparison. RESULTS Out of 2990 unique articles identified through the search, 441 articles met the eligibility criteria, with an estimated growth rate of 10% per year. We present a general overview and trends in the field with regard to publication sources, segmentation principles used and types of lesions. Algorithms are predominantly evaluated by measuring the agreement of segmentation results with a trusted reference. Few articles describe measures of clinical validity. CONCLUSIONS The observed reporting practices leave room for improvement with a view to studying replication, method comparison and clinical applicability. To promote this improvement, we propose a list of recommendations for future studies in the field.
Collapse
Affiliation(s)
- Emilia Gryska
- Medical Radiation Sciences, Goteborgs universitet Institutionen for kliniska vetenskaper, Goteborg, Sweden
| | - Justin Schneiderman
- Sektionen för klinisk neurovetenskap, Goteborgs Universitet Institutionen for Neurovetenskap och fysiologi, Goteborg, Sweden
| | | | - Rolf A Heckemann
- Medical Radiation Sciences, Goteborgs universitet Institutionen for kliniska vetenskaper, Goteborg, Sweden
| |
Collapse
|
38
|
Automatic segmentation of white matter hyperintensities from brain magnetic resonance images in the era of deep learning and big data - A systematic review. Comput Med Imaging Graph 2021; 88:101867. [PMID: 33508567 DOI: 10.1016/j.compmedimag.2021.101867] [Citation(s) in RCA: 17] [Impact Index Per Article: 5.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/09/2020] [Revised: 12/23/2020] [Accepted: 12/31/2020] [Indexed: 11/20/2022]
Abstract
BACKGROUND White matter hyperintensities (WMH), of presumed vascular origin, are visible and quantifiable neuroradiological markers of brain parenchymal change. These changes may range from damage secondary to inflammation and other neurological conditions, through to healthy ageing. Fully automatic WMH quantification methods are promising, but still, traditional semi-automatic methods seem to be preferred in clinical research. We systematically reviewed the literature for fully automatic methods developed in the last five years, to assess what are considered state-of-the-art techniques, as well as trends in the analysis of WMH of presumed vascular origin. METHOD We registered the systematic review protocol with the International Prospective Register of Systematic Reviews (PROSPERO), registration number - CRD42019132200. We conducted the search for fully automatic methods developed from 2015 to July 2020 on Medline, Science direct, IEE Explore, and Web of Science. We assessed risk of bias and applicability of the studies using QUADAS 2. RESULTS The search yielded 2327 papers after removing 104 duplicates. After screening titles, abstracts and full text, 37 were selected for detailed analysis. Of these, 16 proposed a supervised segmentation method, 10 proposed an unsupervised segmentation method, and 11 proposed a deep learning segmentation method. Average DSC values ranged from 0.538 to 0.91, being the highest value obtained from an unsupervised segmentation method. Only four studies validated their method in longitudinal samples, and eight performed an additional validation using clinical parameters. Only 8/37 studies made available their methods in public repositories. CONCLUSIONS We found no evidence that favours deep learning methods over the more established k-NN, linear regression and unsupervised methods in this task. Data and code availability, bias in study design and ground truth generation influence the wider validation and applicability of these methods in clinical research.
Collapse
|
39
|
McKinley R, Wepfer R, Aschwanden F, Grunder L, Muri R, Rummel C, Verma R, Weisstanner C, Reyes M, Salmen A, Chan A, Wagner F, Wiest R. Simultaneous lesion and brain segmentation in multiple sclerosis using deep neural networks. Sci Rep 2021; 11:1087. [PMID: 33441684 PMCID: PMC7806997 DOI: 10.1038/s41598-020-79925-4] [Citation(s) in RCA: 30] [Impact Index Per Article: 10.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/23/2020] [Accepted: 12/04/2020] [Indexed: 12/12/2022] Open
Abstract
Segmentation of white matter lesions and deep grey matter structures is an important task in the quantification of magnetic resonance imaging in multiple sclerosis. In this paper we explore segmentation solutions based on convolutional neural networks (CNNs) for providing fast, reliable segmentations of lesions and grey-matter structures in multi-modal MR imaging, and the performance of these methods when applied to out-of-centre data. We trained two state-of-the-art fully convolutional CNN architectures on the 2016 MSSEG training dataset, which was annotated by seven independent human raters: a reference implementation of a 3D Unet, and a more recently proposed 3D-to-2D architecture (DeepSCAN). We then retrained those methods on a larger dataset from a single centre, with and without labels for other brain structures. We quantified changes in performance owing to dataset shift, and changes in performance by adding the additional brain-structure labels. We also compared performance with freely available reference methods. Both fully-convolutional CNN methods substantially outperform other approaches in the literature when trained and evaluated in cross-validation on the MSSEG dataset, showing agreement with human raters in the range of human inter-rater variability. Both architectures showed drops in performance when trained on single-centre data and tested on the MSSEG dataset. When trained with the addition of weak anatomical labels derived from Freesurfer, the performance of the 3D Unet degraded, while the performance of the DeepSCAN net improved. Overall, the DeepSCAN network predicting both lesion and anatomical labels was the best-performing network examined.
Collapse
Affiliation(s)
- Richard McKinley
- Support Center for Advanced Neuroimaging, University Institute for Diagnostic and Interventional Neuroradiology, Inselspital, Bern University Hospital, Bern, Switzerland.
| | - Rik Wepfer
- Support Center for Advanced Neuroimaging, University Institute for Diagnostic and Interventional Neuroradiology, Inselspital, Bern University Hospital, Bern, Switzerland
| | - Fabian Aschwanden
- Support Center for Advanced Neuroimaging, University Institute for Diagnostic and Interventional Neuroradiology, Inselspital, Bern University Hospital, Bern, Switzerland
| | - Lorenz Grunder
- Support Center for Advanced Neuroimaging, University Institute for Diagnostic and Interventional Neuroradiology, Inselspital, Bern University Hospital, Bern, Switzerland
| | - Raphaela Muri
- Support Center for Advanced Neuroimaging, University Institute for Diagnostic and Interventional Neuroradiology, Inselspital, Bern University Hospital, Bern, Switzerland
| | - Christian Rummel
- Support Center for Advanced Neuroimaging, University Institute for Diagnostic and Interventional Neuroradiology, Inselspital, Bern University Hospital, Bern, Switzerland
| | | | | | - Mauricio Reyes
- ARTORG Centre for Biomedical Engineering Research, University of Bern, Bern, Switzerland
| | - Anke Salmen
- University Clinic for Neurology, Inselspital, Bern University Hospital, Bern, Switzerland
| | - Andrew Chan
- University Clinic for Neurology, Inselspital, Bern University Hospital, Bern, Switzerland
| | - Franca Wagner
- Support Center for Advanced Neuroimaging, University Institute for Diagnostic and Interventional Neuroradiology, Inselspital, Bern University Hospital, Bern, Switzerland
| | - Roland Wiest
- Support Center for Advanced Neuroimaging, University Institute for Diagnostic and Interventional Neuroradiology, Inselspital, Bern University Hospital, Bern, Switzerland
| |
Collapse
|
40
|
Fenneteau A, Bourdon P, Helbert D, Fernandez-Maloigne C, Habas C, Guillevin R. Investigating efficient CNN architecture for multiple sclerosis lesion segmentation. J Med Imaging (Bellingham) 2021; 8:014504. [PMID: 33569506 PMCID: PMC7867032 DOI: 10.1117/1.jmi.8.1.014504] [Citation(s) in RCA: 4] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/19/2020] [Accepted: 01/11/2021] [Indexed: 11/14/2022] Open
Abstract
Purpose: The automatic segmentation of multiple sclerosis lesions in magnetic resonance imaging has the potential to reduce radiologists' efforts on a daily time-consuming task and to bring more reproducibility. Almost all new segmentation techniques make use of convolutional neural networks with their own different architecture. Architectural choices are rarely explained. We aimed at presenting the relevance of a U-net-like architecture for our specific task and at building an efficient and simple model. Approach: An experimental study was performed by observing the impact of applying different mutations and deletions to a simple U-net-like architecture. Results: The power of the U-net architecture is explained by the joint benefits of using an encoder-decoder architecture and by linking them with long skip connections. Augmenting the number of convolutional layers and decreasing the number of feature maps allowed us to build an exceptionally light and competitive architecture, the minimally parameterized U-net (MPU-net), with only ∼ 30,000 parameters. Conclusion: The empirical study of the U-net has led to a better understanding of its architecture. It has guided the building of the MPU-net, a model far less parameterized than others (at least by a factor of seven). This neural network achieves a human-level segmentation of multiple sclerosis lesions on fluid-attenuated inversion recovery images only. It shows that this segmentation task does not necessitate overly complicated models to be achieved. This gives the opportunity to build more explainable models that can help such methods to be adopted in a clinical environment.
Collapse
Affiliation(s)
- Alexandre Fenneteau
- Siemens Healthcare, Saint Denis, France
- University of Poitiers, UMR CNRS 7252, XLIM Laboratory, Poitiers, France
- University and Hospital of Poitiers, I3M, Common Laboratory CNRS-Siemens, Poitiers, France
| | - Pascal Bourdon
- University of Poitiers, UMR CNRS 7252, XLIM Laboratory, Poitiers, France
- University and Hospital of Poitiers, I3M, Common Laboratory CNRS-Siemens, Poitiers, France
| | - David Helbert
- University of Poitiers, UMR CNRS 7252, XLIM Laboratory, Poitiers, France
- University and Hospital of Poitiers, I3M, Common Laboratory CNRS-Siemens, Poitiers, France
| | - Christine Fernandez-Maloigne
- University of Poitiers, UMR CNRS 7252, XLIM Laboratory, Poitiers, France
- University and Hospital of Poitiers, I3M, Common Laboratory CNRS-Siemens, Poitiers, France
| | - Christophe Habas
- University and Hospital of Poitiers, I3M, Common Laboratory CNRS-Siemens, Poitiers, France
- Quinze-Vingts Hospital, Department of Neuroimaging, Paris, France
| | - Rémy Guillevin
- University and Hospital of Poitiers, I3M, Common Laboratory CNRS-Siemens, Poitiers, France
- Poitiers University Hospital, CHU, Poitiers, France
- University of Poitiers, UMR CNRS 7348, DACTIM-MIS/LMA Laboratory, Poitiers, France
| |
Collapse
|
41
|
Zeng C, Gu L, Liu Z, Zhao S. Review of Deep Learning Approaches for the Segmentation of Multiple Sclerosis Lesions on Brain MRI. Front Neuroinform 2020; 14:610967. [PMID: 33328949 PMCID: PMC7714963 DOI: 10.3389/fninf.2020.610967] [Citation(s) in RCA: 34] [Impact Index Per Article: 8.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/28/2020] [Accepted: 10/26/2020] [Indexed: 11/18/2022] Open
Abstract
In recent years, there have been multiple works of literature reviewing methods for automatically segmenting multiple sclerosis (MS) lesions. However, there is no literature systematically and individually review deep learning-based MS lesion segmentation methods. Although the previous review also included methods based on deep learning, there are some methods based on deep learning that they did not review. In addition, their review of deep learning methods did not go deep into the specific categories of Convolutional Neural Network (CNN). They only reviewed these methods in a generalized form, such as supervision strategy, input data handling strategy, etc. This paper presents a systematic review of the literature in automated multiple sclerosis lesion segmentation based on deep learning. Algorithms based on deep learning reviewed are classified into two categories through their CNN style, and their strengths and weaknesses will also be given through our investigation and analysis. We give a quantitative comparison of the methods reviewed through two metrics: Dice Similarity Coefficient (DSC) and Positive Predictive Value (PPV). Finally, the future direction of the application of deep learning in MS lesion segmentation will be discussed.
Collapse
Affiliation(s)
- Chenyi Zeng
- School of Intelligent Systems Engineering, Sun Yat-Sen University, Guangzhou, China
| | - Lin Gu
- RIKEN AIP, Tokyo, Japan
- The University of Tokyo, Tokyo, Japan
| | - Zhenzhong Liu
- Tianjin Key Laboratory for Advanced Mechatronic System Design and Intelligent Control, School of Mechanical Engineering, Tianjin University of Technology, Tianjin, China
- National Demonstration Center for Experimental Mechanical and Electrical Engineering Education, Tianjin University of Technology, Tianjin, China
| | - Shen Zhao
- School of Intelligent Systems Engineering, Sun Yat-Sen University, Guangzhou, China
| |
Collapse
|
42
|
A contrast-adaptive method for simultaneous whole-brain and lesion segmentation in multiple sclerosis. Neuroimage 2020; 225:117471. [PMID: 33099007 PMCID: PMC7856304 DOI: 10.1016/j.neuroimage.2020.117471] [Citation(s) in RCA: 43] [Impact Index Per Article: 10.8] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/11/2020] [Revised: 10/12/2020] [Accepted: 10/16/2020] [Indexed: 12/24/2022] Open
Abstract
Here we present a method for the simultaneous segmentation of white matter lesions and normal-appearing neuroanatomical structures from multi-contrast brain MRI scans of multiple sclerosis patients. The method integrates a novel model for white matter lesions into a previously validated generative model for whole-brain segmentation. By using separate models for the shape of anatomical structures and their appearance in MRI, the algorithm can adapt to data acquired with different scanners and imaging protocols without retraining. We validate the method using four disparate datasets, showing robust performance in white matter lesion segmentation while simultaneously segmenting dozens of other brain structures. We further demonstrate that the contrast-adaptive method can also be safely applied to MRI scans of healthy controls, and replicate previously documented atrophy patterns in deep gray matter structures in MS. The algorithm is publicly available as part of the open-source neuroimaging package FreeSurfer.
Collapse
|
43
|
|
44
|
Krüger J, Opfer R, Gessert N, Ostwaldt AC, Manogaran P, Kitzler HH, Schlaefer A, Schippling S. Fully automated longitudinal segmentation of new or enlarged multiple sclerosis lesions using 3D convolutional neural networks. NEUROIMAGE-CLINICAL 2020; 28:102445. [PMID: 33038667 PMCID: PMC7554211 DOI: 10.1016/j.nicl.2020.102445] [Citation(s) in RCA: 11] [Impact Index Per Article: 2.8] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 05/08/2020] [Revised: 09/18/2020] [Accepted: 09/20/2020] [Indexed: 12/21/2022]
Abstract
A fully automated segmentation of new or enlarged multiple sclerosis (MS) lesions. 3D convolutional neural network (CNN) with U-net-like encoder-decoder architecture. Simultaneous processing of baseline and follow-up scan of the same patient. Trained on 3253 patient data from over 103 different MR scanners. Fast (<1min), robust algorithm with segmentation results in inter-rater variability.
The quantification of new or enlarged lesions from follow-up MRI scans is an important surrogate of clinical disease activity in patients with multiple sclerosis (MS). Not only is manual segmentation time consuming, but inter-rater variability is high. Currently, only a few fully automated methods are available. We address this gap in the field by employing a 3D convolutional neural network (CNN) with encoder-decoder architecture for fully automatic longitudinal lesion segmentation. Input data consist of two fluid attenuated inversion recovery (FLAIR) images (baseline and follow-up) per patient. Each image is entered into the encoder and the feature maps are concatenated and then fed into the decoder. The output is a 3D mask indicating new or enlarged lesions (compared to the baseline scan). The proposed method was trained on 1809 single point and 1444 longitudinal patient data sets and then validated on 185 independent longitudinal data sets from two different scanners. From the two validation data sets, manual segmentations were available from three experienced raters, respectively. The performance of the proposed method was compared to the open source Lesion Segmentation Toolbox (LST), which is a current state-of-art longitudinal lesion segmentation method. The mean lesion-wise inter-rater sensitivity was 62%, while the mean inter-rater number of false positive (FP) findings was 0.41 lesions per case. The two validated algorithms showed a mean sensitivity of 60% (CNN), 46% (LST) and a mean FP of 0.48 (CNN), 1.86 (LST) per case. Sensitivity and number of FP were not significantly different (p < 0.05) between the CNN and manual raters. New or enlarged lesions counted by the CNN algorithm appeared to be comparable with manual expert ratings. The proposed algorithm seems to outperform currently available approaches, particularly LST. The high inter-rater variability in case of manual segmentation indicates the complexity of identifying new or enlarged lesions. An automated CNN-based approach can quickly provide an independent and deterministic assessment of new or enlarged lesions from baseline to follow-up scans with acceptable reliability.
Collapse
Affiliation(s)
| | | | - Nils Gessert
- Institute of Medical Technology, Hamburg University of Technology, Germany
| | | | - Praveena Manogaran
- Neuroimmunology and Multiple Sclerosis Research, Department of Neurology, University Hospital Zurich and University of Zurich, Switzerland; Department of Information Technology and Electrical Engineering, Swiss Federal Institute of Technology, Zurich, Switzerland
| | - Hagen H Kitzler
- Institute of Diagnostic and Interventional Neuroradiology, University Hospital Carl Gustav Carus, Technische Universität Dresden, Germany
| | | | - Sven Schippling
- Neuroimmunology and Multiple Sclerosis Research, Department of Neurology, University Hospital Zurich and University of Zurich, Switzerland; Neuroscience Center Zurich, University of Zurich and Federal Institute of Technology (ETH), Zurich, Switzerland
| |
Collapse
|
45
|
Honce JM, Nair KV, Hoyt BD, Seale RA, Sillau S, Engebretson E, Schurr B, Corboy JR, Vollmer TL, Alvarez E. Brain Atrophy Rates for Stable Multiple Sclerosis Patients on Long-Term Fingolimod versus Glatiramer Acetate. Front Neurol 2020; 11:1045. [PMID: 33071934 PMCID: PMC7538802 DOI: 10.3389/fneur.2020.01045] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/22/2020] [Accepted: 08/10/2020] [Indexed: 12/31/2022] Open
Abstract
Background: Clinically stable multiple sclerosis (MS) patients on long-term therapy often have negligible acute inflammation on MRI. Brain atrophy may provide insight into subclinical disease progression in such populations. Objective: This study aims to compare brain atrophy for age- and gender-matched MS patients treated for >2 years with fingolimod (FTY) or glatiramer acetate (GA), examining brain volume, cognition, and patient-reported outcomes (PROs). Methods: Stable relapsing-MS patients, age 18-60, on FTY or GA for >2 years were followed up for 2 years. MRI brain and lesion volumes, cognitive measures, and PROs were collected at baseline and annually. Results: Forty-four FTY and forty-three GA patients completed baseline and year 2 visits. No differences in age, gender, or education were observed. Median EDSS was 2.0GA and 2.5FTY (p = 0.22). Treatment duration was longer for GA, 6.50GA vs. 3.73FTY years (p < 0.001). Baseline geometric mean T2LV were different, GA = 1,009.29 cm3 vs. FTY = 2,404.67 cm3 (p = 0.0071). Baseline brain volumes were similar, GA = 1,508 cm3 vs. FTY = 1,489 cm3 (p = 0.2381). Annualized atrophy rates, adjusted for baseline and at mean baseline value, were GA = -0.2775% vs. FTY = -0.2967% (p = 0.7979). No differences in cognitive measures or PROs were observed. Conclusions: Stable MS patients on long-term treatment with FTY and GA have similar brain volume loss rates. Differences in baseline disease severity may suggest patients with more aggressive disease treated with FTY may achieve similar brain volume loss rates as patients with milder baseline disease on GA.
Collapse
Affiliation(s)
- Justin M Honce
- Department of Radiology, University of Colorado Hospital, Aurora, CO, United States
| | - Kavita V Nair
- Department of Clinical Pharmacy, University of Colorado, Aurora, CO, United States.,Department of Neurology, University of Colorado and Rocky Mountain Multiple Sclerosis Center at the University of Colorado, Aurora, CO, United States
| | - Brian D Hoyt
- Department of Neurology, University of Colorado and Rocky Mountain Multiple Sclerosis Center at the University of Colorado, Aurora, CO, United States.,Department of Neurosurgery, University of Colorado, Aurora, CO, United States
| | - Rebecca A Seale
- Department of Neurology, University of Colorado and Rocky Mountain Multiple Sclerosis Center at the University of Colorado, Aurora, CO, United States
| | - Stefan Sillau
- Department of Neurology, University of Colorado and Rocky Mountain Multiple Sclerosis Center at the University of Colorado, Aurora, CO, United States
| | - Eric Engebretson
- Department of Neurology, University of Colorado and Rocky Mountain Multiple Sclerosis Center at the University of Colorado, Aurora, CO, United States
| | - Brittany Schurr
- Department of Neurology, University of Colorado and Rocky Mountain Multiple Sclerosis Center at the University of Colorado, Aurora, CO, United States
| | - John R Corboy
- Department of Neurology, University of Colorado and Rocky Mountain Multiple Sclerosis Center at the University of Colorado, Aurora, CO, United States
| | - Timothy L Vollmer
- Department of Neurology, University of Colorado and Rocky Mountain Multiple Sclerosis Center at the University of Colorado, Aurora, CO, United States
| | - Enrique Alvarez
- Department of Neurology, University of Colorado and Rocky Mountain Multiple Sclerosis Center at the University of Colorado, Aurora, CO, United States
| |
Collapse
|
46
|
Gessert N, Krüger J, Opfer R, Ostwaldt AC, Manogaran P, Kitzler HH, Schippling S, Schlaefer A. Multiple sclerosis lesion activity segmentation with attention-guided two-path CNNs. Comput Med Imaging Graph 2020; 84:101772. [DOI: 10.1016/j.compmedimag.2020.101772] [Citation(s) in RCA: 13] [Impact Index Per Article: 3.3] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/12/2020] [Revised: 07/20/2020] [Accepted: 07/31/2020] [Indexed: 10/23/2022]
|
47
|
Ma S, Huang Y, Che X, Gu R. Faster RCNN-based detection of cervical spinal cord injury and disc degeneration. J Appl Clin Med Phys 2020; 21:235-243. [PMID: 32797664 PMCID: PMC7497907 DOI: 10.1002/acm2.13001] [Citation(s) in RCA: 11] [Impact Index Per Article: 2.8] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/17/2019] [Revised: 05/14/2020] [Accepted: 07/17/2020] [Indexed: 12/19/2022] Open
Abstract
Magnetic resonance imaging (MRI) can indirectly reflect microscopic changes in lesions on the spinal cord; however, the application of deep learning to MRI to classify and detect lesions for cervical spinal cord diseases has not been sufficiently explored. In this study, we implemented a deep neural network for MRI to detect lesions caused by cervical diseases. We retrospectively reviewed the MRI of 1,500 patients irrespective of whether they had cervical diseases. The patients were treated in our hospital from January 2013 to December 2018. We randomly divided the MRI data into three groups of datasets: disc group (800 datasets), injured group (200 datasets), and normal group (500 datasets). We designed the relevant parameters and used a faster-region convolutional neural network (Faster R-CNN) combined with a backbone convolutional feature extractor using the ResNet-50 and VGG-16 networks, to detect lesions during MRI. Experimental results showed that the prediction accuracy and speed of Faster R-CNN with ResNet-50 and VGG-16 in detecting and recognizing lesions from a cervical spinal cord MRI were satisfactory. The mean average precisions (mAPs) for Faster R-CNN with ResNet-50 and VGG-16 were 88.6 and 72.3%, respectively, and the testing times was 0.22 and 0.24 s/image, respectively. Faster R-CNN can identify and detect lesions from cervical MRIs. To some extent, it may aid radiologists and spine surgeons in their diagnoses. The results of our study can provide motivation for future research to combine medical imaging and deep learning.
Collapse
Affiliation(s)
- Shaolong Ma
- Department of orthopedicsChina‐Japan Union Hospital of Jilin UniversityChangchun, JilinChina
| | - Yang Huang
- College of Computer Science and TechnologyJilin universityChangchunChina
| | - Xiangjiu Che
- College of Computer Science and TechnologyJilin universityChangchunChina
| | - Rui Gu
- Department of orthopedicsChina‐Japan Union Hospital of Jilin UniversityChangchun, JilinChina
| |
Collapse
|
48
|
Kim M, Jewells V. Multimodal Image Analysis for Assessing Multiple Sclerosis and Future Prospects Powered by Artificial Intelligence. Semin Ultrasound CT MR 2020; 41:309-318. [DOI: 10.1053/j.sult.2020.02.005] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/11/2022]
|
49
|
Carass A, Roy S, Gherman A, Reinhold JC, Jesson A, Arbel T, Maier O, Handels H, Ghafoorian M, Platel B, Birenbaum A, Greenspan H, Pham DL, Crainiceanu CM, Calabresi PA, Prince JL, Roncal WRG, Shinohara RT, Oguz I. Evaluating White Matter Lesion Segmentations with Refined Sørensen-Dice Analysis. Sci Rep 2020; 10:8242. [PMID: 32427874 PMCID: PMC7237671 DOI: 10.1038/s41598-020-64803-w] [Citation(s) in RCA: 56] [Impact Index Per Article: 14.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/03/2019] [Accepted: 04/20/2020] [Indexed: 11/09/2022] Open
Abstract
The Sørensen-Dice index (SDI) is a widely used measure for evaluating medical image segmentation algorithms. It offers a standardized measure of segmentation accuracy which has proven useful. However, it offers diminishing insight when the number of objects is unknown, such as in white matter lesion segmentation of multiple sclerosis (MS) patients. We present a refinement for finer grained parsing of SDI results in situations where the number of objects is unknown. We explore these ideas with two case studies showing what can be learned from our two presented studies. Our first study explores an inter-rater comparison, showing that smaller lesions cannot be reliably identified. In our second case study, we demonstrate fusing multiple MS lesion segmentation algorithms based on the insights into the algorithms provided by our analysis to generate a segmentation that exhibits improved performance. This work demonstrates the wealth of information that can be learned from refined analysis of medical image segmentations.
Collapse
Affiliation(s)
- Aaron Carass
- Department of Electrical and Computer Engineering, The Johns Hopkins University, Baltimore, MD, 21218, USA.
- Department of Computer Science, The Johns Hopkins University, Baltimore, MD, 21218, USA.
| | - Snehashis Roy
- CNRM, The Henry M. Jackson Foundation for the Advancement of Military Medicine, Bethesda, MD, 20817, USA
| | - Adrian Gherman
- Department of Biostatistics, The Johns Hopkins University, Baltimore, MD, 21205, USA
| | - Jacob C Reinhold
- Department of Electrical and Computer Engineering, The Johns Hopkins University, Baltimore, MD, 21218, USA
| | - Andrew Jesson
- Centre For Intelligent Machines, McGill University, Montréal, QC, H3A 0E9, Canada
| | - Tal Arbel
- Centre For Intelligent Machines, McGill University, Montréal, QC, H3A 0E9, Canada
| | - Oskar Maier
- Institute of Medical Informatics, University of Lübeck, 23538, Lübeck, Germany
| | - Heinz Handels
- Institute of Medical Informatics, University of Lübeck, 23538, Lübeck, Germany
| | - Mohsen Ghafoorian
- Institute for Computing and Information Sciences, Radboud University, 6525, HP, Nijmegen, Netherlands
| | - Bram Platel
- Diagnostic Image Analysis Group, Radboud University Medical Center, 6525, GA, Nijmegen, Netherlands
| | - Ariel Birenbaum
- Department of Electrical Engineering, Tel-Aviv University, Tel-Aviv, 69978, Israel
| | - Hayit Greenspan
- Department of Biomedical Engineering, Tel-Aviv University, Tel-Aviv, 69978, Israel
| | - Dzung L Pham
- CNRM, The Henry M. Jackson Foundation for the Advancement of Military Medicine, Bethesda, MD, 20817, USA
| | - Ciprian M Crainiceanu
- Department of Biostatistics, The Johns Hopkins University, Baltimore, MD, 21205, USA
| | - Peter A Calabresi
- Department of Neurology, The Johns Hopkins University School of Medicine, Baltimore, MD, 21287, USA
| | - Jerry L Prince
- Department of Electrical and Computer Engineering, The Johns Hopkins University, Baltimore, MD, 21218, USA
- Department of Computer Science, The Johns Hopkins University, Baltimore, MD, 21218, USA
| | - William R Gray Roncal
- Department of Computer Science, The Johns Hopkins University, Baltimore, MD, 21218, USA
| | - Russell T Shinohara
- Penn Statistics in Imaging and Visualization Center, Department of Biostatistics & Epidemiology, University of Pennsylvania, Philadelphia, PA, 19104, USA
- Center for Biomedical Image Computing and Analytics, Department of Radiology, University of Pennsylvania, Philadelphia, PA, 19104, USA
| | - Ipek Oguz
- Department of Electrical Engineering and Computer Science, Vanderbilt University, Nashville, TN, 37203, USA
| |
Collapse
|
50
|
Valcarcel AM, Muschelli J, Pham DL, Martin ML, Yushkevich P, Brandstadter R, Patterson KR, Schindler MK, Calabresi PA, Bakshi R, Shinohara RT. TAPAS: A Thresholding Approach for Probability Map Automatic Segmentation in Multiple Sclerosis. Neuroimage Clin 2020; 27:102256. [PMID: 32428847 PMCID: PMC7236059 DOI: 10.1016/j.nicl.2020.102256] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/12/2019] [Revised: 03/24/2020] [Accepted: 03/25/2020] [Indexed: 11/15/2022]
Abstract
Total brain white matter lesion (WML) volume is the most widely established magnetic resonance imaging (MRI) outcome measure in studies of multiple sclerosis (MS). To estimate WML volume, there are a number of automatic segmentation methods available, yet manual delineation remains the gold standard approach. Automatic approaches often yield a probability map to which a threshold is applied to create lesion segmentation masks. Unfortunately, few approaches systematically determine the threshold employed; many methods use a manually selected threshold, thus introducing human error and bias into the automated procedure. In this study, we propose and validate an automatic thresholding algorithm, Thresholding Approach for Probability Map Automatic Segmentation in Multiple Sclerosis (TAPAS), to obtain subject-specific threshold estimates for probability map automatic segmentation of T2-weighted (T2) hyperintense WMLs. Using multimodal MRI, the proposed method applies an automatic segmentation algorithm to obtain probability maps. We obtain the true subject-specific threshold that maximizes the Sørensen-Dice similarity coefficient (DSC). Then the subject-specific thresholds are modeled on a naive estimate of volume using a generalized additive model. Applying this model, we predict a subject-specific threshold in data not used for training. We ran a Monte Carlo-resampled split-sample cross-validation (100 validation sets) using two data sets: the first obtained from the Johns Hopkins Hospital (JHH) on a Philips 3 Tesla (3T) scanner (n = 94) and a second collected at the Brigham and Women's Hospital (BWH) using a Siemens 3T scanner (n = 40). By means of the proposed automated technique, in the JHH data we found an average reduction in subject-level absolute error of 0.1 mL per one mL increase in manual volume. Using Bland-Altman analysis, we found that volumetric bias associated with group-level thresholding was mitigated when applying TAPAS. The BWH data showed similar absolute error estimates using group-level thresholding or TAPAS likely since Bland-Altman analyses indicated no systematic biases associated with group or TAPAS volume estimates. The current study presents the first validated fully automated method for subject-specific threshold prediction to segment brain lesions.
Collapse
Affiliation(s)
- Alessandra M Valcarcel
- Penn Statistics in Imaging and Visualization Center, Department of Biostatistics, Epidemiology, and Informatics, Perelman School of Medicine, University of Pennsylvania, Philadelphia, PA 19104, United States.
| | - John Muschelli
- Department of Biostatistics, Bloomberg School of Public Health, Johns Hopkins University, Baltimore, MD 21287, United States
| | - Dzung L Pham
- Henry M. Jackson Foundation for the Advancement of Military Medicine, Bethesda, MD 20892, United States
| | - Melissa Lynne Martin
- Penn Statistics in Imaging and Visualization Center, Department of Biostatistics, Epidemiology, and Informatics, Perelman School of Medicine, University of Pennsylvania, Philadelphia, PA 19104, United States
| | - Paul Yushkevich
- Penn Image Computing and Science Laboratory (PICSL), Department of Radiology, University of Pennsylvania, Philadelphia, PA 19104, United States
| | - Rachel Brandstadter
- Department of Neurology, Perelman School of Medicine, University of Pennsylvania, Philadelphia, PA 19104, United States
| | - Kristina R Patterson
- Department of Neurology, Perelman School of Medicine, University of Pennsylvania, Philadelphia, PA 19104, United States
| | - Matthew K Schindler
- Department of Neurology, Perelman School of Medicine, University of Pennsylvania, Philadelphia, PA 19104, United States
| | - Peter A Calabresi
- Department of Neurology, School of Medicine Johns Hopkins University, Baltimore, MD 21287, United States
| | - Rohit Bakshi
- Department of Neurology, Brigham Women's Hospital, Harvard Medical School, Boston, MA 02115, United States; Department of Radiology, Brigham Women's Hospital, Harvard Medical School, Boston, MA 02115, United States
| | - Russell T Shinohara
- Penn Statistics in Imaging and Visualization Center, Department of Biostatistics, Epidemiology, and Informatics, Perelman School of Medicine, University of Pennsylvania, Philadelphia, PA 19104, United States; Center for Biomedical Image Computing and Analytics (CBICA), Department of Radiology, Perelman School of Medicine, University of Pennsylvania, Philadelphia, PA 19104, United States
| |
Collapse
|