1
|
Huang L, Shao Y, Yang H, Guo C, Wang Y, Zhao Z, Gong Y. A joint model for lesion segmentation and classification of MS and NMOSD. Front Neurosci 2024; 18:1351387. [PMID: 38863883 PMCID: PMC11166028 DOI: 10.3389/fnins.2024.1351387] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/28/2023] [Accepted: 05/01/2024] [Indexed: 06/13/2024] Open
Abstract
Introduction Multiple sclerosis (MS) and neuromyelitis optic spectrum disorder (NMOSD) are mimic autoimmune diseases of the central nervous system with a very high disability rate. Their clinical symptoms and imaging findings are similar, making it difficult to diagnose and differentiate. Existing research typically employs the T2-weighted fluid-attenuated inversion recovery (T2-FLAIR) MRI imaging technique to focus on a single task in MS and NMOSD lesion segmentation or disease classification, while ignoring the collaboration between the tasks. Methods To make full use of the correlation between lesion segmentation and disease classification tasks of MS and NMOSD, so as to improve the accuracy and speed of the recognition and diagnosis of MS and NMOSD, a joint model is proposed in this study. The joint model primarily comprises three components: an information-sharing subnetwork, a lesion segmentation subnetwork, and a disease classification subnetwork. Among them, the information-sharing subnetwork adopts a dualbranch structure composed of a convolution module and a Swin Transformer module to extract local and global features, respectively. These features are then input into the lesion segmentation subnetwork and disease classification subnetwork to obtain results for both tasks simultaneously. In addition, to further enhance the mutual guidance between the tasks, this study proposes two information interaction methods: a lesion guidance module and a crosstask loss function. Furthermore, the lesion location maps provide interpretability for the diagnosis process of the deep learning model. Results The joint model achieved a Dice similarity coefficient (DSC) of 74.87% on the lesion segmentation task and accuracy (ACC) of 92.36% on the disease classification task, demonstrating its superior performance. By setting up ablation experiments, the effectiveness of information sharing and interaction between tasks is verified. Discussion The results show that the joint model can effectively improve the performance of the two tasks.
Collapse
Affiliation(s)
- Lan Huang
- College of Computer Science and Technology, Jilin University, Changchun, China
| | - Yangguang Shao
- College of Computer Science and Technology, Jilin University, Changchun, China
| | - Hui Yang
- Public Computer Education and Research Center, Jilin University, Changchun, China
| | - Chunjie Guo
- Department of Radiology, The First Hospital of Jilin University, Changchun, China
| | - Yan Wang
- College of Computer Science and Technology, Jilin University, Changchun, China
| | - Ziqi Zhao
- College of Computer Science and Technology, Jilin University, Changchun, China
| | - Yingchun Gong
- College of Computer Science and Technology, Jilin University, Changchun, China
| |
Collapse
|
2
|
Demuth S, Paris J, Faddeenkov I, De Sèze J, Gourraud PA. Clinical applications of deep learning in neuroinflammatory diseases: A scoping review. Rev Neurol (Paris) 2024:S0035-3787(24)00522-8. [PMID: 38772806 DOI: 10.1016/j.neurol.2024.04.004] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/18/2024] [Revised: 03/26/2024] [Accepted: 04/09/2024] [Indexed: 05/23/2024]
Abstract
BACKGROUND Deep learning (DL) is an artificial intelligence technology that has aroused much excitement for predictive medicine due to its ability to process raw data modalities such as images, text, and time series of signals. OBJECTIVES Here, we intend to give the clinical reader elements to understand this technology, taking neuroinflammatory diseases as an illustrative use case of clinical translation efforts. We reviewed the scope of this rapidly evolving field to get quantitative insights about which clinical applications concentrate the efforts and which data modalities are most commonly used. METHODS We queried the PubMed database for articles reporting DL algorithms for clinical applications in neuroinflammatory diseases and the radiology.healthairegister.com website for commercial algorithms. RESULTS The review included 148 articles published between 2018 and 2024 and five commercial algorithms. The clinical applications could be grouped as computer-aided diagnosis, individual prognosis, functional assessment, the segmentation of radiological structures, and the optimization of data acquisition. Our review highlighted important discrepancies in efforts. The segmentation of radiological structures and computer-aided diagnosis currently concentrate most efforts with an overrepresentation of imaging. Various model architectures have addressed different applications, relatively low volume of data, and diverse data modalities. We report the high-level technical characteristics of the algorithms and synthesize narratively the clinical applications. Predictive performances and some common a priori on this topic are finally discussed. CONCLUSION The currently reported efforts position DL as an information processing technology, enhancing existing modalities of paraclinical investigations and bringing perspectives to make innovative ones actionable for healthcare.
Collapse
Affiliation(s)
- S Demuth
- Inserm U1064, CR2TI - Center for Research in Transplantation and Translational Immunology, Nantes University, 44000 Nantes, France; Inserm U1119 : biopathologie de la myéline, neuroprotection et stratégies thérapeutiques, University of Strasbourg, 1, rue Eugène-Boeckel - CS 60026, 67084 Strasbourg, France.
| | - J Paris
- Inserm U1064, CR2TI - Center for Research in Transplantation and Translational Immunology, Nantes University, 44000 Nantes, France
| | - I Faddeenkov
- Inserm U1064, CR2TI - Center for Research in Transplantation and Translational Immunology, Nantes University, 44000 Nantes, France
| | - J De Sèze
- Inserm U1119 : biopathologie de la myéline, neuroprotection et stratégies thérapeutiques, University of Strasbourg, 1, rue Eugène-Boeckel - CS 60026, 67084 Strasbourg, France; Department of Neurology, University Hospital of Strasbourg, 1, avenue Molière, 67200 Strasbourg, France; Inserm CIC 1434 Clinical Investigation Center, University Hospital of Strasbourg, 1, avenue Molière, 67200 Strasbourg, France
| | - P-A Gourraud
- Inserm U1064, CR2TI - Center for Research in Transplantation and Translational Immunology, Nantes University, 44000 Nantes, France; "Data clinic", Department of Public Health, University Hospital of Nantes, Nantes, France
| |
Collapse
|
3
|
Liu Y, Nezami FR, Edelman ER. A transformer-based pyramid network for coronary calcified plaque segmentation in intravascular optical coherence tomography images. Comput Med Imaging Graph 2024; 113:102347. [PMID: 38341945 PMCID: PMC11225546 DOI: 10.1016/j.compmedimag.2024.102347] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/21/2023] [Revised: 01/31/2024] [Accepted: 01/31/2024] [Indexed: 02/13/2024]
Abstract
Characterizing coronary calcified plaque (CCP) provides essential insight into diagnosis and treatment of atherosclerosis. Intravascular optical coherence tomography (OCT) offers significant advantages for detecting CCP and even automated segmentation with recent advances in deep learning techniques. Most of current methods have achieved promising results by adopting existing convolution neural networks (CNNs) in computer vision domain. However, their performance can be detrimentally affected by unseen plaque patterns and artifacts due to inherent limitation of CNNs in contextual reasoning. To overcome this obstacle, we proposed a Transformer-based pyramid network called AFS-TPNet for robust, end-to-end segmentation of CCP from OCT images. Its encoder is built upon CSWin Transformer architecture, allowing for better perceptual understanding of calcified arteries at a higher semantic level. Specifically, an augmented feature split (AFS) module and residual convolutional position encoding (RCPE) mechanism are designed to effectively enhance the capability of Transformer in capturing both fine-grained features and global contexts. Extensive experiments showed that AFS-TPNet trained using Lovasz Loss achieved superior performance in segmentation CCP under various contexts, surpassing prior state-of-the-art CNN and Transformer architectures by more than 6.58% intersection over union (IoU) score. The application of this promising method to extract CCP features is expected to enhance clinical intervention and translational research using OCT.
Collapse
Affiliation(s)
- Yiqing Liu
- Institute for Medical Engineering and Science, Massachusetts Institute of Technology, Cambridge, MA 02139, USA.
| | - Farhad R Nezami
- Institute for Medical Engineering and Science, Massachusetts Institute of Technology, Cambridge, MA 02139, USA; Division of Cardiac Surgery, Brigham and Women's Hospital, Harvard Medical School, Boston, MA 02115, USA
| | - Elazer R Edelman
- Institute for Medical Engineering and Science, Massachusetts Institute of Technology, Cambridge, MA 02139, USA; Cardiovascular Division, Brigham and Women's Hospital, Harvard Medical School, Boston, MA 02115, USA
| |
Collapse
|
4
|
Naji Y, Mahdaoui M, Klevor R, Kissani N. Artificial Intelligence and Multiple Sclerosis: Up-to-Date Review. Cureus 2023; 15:e45412. [PMID: 37854769 PMCID: PMC10581506 DOI: 10.7759/cureus.45412] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Accepted: 09/17/2023] [Indexed: 10/20/2023] Open
Abstract
Multiple sclerosis (MS) remains a challenging neurological disorder for the clinician in terms of diagnosis and management. The growing integration of AI-based algorithms in healthcare offers a golden opportunity for clinicians and patients with MS. AI models are based on statistical analyses of large quantities of data from patients including "demographics, genetics, clinical and radiological presentation." These approaches are promising in the quest for greater diagnostic accuracy, tailored management plans, and better prognostication of disease. The use of AI in multiple sclerosis represents a paradigm shift in disease management. With ongoing advancements in AI technologies and the increasing availability of large-scale datasets, the potential for further innovation is immense. As AI continues to evolve, its integration into clinical practice will play a vital role in improving diagnostics, optimizing treatment strategies, and enhancing patient outcomes for MS. This review is about conducting a literature review to identify relevant studies on AI applications in MS. Only peer-reviewed studies published in the last four years have been selected. Data related to AI techniques, advancements, and implications are extracted. Through data analysis, key themes and tendencies are identified. The review presents a cohesive synthesis of the current state of AI and MS, highlighting potential implications and new advancements.
Collapse
Affiliation(s)
- Yahya Naji
- Neurology Department, REGNE Research Laboratory, Faculty of Medicine and Pharmacy, Ibn Zohr University, Agadir, MAR
- Neurology Department, Agadir University Hospital, Agadir, MAR
| | - Mohamed Mahdaoui
- Neurology Department, University Hospital Mohammed VI, Marrakech, MAR
- Neuroscience Research Laboratory, Faculty of Medicine and Pharmacy, Cadi Ayyad University, Marrakech, MAR
| | - Raymond Klevor
- Neurology Department, University Hospital Mohammed VI, Marrakech, MAR
- Neuroscience Research Laboratory, Faculty of Medicine and Pharmacy, Cadi Ayyad University, Marrakech, MAR
| | - Najib Kissani
- Neurology Department, University Hospital Mohammed VI, Marrakech, MAR
- Neuroscience Research Laboratory, Faculty of Medicine and Pharmacy, Cadi Ayyad University, Marrakech, MAR
| |
Collapse
|
5
|
Spagnolo F, Depeursinge A, Schädelin S, Akbulut A, Müller H, Barakovic M, Melie-Garcia L, Bach Cuadra M, Granziera C. How far MS lesion detection and segmentation are integrated into the clinical workflow? A systematic review. Neuroimage Clin 2023; 39:103491. [PMID: 37659189 PMCID: PMC10480555 DOI: 10.1016/j.nicl.2023.103491] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/02/2023] [Accepted: 08/04/2023] [Indexed: 09/04/2023]
Abstract
INTRODUCTION Over the past few years, the deep learning community has developed and validated a plethora of tools for lesion detection and segmentation in Multiple Sclerosis (MS). However, there is an important gap between validating models technically and clinically. To this end, a six-step framework necessary for the development, validation, and integration of quantitative tools in the clinic was recently proposed under the name of the Quantitative Neuroradiology Initiative (QNI). AIMS Investigate to what extent automatic tools in MS fulfill the QNI framework necessary to integrate automated detection and segmentation into the clinical neuroradiology workflow. METHODS Adopting the systematic Cochrane literature review methodology, we screened and summarised published scientific articles that perform automatic MS lesions detection and segmentation. We categorised the retrieved studies based on their degree of fulfillment of QNI's six-steps, which include a tool's technical assessment, clinical validation, and integration. RESULTS We found 156 studies; 146/156 (94%) fullfilled the first QNI step, 155/156 (99%) the second, 8/156 (5%) the third, 3/156 (2%) the fourth, 5/156 (3%) the fifth and only one the sixth. CONCLUSIONS To date, little has been done to evaluate the clinical performance and the integration in the clinical workflow of available methods for MS lesion detection/segmentation. In addition, the socio-economic effects and the impact on patients' management of such tools remain almost unexplored.
Collapse
Affiliation(s)
- Federico Spagnolo
- Translational Imaging in Neurology (ThINK) Basel, Department of Biomedical Engineering, Faculty of Medicine, University Hospital Basel and University of Basel, Basel, Switzerland; Department of Neurology, University Hospital Basel, Basel, Switzerland; Research Center for Clinical Neuroimmunology and Neuroscience Basel (RC2NB), University Hospital Basel and University of Basel, Basel, Switzerland; MedGIFT, Institute of Informatics, School of Management, HES-SO Valais-Wallis University of Applied Sciences and Arts Western Switzerland, Sierre, Switzerland
| | - Adrien Depeursinge
- MedGIFT, Institute of Informatics, School of Management, HES-SO Valais-Wallis University of Applied Sciences and Arts Western Switzerland, Sierre, Switzerland; Nuclear Medicine and Molecular Imaging Department, Lausanne University Hospital (CHUV) and University of Lausanne, Lausanne, Switzerland
| | - Sabine Schädelin
- Translational Imaging in Neurology (ThINK) Basel, Department of Biomedical Engineering, Faculty of Medicine, University Hospital Basel and University of Basel, Basel, Switzerland; Clinical Trial Unit, Department of Clinical Research, University Hospital Basel, University of Basel, Basel, Switzerland
| | - Aysenur Akbulut
- Translational Imaging in Neurology (ThINK) Basel, Department of Biomedical Engineering, Faculty of Medicine, University Hospital Basel and University of Basel, Basel, Switzerland; Ankara University School of Medicine, Ankara, Turkey
| | - Henning Müller
- MedGIFT, Institute of Informatics, School of Management, HES-SO Valais-Wallis University of Applied Sciences and Arts Western Switzerland, Sierre, Switzerland; The Sense Research and Innovation Center, Lausanne and Sion, Switzerland
| | - Muhamed Barakovic
- Translational Imaging in Neurology (ThINK) Basel, Department of Biomedical Engineering, Faculty of Medicine, University Hospital Basel and University of Basel, Basel, Switzerland; Department of Neurology, University Hospital Basel, Basel, Switzerland; Research Center for Clinical Neuroimmunology and Neuroscience Basel (RC2NB), University Hospital Basel and University of Basel, Basel, Switzerland
| | - Lester Melie-Garcia
- Translational Imaging in Neurology (ThINK) Basel, Department of Biomedical Engineering, Faculty of Medicine, University Hospital Basel and University of Basel, Basel, Switzerland; Department of Neurology, University Hospital Basel, Basel, Switzerland; Research Center for Clinical Neuroimmunology and Neuroscience Basel (RC2NB), University Hospital Basel and University of Basel, Basel, Switzerland
| | - Meritxell Bach Cuadra
- CIBM Center for Biomedical Imaging, Lausanne, Switzerland; Radiology Department, Lausanne University Hospital (CHUV) and University of Lausanne, Lausanne, Switzerland
| | - Cristina Granziera
- Translational Imaging in Neurology (ThINK) Basel, Department of Biomedical Engineering, Faculty of Medicine, University Hospital Basel and University of Basel, Basel, Switzerland; Department of Neurology, University Hospital Basel, Basel, Switzerland; Research Center for Clinical Neuroimmunology and Neuroscience Basel (RC2NB), University Hospital Basel and University of Basel, Basel, Switzerland.
| |
Collapse
|
6
|
Rondinella A, Crispino E, Guarnera F, Giudice O, Ortis A, Russo G, Di Lorenzo C, Maimone D, Pappalardo F, Battiato S. Boosting multiple sclerosis lesion segmentation through attention mechanism. Comput Biol Med 2023; 161:107021. [PMID: 37216775 DOI: 10.1016/j.compbiomed.2023.107021] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/19/2023] [Revised: 04/11/2023] [Accepted: 05/05/2023] [Indexed: 05/24/2023]
Abstract
Magnetic resonance imaging is a fundamental tool to reach a diagnosis of multiple sclerosis and monitoring its progression. Although several attempts have been made to segment multiple sclerosis lesions using artificial intelligence, fully automated analysis is not yet available. State-of-the-art methods rely on slight variations in segmentation architectures (e.g. U-Net, etc.). However, recent research has demonstrated how exploiting temporal-aware features and attention mechanisms can provide a significant boost to traditional architectures. This paper proposes a framework that exploits an augmented U-Net architecture with a convolutional long short-term memory layer and attention mechanism which is able to segment and quantify multiple sclerosis lesions detected in magnetic resonance images. Quantitative and qualitative evaluation on challenging examples demonstrated how the method outperforms previous state-of-the-art approaches, reporting an overall Dice score of 89% and also demonstrating robustness and generalization ability on never seen new test samples of a new dedicated under construction dataset.
Collapse
Affiliation(s)
- Alessia Rondinella
- Department of Mathematics and Computer Science, University of Catania, Viale Andrea Doria 6, Catania, 95125, Italy.
| | - Elena Crispino
- Department of Biomedical and Biotechnological Sciences, University of Catania, Via Santa Sofia 97, Catania, 95125, Italy
| | - Francesco Guarnera
- Department of Mathematics and Computer Science, University of Catania, Viale Andrea Doria 6, Catania, 95125, Italy
| | - Oliver Giudice
- Department of Mathematics and Computer Science, University of Catania, Viale Andrea Doria 6, Catania, 95125, Italy
| | - Alessandro Ortis
- Department of Mathematics and Computer Science, University of Catania, Viale Andrea Doria 6, Catania, 95125, Italy
| | - Giulia Russo
- Department of Drug and Health Sciences, University of Catania, Viale Andrea Doria 6, Catania, 95125, Italy
| | - Clara Di Lorenzo
- UOC Radiologia, ARNAS Garibaldi, P.zza S. Maria di Gesù, Catania, 95124, Italy
| | - Davide Maimone
- Centro Sclerosi Multipla, UOC Neurologia, ARNAS Garibaldi, P.zza S. Maria di Gesù, Catania, 95124, Italy
| | - Francesco Pappalardo
- Department of Drug and Health Sciences, University of Catania, Viale Andrea Doria 6, Catania, 95125, Italy
| | - Sebastiano Battiato
- Department of Mathematics and Computer Science, University of Catania, Viale Andrea Doria 6, Catania, 95125, Italy
| |
Collapse
|
7
|
Mendelsohn Z, Pemberton HG, Gray J, Goodkin O, Carrasco FP, Scheel M, Nawabi J, Barkhof F. Commercial volumetric MRI reporting tools in multiple sclerosis: a systematic review of the evidence. Neuroradiology 2023; 65:5-24. [PMID: 36331588 PMCID: PMC9816195 DOI: 10.1007/s00234-022-03074-w] [Citation(s) in RCA: 5] [Impact Index Per Article: 5.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/13/2022] [Accepted: 09/29/2022] [Indexed: 11/06/2022]
Abstract
PURPOSE MRI is integral to the diagnosis of multiple sclerosis (MS) and is important for clinical prognostication. Quantitative volumetric reporting tools (QReports) can improve the accuracy and objectivity of MRI-based assessments. Several QReports are commercially available; however, validation can be difficult to establish and does not currently follow a common pathway. To aid evidence-based clinical decision-making, we performed a systematic review of commercial QReports for use in MS including technical details and published reports of validation and in-use evaluation. METHODS We categorized studies into three types of testing: technical validation, for example, comparison to manual segmentation, clinical validation by clinicians or interpretation of results alongside clinician-rated variables, and in-use evaluation, such as health economic assessment. RESULTS We identified 10 companies, which provide MS lesion and brain segmentation and volume quantification, and 38 relevant publications. Tools received regulatory approval between 2006 and 2020, contextualize results to normative reference populations, ranging from 620 to 8000 subjects, and require T1- and T2-FLAIR-weighted input sequences for longitudinal assessment of whole-brain volume and lesions. In MS, six QReports provided evidence of technical validation, four companies have conducted clinical validation by correlating results with clinical variables, only one has tested their QReport by clinician end-users, and one has performed a simulated in-use socioeconomic evaluation. CONCLUSION We conclude that there is limited evidence in the literature regarding clinical validation and in-use evaluation of commercial MS QReports with a particular lack of clinician end-user testing. Our systematic review provides clinicians and institutions with the available evidence when considering adopting a quantitative reporting tool for MS.
Collapse
Affiliation(s)
- Zoe Mendelsohn
- grid.83440.3b0000000121901201Neuroradiological Academic Unit, UCL Queen Square Institute of Neurology, University College London, London, UK ,grid.83440.3b0000000121901201Department of Medical Physics and Bioengineering, Centre for Medical Image Computing (CMIC), University College London, London, UK ,grid.83440.3b0000000121901201Department of Neuroinflammation, Queen Square Multiple Sclerosis Centre, UCL Institute of Neurology, University College London, London, UK ,grid.6363.00000 0001 2218 4662Department of Neuroradiology, Charité School of Medicine and University Hospital Berlin, Berlin, Germany ,grid.6363.00000 0001 2218 4662Department of Radiology, Charité School of Medicine and University Hospital Berlin, Berlin, Germany
| | - Hugh G. Pemberton
- grid.83440.3b0000000121901201Neuroradiological Academic Unit, UCL Queen Square Institute of Neurology, University College London, London, UK ,grid.83440.3b0000000121901201Department of Medical Physics and Bioengineering, Centre for Medical Image Computing (CMIC), University College London, London, UK ,grid.420685.d0000 0001 1940 6527GE Healthcare, Amersham, UK
| | - James Gray
- grid.416626.10000 0004 0391 2793Stepping Hill Hospital, NHS Foundation Trust, Stockport, UK
| | - Olivia Goodkin
- grid.83440.3b0000000121901201Neuroradiological Academic Unit, UCL Queen Square Institute of Neurology, University College London, London, UK ,grid.83440.3b0000000121901201Department of Medical Physics and Bioengineering, Centre for Medical Image Computing (CMIC), University College London, London, UK ,grid.83440.3b0000000121901201Department of Neuroinflammation, Queen Square Multiple Sclerosis Centre, UCL Institute of Neurology, University College London, London, UK
| | - Ferran Prados Carrasco
- grid.83440.3b0000000121901201Department of Medical Physics and Bioengineering, Centre for Medical Image Computing (CMIC), University College London, London, UK ,grid.83440.3b0000000121901201Department of Neuroinflammation, Queen Square Multiple Sclerosis Centre, UCL Institute of Neurology, University College London, London, UK ,grid.36083.3e0000 0001 2171 6620E-Health Centre, Universitat Oberta de Catalunya, Barcelona, Spain
| | - Michael Scheel
- grid.6363.00000 0001 2218 4662Department of Neuroradiology, Charité School of Medicine and University Hospital Berlin, Berlin, Germany
| | - Jawed Nawabi
- grid.6363.00000 0001 2218 4662Department of Radiology, Charité School of Medicine and University Hospital Berlin, Berlin, Germany ,grid.484013.a0000 0004 6879 971XBerlin Institute of Health at Charité – Universitätsmedizin Berlin, BIH Biomedical Innovation Academy, BIH Charité Digital Clinician Scientist Program, Berlin, Germany
| | - Frederik Barkhof
- grid.83440.3b0000000121901201Neuroradiological Academic Unit, UCL Queen Square Institute of Neurology, University College London, London, UK ,grid.83440.3b0000000121901201Department of Medical Physics and Bioengineering, Centre for Medical Image Computing (CMIC), University College London, London, UK ,grid.83440.3b0000000121901201Department of Neuroinflammation, Queen Square Multiple Sclerosis Centre, UCL Institute of Neurology, University College London, London, UK ,grid.12380.380000 0004 1754 9227Radiology & Nuclear Medicine, Amsterdam University Medical Centers, Vrije Universiteit, Amsterdam, The Netherlands
| |
Collapse
|
8
|
Schmidt-Mengin M, Soulier T, Hamzaoui M, Yazdan-Panah A, Bodini B, Ayache N, Stankoff B, Colliot O. Online hard example mining vs. fixed oversampling strategy for segmentation of new multiple sclerosis lesions from longitudinal FLAIR MRI. Front Neurosci 2022; 16:1004050. [PMID: 36408404 PMCID: PMC9672803 DOI: 10.3389/fnins.2022.1004050] [Citation(s) in RCA: 4] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/26/2022] [Accepted: 10/11/2022] [Indexed: 11/06/2022] Open
Abstract
Detecting new lesions is a key aspect of the radiological follow-up of patients with Multiple Sclerosis (MS), leading to eventual changes in their therapeutics. This paper presents our contribution to the MSSEG-2 MICCAI 2021 challenge. The challenge is focused on the segmentation of new MS lesions using two consecutive Fluid Attenuated Inversion Recovery (FLAIR) Magnetic Resonance Imaging (MRI). In other words, considering longitudinal data composed of two time points as input, the aim is to segment the lesional areas, which are present only in the follow-up scan and not in the baseline. The backbone of our segmentation method is a 3D UNet applied patch-wise to the images, and in which, to take into account both time points, we simply concatenate the baseline and follow-up images along the channel axis before passing them to the 3D UNet. Our key methodological contribution is the use of online hard example mining to address the challenge of class imbalance. Indeed, there are very few voxels belonging to new lesions which makes training deep-learning models difficult. Instead of using handcrafted priors like brain masks or multi-stage methods, we experiment with a novel modification to online hard example mining (OHEM), where we use an exponential moving average (i.e., its weights are updated with momentum) of the 3D UNet to mine hard examples. Using a moving average instead of the raw model should allow smoothing of its predictions and allow it to give more consistent feedback for OHEM.
Collapse
Affiliation(s)
- Marius Schmidt-Mengin
- Institut du Cerveau-Paris Brain Institute, Centre National de la Recherche Scientifique, Inria, Inserm, Assistance Publique-Hôpitaux de Paris, Hôpital de la Pitié Salpêtrière, Sorbonne Université, Paris, France
- Institut du Cerveau-Paris Brain Institute, Centre National de la Recherche Scientifique, Inserm, Assistance Publique-Hôpitaux de Paris, Hôpital de la Pitié Salpêtrière, Sorbonne Université, Paris, France
| | - Théodore Soulier
- Institut du Cerveau-Paris Brain Institute, Centre National de la Recherche Scientifique, Inserm, Assistance Publique-Hôpitaux de Paris, Hôpital de la Pitié Salpêtrière, Sorbonne Université, Paris, France
| | - Mariem Hamzaoui
- Institut du Cerveau-Paris Brain Institute, Centre National de la Recherche Scientifique, Inserm, Assistance Publique-Hôpitaux de Paris, Hôpital de la Pitié Salpêtrière, Sorbonne Université, Paris, France
| | - Arya Yazdan-Panah
- Institut du Cerveau-Paris Brain Institute, Centre National de la Recherche Scientifique, Inria, Inserm, Assistance Publique-Hôpitaux de Paris, Hôpital de la Pitié Salpêtrière, Sorbonne Université, Paris, France
- Institut du Cerveau-Paris Brain Institute, Centre National de la Recherche Scientifique, Inserm, Assistance Publique-Hôpitaux de Paris, Hôpital de la Pitié Salpêtrière, Sorbonne Université, Paris, France
| | - Benedetta Bodini
- Institut du Cerveau-Paris Brain Institute, Centre National de la Recherche Scientifique, Inserm, Assistance Publique-Hôpitaux de Paris, Hôpital de la Pitié Salpêtrière, Sorbonne Université, Paris, France
- Department of Neurology, Assistance Publique-Hôpitaux de Paris, Hôpital Saint-Antoine, Paris, France
| | | | - Bruno Stankoff
- Institut du Cerveau-Paris Brain Institute, Centre National de la Recherche Scientifique, Inserm, Assistance Publique-Hôpitaux de Paris, Hôpital de la Pitié Salpêtrière, Sorbonne Université, Paris, France
- Department of Neurology, Assistance Publique-Hôpitaux de Paris, Hôpital Saint-Antoine, Paris, France
| | - Olivier Colliot
- Institut du Cerveau-Paris Brain Institute, Centre National de la Recherche Scientifique, Inria, Inserm, Assistance Publique-Hôpitaux de Paris, Hôpital de la Pitié Salpêtrière, Sorbonne Université, Paris, France
- *Correspondence: Olivier Colliot
| |
Collapse
|
9
|
Ashtari P, Barile B, Van Huffel S, Sappey-Marinier D. New multiple sclerosis lesion segmentation and detection using pre-activation U-Net. Front Neurosci 2022; 16:975862. [PMID: 36389254 PMCID: PMC9646406 DOI: 10.3389/fnins.2022.975862] [Citation(s) in RCA: 4] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/22/2022] [Accepted: 09/14/2022] [Indexed: 12/02/2022] Open
Abstract
Automated segmentation of new multiple sclerosis (MS) lesions in 3D MRI data is an essential prerequisite for monitoring and quantifying MS progression. Manual delineation of such lesions is time-consuming and expensive, especially because raters need to deal with 3D images and several modalities. In this paper, we propose Pre-U-Net, a 3D encoder-decoder architecture with pre-activation residual blocks, for the segmentation and detection of new MS lesions. Due to the limited training set and the class imbalance problem, we apply intensive data augmentation and use deep supervision to train our models effectively. Following the same U-shaped architecture but different blocks, Pre-U-Net outperforms U-Net and Res-U-Net on the MSSEG-2 dataset, achieving a Dice score of 40.3% on new lesion segmentation and an F1 score of 48.1% on new lesion detection. The codes and trained models are publicly available at https://github.com/pashtari/xunet.
Collapse
Affiliation(s)
- Pooya Ashtari
- Department of Electrical Engineering (ESAT), STADIUS Centre for Dynamical Systems, Signal Processing and Data Analytics, KU Leuven, Leuven, Belgium
- CREATIS (UMR 5220 CNRS – U1294 INSERM), Université Claude Bernard Lyon 1, Université de Lyon, Villeurbanne, France
- *Correspondence: Pooya Ashtari
| | - Berardino Barile
- Department of Electrical Engineering (ESAT), STADIUS Centre for Dynamical Systems, Signal Processing and Data Analytics, KU Leuven, Leuven, Belgium
- CREATIS (UMR 5220 CNRS – U1294 INSERM), Université Claude Bernard Lyon 1, Université de Lyon, Villeurbanne, France
| | - Sabine Van Huffel
- Department of Electrical Engineering (ESAT), STADIUS Centre for Dynamical Systems, Signal Processing and Data Analytics, KU Leuven, Leuven, Belgium
| | - Dominique Sappey-Marinier
- CREATIS (UMR 5220 CNRS – U1294 INSERM), Université Claude Bernard Lyon 1, Université de Lyon, Villeurbanne, France
| |
Collapse
|
10
|
Khezrpour S, Seyedarabi H, Razavi SN, Farhoudi M. Automatic segmentation of the brain stroke lesions from MR flair scans using improved U-net framework. Biomed Signal Process Control 2022. [DOI: 10.1016/j.bspc.2022.103978] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/25/2022]
|
11
|
Hitziger S, Ling WX, Fritz T, D'Albis T, Lemke A, Grilo J. Triplanar U-Net with lesion-wise voting for the segmentation of new lesions on longitudinal MRI studies. Front Neurosci 2022; 16:964250. [PMID: 36033604 PMCID: PMC9412001 DOI: 10.3389/fnins.2022.964250] [Citation(s) in RCA: 8] [Impact Index Per Article: 4.0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/08/2022] [Accepted: 07/22/2022] [Indexed: 11/13/2022] Open
Abstract
We present a deep learning method for the segmentation of new lesions in longitudinal FLAIR MRI sequences acquired at two different time points. In our approach, the 3D volumes are processed slice-wise across the coronal, axial, and sagittal planes and the predictions from the three orientations are merged using an optimized voting strategy. Our method achieved best F1 score (0.541) among all participating methods in the MICCAI 2021 challenge Multiple sclerosis new lesions segmentation (MSSEG-2). Moreover, we show that our method is on par with the challenge's expert neuroradiologists: on an unbiased ground truth, our method achieves results comparable to those of the four experts in terms of detection (F1 score) and segmentation accuracy (Dice score).
Collapse
|
12
|
Sarica B, Seker DZ. New MS lesion segmentation with deep residual attention gate U-Net utilizing 2D slices of 3D MR images. Front Neurosci 2022; 16:912000. [PMID: 35968389 PMCID: PMC9365701 DOI: 10.3389/fnins.2022.912000] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/03/2022] [Accepted: 06/27/2022] [Indexed: 11/13/2022] Open
Abstract
Multiple sclerosis (MS) is an autoimmune disease that causes lesions in the central nervous system of humans due to demyelinating axons. Magnetic resonance imaging (MRI) is widely used for monitoring and measuring MS lesions. Automated methods for MS lesion segmentation have usually been performed on individual MRI scans. Recently, tracking lesion activity for quantifying and monitoring MS disease progression, especially detecting new lesions, has become an important biomarker. In this study, a unique pipeline with a deep neural network that combines U-Net, attention gate, and residual learning is proposed to perform better new MS lesion segmentation using baseline and follow-up 3D FLAIR MR images. The proposed network has a similar architecture to U-Net and is formed from residual units which facilitate the training of deep networks. Networks with fewer parameters are designed with better performance through the skip connections of U-Net and residual units, which facilitate information propagation without degradation. Attention gates also learn to focus on salient features of the target structures of various sizes and shapes. The MSSEG-2 dataset was used for training and testing the proposed pipeline, and the results were compared with those of other proposed pipelines of the challenge and experts who participated in the same challenge. According to the results over the testing set, the lesion-wise F1 and dice scores were obtained as a mean of 48 and 44.30%. For the no-lesion cases, the number of tested and volume of tested lesions were obtained as a mean of 0.148 and 1.488, respectively. The proposed pipeline outperformed 22 proposed pipelines and ranked 8th in the challenge.
Collapse
Affiliation(s)
- Beytullah Sarica
- Department of Applied Informatics, Graduate School, Istanbul Technical University, Istanbul, Turkey
| | - Dursun Zafer Seker
- Department of Geomatics Engineering, Faculty of Civil Engineering, Istanbul Technical University, Istanbul, Turkey
| |
Collapse
|
13
|
Diaz-Hurtado M, Martínez-Heras E, Solana E, Casas-Roma J, Llufriu S, Kanber B, Prados F. Recent advances in the longitudinal segmentation of multiple sclerosis lesions on magnetic resonance imaging: a review. Neuroradiology 2022; 64:2103-2117. [PMID: 35864180 DOI: 10.1007/s00234-022-03019-3] [Citation(s) in RCA: 7] [Impact Index Per Article: 3.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/29/2022] [Accepted: 07/12/2022] [Indexed: 01/18/2023]
Abstract
Multiple sclerosis (MS) is a chronic autoimmune disease characterized by demyelinating lesions that are often visible on magnetic resonance imaging (MRI). Segmentation of these lesions can provide imaging biomarkers of disease burden that can help monitor disease progression and the imaging response to treatment. Manual delineation of MRI lesions is tedious and prone to subjective bias, while automated lesion segmentation methods offer objectivity and speed, the latter being particularly important when analysing large datasets. Lesion segmentation can be broadly categorised into two groups: cross-sectional methods, which use imaging data acquired at a single time-point to characterise MRI lesions; and longitudinal methods, which use imaging data from the same subject acquired at two or more different time-points to characterise lesions over time. The main objective of longitudinal segmentation approaches is to more accurately detect the presence of new MS lesions and the growth or remission of existing lesions, which may be effective biomarkers of disease progression and treatment response. This paper reviews articles on longitudinal MS lesion segmentation methods published over the past 10 years. These are divided into traditional machine learning methods and deep learning techniques. PubMed articles using longitudinal information and comparing fully automatic two time point segmentations in any step of the process were selected. Nineteen articles were reviewed. There is an increasing number of deep learning techniques for longitudinal MS lesion segmentation that are promising to help better understand disease progression.
Collapse
Affiliation(s)
| | - Eloy Martínez-Heras
- Center of Neuroimmunology, Laboratory of Advanced Imaging in Neuroimmunological Diseases, Hospital Clinic Barcelona, Institut d'Investigacions Biomèdiques August Pi I Sunyer (IDIBAPS), Universitat de Barcelona, Barcelona, Spain
| | - Elisabeth Solana
- Center of Neuroimmunology, Laboratory of Advanced Imaging in Neuroimmunological Diseases, Hospital Clinic Barcelona, Institut d'Investigacions Biomèdiques August Pi I Sunyer (IDIBAPS), Universitat de Barcelona, Barcelona, Spain
| | - Jordi Casas-Roma
- E-Health Center, Universitat Oberta de Catalunya, Barcelona, Spain
| | - Sara Llufriu
- Center of Neuroimmunology, Laboratory of Advanced Imaging in Neuroimmunological Diseases, Hospital Clinic Barcelona, Institut d'Investigacions Biomèdiques August Pi I Sunyer (IDIBAPS), Universitat de Barcelona, Barcelona, Spain
| | - Baris Kanber
- Centre for Medical Image Computing, Department of Medical Physics and Biomedical Engineering, University College London, London, UK.,National Institute for Health Research Biomedical Research Centre, University College London, London, UK.,Queen Square MS Centre, Department of Neuroinflammation, Faculty of Brain Sciences, UCL Institute of Neurology, University College London, London, UK
| | - Ferran Prados
- E-Health Center, Universitat Oberta de Catalunya, Barcelona, Spain.,Centre for Medical Image Computing, Department of Medical Physics and Biomedical Engineering, University College London, London, UK.,National Institute for Health Research Biomedical Research Centre, University College London, London, UK.,Queen Square MS Centre, Department of Neuroinflammation, Faculty of Brain Sciences, UCL Institute of Neurology, University College London, London, UK
| |
Collapse
|
14
|
Papadopoulos TG, Tripoliti EE, Plati D, Zelilidou S, Vlachos K, Konitsiotis S, Fotiadis DI. White Matter Lesion Segmentation for Multiple Sclerosis Patients implementing deep learning. ANNUAL INTERNATIONAL CONFERENCE OF THE IEEE ENGINEERING IN MEDICINE AND BIOLOGY SOCIETY. IEEE ENGINEERING IN MEDICINE AND BIOLOGY SOCIETY. ANNUAL INTERNATIONAL CONFERENCE 2022; 2022:3818-3821. [PMID: 36085898 DOI: 10.1109/embc48229.2022.9871401] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/15/2023]
Abstract
The aim of this work is to address the problem of White Matter Lesion (WML) segmentation employing Magnetic Resonance Imaging (MRI) images from Multiple Sclerosis (MS) patients through the application of deep learning. A U-net based architecture containing a contrastive path and an expanding path prior to the final pixel-wise classification is implemented. The data are provided by the Ippokratio Radiology Center of Ioannina and include Fluid-Attenuated Inversion Recovery (FLAIR) MRI images from 30 patients in three phases, baseline and two follow ups. The prediction results are quite significant in terms of pixel-wise classification. The implemented deep learning model demonstrates Dice coefficient 0.7292, Precision 75.92% and Recall 70.16% in 2D slices of FLAIR MRI non-skull stripped images.
Collapse
|
15
|
Foundations of Lesion Detection Using Machine Learning in Clinical Neuroimaging. ACTA NEUROCHIRURGICA. SUPPLEMENT 2021; 134:171-182. [PMID: 34862541 DOI: 10.1007/978-3-030-85292-4_21] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [Subscribe] [Scholar Register] [Indexed: 12/22/2022]
Abstract
This chapter describes technical considerations and current and future clinical applications of lesion detection using machine learning in the clinical setting. Lesion detection is central to neuroradiology and precedes all further processes which include but are not limited to lesion characterization, quantification, longitudinal disease assessment, prognosis, and prediction of treatment response. A number of machine learning algorithms focusing on lesion detection have been developed or are currently under development which may either support or extend the imaging process. Examples include machine learning applications in stroke, aneurysms, multiple sclerosis, neuro-oncology, neurodegeneration, and epilepsy.
Collapse
|
16
|
Combès B, Kerbrat A, Pasquier G, Commowick O, Le Bon B, Galassi F, L'Hostis P, El Graoui N, Chouteau R, Cordonnier E, Edan G, Ferré JC. A Clinically-Compatible Workflow for Computer-Aided Assessment of Brain Disease Activity in Multiple Sclerosis Patients. Front Med (Lausanne) 2021; 8:740248. [PMID: 34805206 PMCID: PMC8595265 DOI: 10.3389/fmed.2021.740248] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/12/2021] [Accepted: 09/30/2021] [Indexed: 11/21/2022] Open
Abstract
Over the last 10 years, the number of approved disease modifying drugs acting on the focal inflammatory process in Multiple Sclerosis (MS) has increased from 3 to 10. This wide choice offers the opportunity of a personalized medicine with the objective of no clinical and radiological activity for each patient. This new paradigm requires the optimization of the detection of new FLAIR lesions on longitudinal MRI. In this paper, we describe a complete workflow—that we developed, implemented, deployed, and evaluated—to facilitate the monitoring of new FLAIR lesions on longitudinal MRI of MS patients. This workflow has been designed to be usable by both hospital and private neurologists and radiologists in France. It consists of three main components: (i) a software component that allows for automated and secured anonymization and transfer of MRI data from the clinical Picture Archive and Communication System (PACS) to a processing server (and vice-versa); (ii) a fully automated segmentation core that enables detection of focal longitudinal changes in patients from T1-weighted, T2-weighted and FLAIR brain MRI scans, and (iii) a dedicated web viewer that provides an intuitive visualization of new lesions to radiologists and neurologists. We first present these different components. Then, we evaluate the workflow on 54 pairs of longitudinal MRI scans that were analyzed by 3 experts (1 neuroradiologist, 1 radiologist, and 1 neurologist) with and without the proposed workflow. We show that our workflow provided a valuable aid to clinicians in detecting new MS lesions both in terms of accuracy (mean number of detected lesions per patient and per expert 1.8 without the workflow vs. 2.3 with the workflow, p = 5.10−4) and of time dedicated by the experts (mean time difference 2′45″, p = 10−4). This increase in the number of detected lesions has implications in the classification of MS patients as stable or active, even for the most experienced neuroradiologist (mean sensitivity was 0.74 without the workflow and 0.90 with the workflow, p-value for no difference = 0.003). It therefore has potential consequences on the therapeutic management of MS patients.
Collapse
Affiliation(s)
- Benoit Combès
- Univ Rennes, Inria, CNRS, Inserm IRISA UMR 6074, Empenn ERL U 1228, Rennes, France
| | - Anne Kerbrat
- Univ Rennes, Inria, CNRS, Inserm IRISA UMR 6074, Empenn ERL U 1228, Rennes, France.,Neurology Department, Rennes University Hospital, Rennes, France
| | | | - Olivier Commowick
- Univ Rennes, Inria, CNRS, Inserm IRISA UMR 6074, Empenn ERL U 1228, Rennes, France
| | - Brandon Le Bon
- Univ Rennes, Inria, CNRS, Inserm IRISA UMR 6074, Empenn ERL U 1228, Rennes, France
| | - Francesca Galassi
- Univ Rennes, Inria, CNRS, Inserm IRISA UMR 6074, Empenn ERL U 1228, Rennes, France
| | | | - Nora El Graoui
- Univ Rennes, Inria, CNRS, Inserm IRISA UMR 6074, Empenn ERL U 1228, Rennes, France.,CHU Rennes, Department of Neuroradiology, Rennes, France
| | - Raphael Chouteau
- Neurology Department, Rennes University Hospital, Rennes, France
| | | | - Gilles Edan
- Univ Rennes, Inria, CNRS, Inserm IRISA UMR 6074, Empenn ERL U 1228, Rennes, France.,Neurology Department, Rennes University Hospital, Rennes, France
| | - Jean-Christophe Ferré
- Univ Rennes, Inria, CNRS, Inserm IRISA UMR 6074, Empenn ERL U 1228, Rennes, France.,CHU Rennes, Department of Neuroradiology, Rennes, France
| |
Collapse
|
17
|
Kontopodis EE, Papadaki E, Trivizakis E, Maris TG, Simos P, Papadakis GZ, Tsatsakis A, Spandidos DA, Karantanas A, Marias K. Emerging deep learning techniques using magnetic resonance imaging data applied in multiple sclerosis and clinical isolated syndrome patients (Review). Exp Ther Med 2021; 22:1149. [PMID: 34504594 PMCID: PMC8393268 DOI: 10.3892/etm.2021.10583] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/12/2021] [Accepted: 07/29/2021] [Indexed: 12/18/2022] Open
Abstract
Computer-aided diagnosis systems aim to assist clinicians in the early identification of abnormal signs in order to optimize the interpretation of medical images and increase diagnostic precision. Multiple sclerosis (MS) and clinically isolated syndrome (CIS) are chronic inflammatory, demyelinating diseases affecting the central nervous system. Recent advances in deep learning (DL) techniques have led to novel computational paradigms in MS and CIS imaging designed for automatic segmentation and detection of areas of interest and automatic classification of anatomic structures, as well as optimization of neuroimaging protocols. To this end, there are several publications presenting artificial intelligence-based predictive models aiming to increase diagnostic accuracy and to facilitate optimal clinical management in patients diagnosed with MS and/or CIS. The current study presents a thorough review covering DL techniques that have been applied in MS and CIS during recent years, shedding light on their current advances and limitations.
Collapse
Affiliation(s)
- Eleftherios E Kontopodis
- Computational BioMedicine Laboratory, Institute of Computer Science, Foundation for Research and Technology-Hellas, 70013 Heraklion, Greece.,Department of Radiology, Medical School, University of Crete, 70013 Heraklion, Greece
| | - Efrosini Papadaki
- Computational BioMedicine Laboratory, Institute of Computer Science, Foundation for Research and Technology-Hellas, 70013 Heraklion, Greece.,Department of Radiology, Medical School, University of Crete, 70013 Heraklion, Greece
| | - Eleftherios Trivizakis
- Computational BioMedicine Laboratory, Institute of Computer Science, Foundation for Research and Technology-Hellas, 70013 Heraklion, Greece.,Department of Radiology, Medical School, University of Crete, 70013 Heraklion, Greece
| | - Thomas G Maris
- Computational BioMedicine Laboratory, Institute of Computer Science, Foundation for Research and Technology-Hellas, 70013 Heraklion, Greece.,Department of Radiology, Medical School, University of Crete, 70013 Heraklion, Greece
| | - Panagiotis Simos
- Computational BioMedicine Laboratory, Institute of Computer Science, Foundation for Research and Technology-Hellas, 70013 Heraklion, Greece.,Department of Psychiatry and Behavioral Sciences, Medical School, University of Crete, 70013 Heraklion, Greece
| | - Georgios Z Papadakis
- Computational BioMedicine Laboratory, Institute of Computer Science, Foundation for Research and Technology-Hellas, 70013 Heraklion, Greece.,Department of Radiology, Medical School, University of Crete, 70013 Heraklion, Greece
| | - Aristidis Tsatsakis
- Centre of Toxicology Science and Research, Faculty of Medicine, University of Crete, 71003 Heraklion, Greece
| | - Demetrios A Spandidos
- Laboratory of Clinical Virology, Medical School, University of Crete, 71003 Heraklion, Greece
| | - Apostolos Karantanas
- Computational BioMedicine Laboratory, Institute of Computer Science, Foundation for Research and Technology-Hellas, 70013 Heraklion, Greece.,Department of Radiology, Medical School, University of Crete, 70013 Heraklion, Greece
| | - Kostas Marias
- Computational BioMedicine Laboratory, Institute of Computer Science, Foundation for Research and Technology-Hellas, 70013 Heraklion, Greece.,Department of Electrical and Computer Engineering, Hellenic Mediterranean University, 71410 Heraklion, Greece
| |
Collapse
|
18
|
Krishna Priya R, Chacko S. Improved particle swarm optimized deep convolutional neural network with super-pixel clustering for multiple sclerosis lesion segmentation in brain MRI imaging. INTERNATIONAL JOURNAL FOR NUMERICAL METHODS IN BIOMEDICAL ENGINEERING 2021; 37:e3506. [PMID: 34181310 DOI: 10.1002/cnm.3506] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 03/31/2020] [Revised: 02/09/2021] [Accepted: 03/12/2021] [Indexed: 06/13/2023]
Abstract
A central nervous system (CNS) disease affecting the insulating myelin sheaths around the brain axons is called multiple sclerosis (MS). In today's world, MS is extensively diagnosed and monitored using the MRI, because of the structural MRI sensitivity in dissemination of white matter lesions with respect to space and time. The main aim of this study is to propose Multiple Sclerosis Lesion Segmentation in Brain MRI imaging using Optimized Deep Convolutional Neural Network and Super-pixel Clustering. Three stages included in the proposed methodology are: (a) preprocessing, (b) segmentation of super-pixel, and (c) classification of super-pixel. In the first stage, image enhancement and skull stripping is done through performing a preprocessing step. In the second stage, the MS lesion and Non-MS lesion regions are segmented through applying SLICO algorithm over each slice of the volume. In the fourth stage, a CNN training and classification is performed using this segmented lesion and non-lesion regions. To handle this complex task, a newly developed Improved Particle Swarm Optimization (IPSO) based optimized convolutional neural network classifier is applied. On clinical MS data, the approach exhibits a significant increase in the accuracy segmenting of WM lesions when compared with the rest of evaluated methods.
Collapse
Affiliation(s)
- R Krishna Priya
- Department of Electrical and Communication Engineering, National University of Science and Technology, Oman
| | - Susamma Chacko
- Department of Quality Enhancement and Assurance, National University of Science and Technology, Oman
| |
Collapse
|
19
|
Moazami F, Lefevre-Utile A, Papaloukas C, Soumelis V. Machine Learning Approaches in Study of Multiple Sclerosis Disease Through Magnetic Resonance Images. Front Immunol 2021; 12:700582. [PMID: 34456913 PMCID: PMC8385534 DOI: 10.3389/fimmu.2021.700582] [Citation(s) in RCA: 12] [Impact Index Per Article: 4.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/28/2021] [Accepted: 07/26/2021] [Indexed: 11/13/2022] Open
Abstract
Multiple sclerosis (MS) is one of the most common autoimmune diseases which is commonly diagnosed and monitored using magnetic resonance imaging (MRI) with a combination of clinical manifestations. The purpose of this review is to highlight the main applications of Machine Learning (ML) models and their performance in the MS field using MRI. We reviewed the articles of the last decade and grouped them based on the applications of ML in MS using MRI data into four categories: 1) Automated diagnosis of MS, 2) Prediction of MS disease progression, 3) Differentiation of MS stages, 4) Differentiation of MS from similar disorders.
Collapse
Affiliation(s)
- Faezeh Moazami
- Université de Paris, Institut de Recherche Saint-Louis, Institut National de la Santé et de la Recherche Médicale (INSERM) U976, Hôpital Saint-Louis, Paris, France
| | - Alain Lefevre-Utile
- Université de Paris, Institut de Recherche Saint-Louis, Institut National de la Santé et de la Recherche Médicale (INSERM) U976, Hôpital Saint-Louis, Paris, France.,Université Paris-Saclay, Saint Aubin, France.,Assistance Publique Hopitaux de Paris (APHP), General Pediatric and Pediatric Emergency Department, Jean Verdier Hospital, Bondy, France
| | - Costas Papaloukas
- Department of Biological Applications and Technology, University of Ioannina, Ioannina, Greece
| | - Vassili Soumelis
- Université de Paris, Institut de Recherche Saint-Louis, Institut National de la Santé et de la Recherche Médicale (INSERM) U976, Hôpital Saint-Louis, Paris, France.,Assistance Publique Hopitaux de Paris (APHP), Hôpital Saint-Louis, Immunology-Histocompatibility Department, Paris, France
| |
Collapse
|
20
|
Shoeibi A, Khodatars M, Jafari M, Moridian P, Rezaei M, Alizadehsani R, Khozeimeh F, Gorriz JM, Heras J, Panahiazar M, Nahavandi S, Acharya UR. Applications of deep learning techniques for automated multiple sclerosis detection using magnetic resonance imaging: A review. Comput Biol Med 2021; 136:104697. [PMID: 34358994 DOI: 10.1016/j.compbiomed.2021.104697] [Citation(s) in RCA: 50] [Impact Index Per Article: 16.7] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/11/2021] [Revised: 07/22/2021] [Accepted: 07/25/2021] [Indexed: 11/18/2022]
Abstract
Multiple Sclerosis (MS) is a type of brain disease which causes visual, sensory, and motor problems for people with a detrimental effect on the functioning of the nervous system. In order to diagnose MS, multiple screening methods have been proposed so far; among them, magnetic resonance imaging (MRI) has received considerable attention among physicians. MRI modalities provide physicians with fundamental information about the structure and function of the brain, which is crucial for the rapid diagnosis of MS lesions. Diagnosing MS using MRI is time-consuming, tedious, and prone to manual errors. Research on the implementation of computer aided diagnosis system (CADS) based on artificial intelligence (AI) to diagnose MS involves conventional machine learning and deep learning (DL) methods. In conventional machine learning, feature extraction, feature selection, and classification steps are carried out by using trial and error; on the contrary, these steps in DL are based on deep layers whose values are automatically learn. In this paper, a complete review of automated MS diagnosis methods performed using DL techniques with MRI neuroimaging modalities is provided. Initially, the steps involved in various CADS proposed using MRI modalities and DL techniques for MS diagnosis are investigated. The important preprocessing techniques employed in various works are analyzed. Most of the published papers on MS diagnosis using MRI modalities and DL are presented. The most significant challenges facing and future direction of automated diagnosis of MS using MRI modalities and DL techniques are also provided.
Collapse
Affiliation(s)
- Afshin Shoeibi
- Faculty of Electrical Engineering, Biomedical Data Acquisition Lab (BDAL), K. N. Toosi University of Technology, Tehran, Iran.
| | - Marjane Khodatars
- Faculty of Engineering, Mashhad Branch, Islamic Azad University, Mashhad, Iran
| | - Mahboobeh Jafari
- Electrical and Computer Engineering Faculty, Semnan University, Semnan, Iran
| | - Parisa Moridian
- Faculty of Engineering, Science and Research Branch, Islamic Azad University, Tehran, Iran
| | - Mitra Rezaei
- Electrical and Computer Engineering Dept., Tarbiat Modares University, Tehran, Iran
| | - Roohallah Alizadehsani
- Institute for Intelligent Systems Research and Innovation (IISRI), Deakin University, Geelong, Australia
| | - Fahime Khozeimeh
- Institute for Intelligent Systems Research and Innovation (IISRI), Deakin University, Geelong, Australia
| | - Juan Manuel Gorriz
- Department of Signal Theory, Networking and Communications, Universidad de Granada, Spain; Department of Psychiatry. University of Cambridge, UK
| | - Jónathan Heras
- Department of Mathematics and Computer Science, University of La Rioja, La Rioja, Spain
| | | | - Saeid Nahavandi
- Institute for Intelligent Systems Research and Innovation (IISRI), Deakin University, Geelong, Australia
| | - U Rajendra Acharya
- Department of Biomedical Engineering, School of Science and Technology, Singapore University of Social Sciences, Singapore; Dept. of Electronics and Computer Engineering, Ngee Ann Polytechnic, 599489, Singapore; Department of Bioinformatics and Medical Engineering, Asia University, Taiwan
| |
Collapse
|
21
|
Zeng C, Gu L, Liu Z, Zhao S. Review of Deep Learning Approaches for the Segmentation of Multiple Sclerosis Lesions on Brain MRI. Front Neuroinform 2020; 14:610967. [PMID: 33328949 PMCID: PMC7714963 DOI: 10.3389/fninf.2020.610967] [Citation(s) in RCA: 34] [Impact Index Per Article: 8.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/28/2020] [Accepted: 10/26/2020] [Indexed: 11/18/2022] Open
Abstract
In recent years, there have been multiple works of literature reviewing methods for automatically segmenting multiple sclerosis (MS) lesions. However, there is no literature systematically and individually review deep learning-based MS lesion segmentation methods. Although the previous review also included methods based on deep learning, there are some methods based on deep learning that they did not review. In addition, their review of deep learning methods did not go deep into the specific categories of Convolutional Neural Network (CNN). They only reviewed these methods in a generalized form, such as supervision strategy, input data handling strategy, etc. This paper presents a systematic review of the literature in automated multiple sclerosis lesion segmentation based on deep learning. Algorithms based on deep learning reviewed are classified into two categories through their CNN style, and their strengths and weaknesses will also be given through our investigation and analysis. We give a quantitative comparison of the methods reviewed through two metrics: Dice Similarity Coefficient (DSC) and Positive Predictive Value (PPV). Finally, the future direction of the application of deep learning in MS lesion segmentation will be discussed.
Collapse
Affiliation(s)
- Chenyi Zeng
- School of Intelligent Systems Engineering, Sun Yat-Sen University, Guangzhou, China
| | - Lin Gu
- RIKEN AIP, Tokyo, Japan
- The University of Tokyo, Tokyo, Japan
| | - Zhenzhong Liu
- Tianjin Key Laboratory for Advanced Mechatronic System Design and Intelligent Control, School of Mechanical Engineering, Tianjin University of Technology, Tianjin, China
- National Demonstration Center for Experimental Mechanical and Electrical Engineering Education, Tianjin University of Technology, Tianjin, China
| | - Shen Zhao
- School of Intelligent Systems Engineering, Sun Yat-Sen University, Guangzhou, China
| |
Collapse
|