1
|
Sudre CH, Van Wijnen K, Dubost F, Adams H, Atkinson D, Barkhof F, Birhanu MA, Bron EE, Camarasa R, Chaturvedi N, Chen Y, Chen Z, Chen S, Dou Q, Evans T, Ezhov I, Gao H, Girones Sanguesa M, Gispert JD, Gomez Anson B, Hughes AD, Ikram MA, Ingala S, Jaeger HR, Kofler F, Kuijf HJ, Kutnar D, Lee M, Li B, Lorenzini L, Menze B, Molinuevo JL, Pan Y, Puybareau E, Rehwald R, Su R, Shi P, Smith L, Tillin T, Tochon G, Urien H, van der Velden BHM, van der Velpen IF, Wiestler B, Wolters FJ, Yilmaz P, de Groot M, Vernooij MW, de Bruijne M. Where is VALDO? VAscular Lesions Detection and segmentatiOn challenge at MICCAI 2021. Med Image Anal 2024; 91:103029. [PMID: 37988921 DOI: 10.1016/j.media.2023.103029] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/01/2022] [Revised: 08/09/2023] [Accepted: 11/13/2023] [Indexed: 11/23/2023]
Abstract
Imaging markers of cerebral small vessel disease provide valuable information on brain health, but their manual assessment is time-consuming and hampered by substantial intra- and interrater variability. Automated rating may benefit biomedical research, as well as clinical assessment, but diagnostic reliability of existing algorithms is unknown. Here, we present the results of the VAscular Lesions DetectiOn and Segmentation (Where is VALDO?) challenge that was run as a satellite event at the international conference on Medical Image Computing and Computer Aided Intervention (MICCAI) 2021. This challenge aimed to promote the development of methods for automated detection and segmentation of small and sparse imaging markers of cerebral small vessel disease, namely enlarged perivascular spaces (EPVS) (Task 1), cerebral microbleeds (Task 2) and lacunes of presumed vascular origin (Task 3) while leveraging weak and noisy labels. Overall, 12 teams participated in the challenge proposing solutions for one or more tasks (4 for Task 1-EPVS, 9 for Task 2-Microbleeds and 6 for Task 3-Lacunes). Multi-cohort data was used in both training and evaluation. Results showed a large variability in performance both across teams and across tasks, with promising results notably for Task 1-EPVS and Task 2-Microbleeds and not practically useful results yet for Task 3-Lacunes. It also highlighted the performance inconsistency across cases that may deter use at an individual level, while still proving useful at a population level.
Collapse
Affiliation(s)
- Carole H Sudre
- MRC Unit for Lifelong Health and Ageing at UCL, Department of Population Science and Experimental Medicine, University College London, London, United Kingdom; Centre for Medical Image Computing, University College London, London, United Kingdom; School of Biomedical Engineering and Imaging Sciences, King's College London, London, United Kingdom.
| | - Kimberlin Van Wijnen
- Biomedical Imaging Group Rotterdam, Department of Radiology and Nuclear Medicine, Erasmus MC, Rotterdam, The Netherlands
| | - Florian Dubost
- Biomedical Imaging Group Rotterdam, Department of Radiology and Nuclear Medicine, Erasmus MC, Rotterdam, The Netherlands
| | - Hieab Adams
- Department of Clinical Genetics and Radiology, Erasmus MC, Rotterdam, The Netherlands
| | - David Atkinson
- Centre for Medical Imaging, University College London, London, United Kingdom
| | - Frederik Barkhof
- Centre for Medical Image Computing, University College London, London, United Kingdom; Department of Radiology and Nuclear Medicine, Amsterdam University Medical Centre, Amsterdam, The Netherlands
| | - Mahlet A Birhanu
- Biomedical Imaging Group Rotterdam, Department of Radiology and Nuclear Medicine, Erasmus MC, Rotterdam, The Netherlands
| | - Esther E Bron
- Biomedical Imaging Group Rotterdam, Department of Radiology and Nuclear Medicine, Erasmus MC, Rotterdam, The Netherlands
| | - Robin Camarasa
- Biomedical Imaging Group Rotterdam, Department of Radiology and Nuclear Medicine, Erasmus MC, Rotterdam, The Netherlands
| | - Nish Chaturvedi
- MRC Unit for Lifelong Health and Ageing at UCL, Department of Population Science and Experimental Medicine, University College London, London, United Kingdom
| | - Yuan Chen
- Department of Radiology, University of Massachusetts Medical School, Worcester, USA
| | - Zihao Chen
- School of Biomedical Engineering, Shanghai Jiao Tong University, Shanghai, China
| | - Shuai Chen
- Biomedical Imaging Group Rotterdam, Department of Radiology and Nuclear Medicine, Erasmus MC, Rotterdam, The Netherlands
| | - Qi Dou
- Department of Computer Science and Engineering, The Chinese University of Hong Kong, China
| | - Tavia Evans
- Department of Clinical Genetics and Radiology, Erasmus MC, Rotterdam, The Netherlands
| | - Ivan Ezhov
- Department of Informatics, Technische Universitat Munchen, Munich, Germany; TranslaTUM - Central Institute for Translational Cancer Research, Technical University of Munich, Germany
| | - Haojun Gao
- Department of Radiology, Zhejiang University, Hangzhou, China
| | | | - Juan Domingo Gispert
- Barcelonaß Brain Research Center (BBRC), Pasqual Maragall Foundation, Barcelona, Spain; Hospital del Mar Medical Research Institute (IMIM), Barcelona, Spain; Centro de Investigación Biomédica en Red Bioingeniería, Biomateriales y Nanomedicina, (CIBER-BBN), Barcelona, Spain
| | | | - Alun D Hughes
- MRC Unit for Lifelong Health and Ageing at UCL, Department of Population Science and Experimental Medicine, University College London, London, United Kingdom
| | - M Arfan Ikram
- Department of Epidemiology, Erasmus MC, Rotterdam, The Netherlands
| | - Silvia Ingala
- Department of Radiology and Nuclear Medicine, Amsterdam University Medical Centre, Amsterdam, The Netherlands
| | - H Rolf Jaeger
- Institute of Neurology, University College London, London, United Kingdom
| | - Florian Kofler
- Department of Informatics, Technische Universitat Munchen, Munich, Germany; Department of Diagnostic and Interventional Neuroradiology, School of Medicine, Klinikum rechts der Isar, Technical University of Munich, Germany; TranslaTUM - Central Institute for Translational Cancer Research, Technical University of Munich, Germany
| | - Hugo J Kuijf
- Image Sciences Institute, University Medical Center Utrecht, Utrecht, The Netherlands
| | - Denis Kutnar
- Image Sciences Institute, University Medical Center Utrecht, Utrecht, The Netherlands
| | | | - Bo Li
- Biomedical Imaging Group Rotterdam, Department of Radiology and Nuclear Medicine, Erasmus MC, Rotterdam, The Netherlands
| | - Luigi Lorenzini
- Department of Radiology and Nuclear Medicine, Amsterdam University Medical Centre, Amsterdam, The Netherlands
| | - Bjoern Menze
- Department of Informatics, Technische Universitat Munchen, Munich, Germany; Department of Quantitative Biomedicine, University of Zurich, Switzerland
| | - Jose Luis Molinuevo
- Barcelonaß Brain Research Center (BBRC), Pasqual Maragall Foundation, Barcelona, Spain; H. Lundbeck A/S, Copenhagen, Denmark
| | - Yiwei Pan
- Department of Electronic and Information Engineering, Harbin Institute of Technology at Shenzhen, Shenzhen, China
| | | | - Rafael Rehwald
- Institute of Neurology, University College London, London, United Kingdom
| | - Ruisheng Su
- Biomedical Imaging Group Rotterdam, Department of Radiology and Nuclear Medicine, Erasmus MC, Rotterdam, The Netherlands
| | - Pengcheng Shi
- Department of Electronic and Information Engineering, Harbin Institute of Technology at Shenzhen, Shenzhen, China
| | | | - Therese Tillin
- MRC Unit for Lifelong Health and Ageing at UCL, Department of Population Science and Experimental Medicine, University College London, London, United Kingdom
| | | | - Hélène Urien
- ISEP-Institut Supérieur d'Électronique de Paris, Issy-les-Moulineaux, France
| | | | - Isabelle F van der Velpen
- Department of Radiology and Nuclear Medicine, Erasmus MC, Rotterdam, The Netherlands; Department of Epidemiology, Erasmus MC, Rotterdam, The Netherlands
| | - Benedikt Wiestler
- Department of Diagnostic and Interventional Neuroradiology, School of Medicine, Klinikum rechts der Isar, Technical University of Munich, Germany
| | - Frank J Wolters
- Department of Radiology and Nuclear Medicine, Erasmus MC, Rotterdam, The Netherlands; Department of Epidemiology, Erasmus MC, Rotterdam, The Netherlands
| | - Pinar Yilmaz
- Department of Epidemiology, Erasmus MC, Rotterdam, The Netherlands
| | - Marius de Groot
- Biomedical Imaging Group Rotterdam, Department of Radiology and Nuclear Medicine, Erasmus MC, Rotterdam, The Netherlands; GlaxoSmithKline Research, Stevenage, United Kingdom
| | - Meike W Vernooij
- Department of Radiology and Nuclear Medicine, Erasmus MC, Rotterdam, The Netherlands; Department of Epidemiology, Erasmus MC, Rotterdam, The Netherlands
| | - Marleen de Bruijne
- Biomedical Imaging Group Rotterdam, Department of Radiology and Nuclear Medicine, Erasmus MC, Rotterdam, The Netherlands; Department of Computer Science, University of Copenhagen, Copenhagen, Denmark
| |
Collapse
|
2
|
Li L, Wu F, Wang S, Luo X, Martín-Isla C, Zhai S, Zhang J, Liu Y, Zhang Z, Ankenbrand MJ, Jiang H, Zhang X, Wang L, Arega TW, Altunok E, Zhao Z, Li F, Ma J, Yang X, Puybareau E, Oksuz I, Bricq S, Li W, Punithakumar K, Tsaftaris SA, Schreiber LM, Yang M, Liu G, Xia Y, Wang G, Escalera S, Zhuang X. MyoPS: A benchmark of myocardial pathology segmentation combining three-sequence cardiac magnetic resonance images. Med Image Anal 2023; 87:102808. [PMID: 37087838 DOI: 10.1016/j.media.2023.102808] [Citation(s) in RCA: 2] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/08/2022] [Revised: 01/11/2023] [Accepted: 03/30/2023] [Indexed: 04/07/2023]
Abstract
Assessment of myocardial viability is essential in diagnosis and treatment management of patients suffering from myocardial infarction, and classification of pathology on the myocardium is the key to this assessment. This work defines a new task of medical image analysis, i.e., to perform myocardial pathology segmentation (MyoPS) combining three-sequence cardiac magnetic resonance (CMR) images, which was first proposed in the MyoPS challenge, in conjunction with MICCAI 2020. Note that MyoPS refers to both myocardial pathology segmentation and the challenge in this paper. The challenge provided 45 paired and pre-aligned CMR images, allowing algorithms to combine the complementary information from the three CMR sequences for pathology segmentation. In this article, we provide details of the challenge, survey the works from fifteen participants and interpret their methods according to five aspects, i.e., preprocessing, data augmentation, learning strategy, model architecture and post-processing. In addition, we analyze the results with respect to different factors, in order to examine the key obstacles and explore the potential of solutions, as well as to provide a benchmark for future research. The average Dice scores of submitted algorithms were 0.614±0.231 and 0.644±0.153 for myocardial scars and edema, respectively. We conclude that while promising results have been reported, the research is still in the early stage, and more in-depth exploration is needed before a successful application to the clinics. MyoPS data and evaluation tool continue to be publicly available upon registration via its homepage (www.sdspeople.fudan.edu.cn/zhuangxiahai/0/myops20/).
Collapse
Affiliation(s)
- Lei Li
- School of Data Science, Fudan University, Shanghai, China; School of Biomedical Engineering, Shanghai Jiao Tong University, Shanghai, China.
| | - Fuping Wu
- School of Data Science, Fudan University, Shanghai, China.
| | - Sihan Wang
- School of Data Science, Fudan University, Shanghai, China.
| | - Xinzhe Luo
- School of Data Science, Fudan University, Shanghai, China
| | - Carlos Martín-Isla
- Departament de Matemàtiques & Informàtica, Universitat de Barcelona, Barcelona, Spain
| | - Shuwei Zhai
- School of Mechanical and Electrical Engineering, University of Electronic Science and Technology of China, Chengdu, China
| | - Jianpeng Zhang
- School of Computer Science and Engineering, Northwestern Polytechnical University, Xi'an, China
| | - Yanfei Liu
- College of Electrical and Information Engineering, Hunan University, Changsha, China
| | - Zhen Zhang
- College of Physics and Information Engineering, Fuzhou University, Fuzhou, China
| | - Markus J Ankenbrand
- Chair of Molecular and Cellular Imaging, Comprehensive Heart Failure Center, Wuerzburg University Hospitals, Wuerzburg, Germany
| | - Haochuan Jiang
- School of Engineering, University of Edinburgh, Edinburgh, UK; School of Robotics, Xi'an Jiaotong-Liverpool University, Suzhou, China
| | - Xiaoran Zhang
- Department of Electrical and Computer Engineering, University of California, LA, USA
| | - Linhong Wang
- Chongqing Key Laboratory of Image Cognition, Chongqing University of Posts and Telecommunications, Chongqing, China
| | | | - Elif Altunok
- Computer Engineering Department, Istanbul Technical University, Istanbul, Turkey
| | - Zhou Zhao
- EPITA Research and Development Laboratory (LRDE), Le Kremlin-Bicêtre, France
| | - Feiyan Li
- Chongqing Key Laboratory of Image Cognition, Chongqing University of Posts and Telecommunications, Chongqing, China
| | - Jun Ma
- Department of Mathematics, Nanjing University of Science and Technology, Nanjing, China
| | - Xiaoping Yang
- Department of Mathematics, Nanjing University, Nanjing, China
| | - Elodie Puybareau
- EPITA Research and Development Laboratory (LRDE), Le Kremlin-Bicêtre, France
| | - Ilkay Oksuz
- Computer Engineering Department, Istanbul Technical University, Istanbul, Turkey
| | - Stephanie Bricq
- ImViA Laboratory, Université Bourgogne Franche-Comté, Dijon, France
| | - Weisheng Li
- Chongqing Key Laboratory of Image Cognition, Chongqing University of Posts and Telecommunications, Chongqing, China
| | | | | | - Laura M Schreiber
- Chair of Molecular and Cellular Imaging, Comprehensive Heart Failure Center, Wuerzburg University Hospitals, Wuerzburg, Germany
| | - Mingjing Yang
- College of Physics and Information Engineering, Fuzhou University, Fuzhou, China
| | - Guocai Liu
- College of Electrical and Information Engineering, Hunan University, Changsha, China; National Engineering Laboratory for Robot Visual Perception and Control Technology, Changsha, China
| | - Yong Xia
- School of Computer Science and Engineering, Northwestern Polytechnical University, Xi'an, China
| | - Guotai Wang
- School of Mechanical and Electrical Engineering, University of Electronic Science and Technology of China, Chengdu, China
| | - Sergio Escalera
- Departament de Matemàtiques & Informàtica, Universitat de Barcelona, Barcelona, Spain; Computer Vision Center, Universitat Autònoma de Barcelona, Spain
| | - Xiahai Zhuang
- School of Data Science, Fudan University, Shanghai, China.
| |
Collapse
|
3
|
Mehta R, Filos A, Baid U, Sako C, McKinley R, Rebsamen M, Dätwyler K, Meier R, Radojewski P, Murugesan GK, Nalawade S, Ganesh C, Wagner B, Yu FF, Fei B, Madhuranthakam AJ, Maldjian JA, Daza L, Gómez C, Arbeláez P, Dai C, Wang S, Reynaud H, Mo Y, Angelini E, Guo Y, Bai W, Banerjee S, Pei L, AK M, Rosas-González S, Zemmoura I, Tauber C, Vu MH, Nyholm T, Löfstedt T, Ballestar LM, Vilaplana V, McHugh H, Maso Talou G, Wang A, Patel J, Chang K, Hoebel K, Gidwani M, Arun N, Gupta S, Aggarwal M, Singh P, Gerstner ER, Kalpathy-Cramer J, Boutry N, Huard A, Vidyaratne L, Rahman MM, Iftekharuddin KM, Chazalon J, Puybareau E, Tochon G, Ma J, Cabezas M, Llado X, Oliver A, Valencia L, Valverde S, Amian M, Soltaninejad M, Myronenko A, Hatamizadeh A, Feng X, Dou Q, Tustison N, Meyer C, Shah NA, Talbar S, Weber MA, Mahajan A, Jakab A, Wiest R, Fathallah-Shaykh HM, Nazeri A, Milchenko1 M, Marcus D, Kotrotsou A, Colen R, Freymann J, Kirby J, Davatzikos C, Menze B, Bakas S, Gal Y, Arbel T. QU-BraTS: MICCAI BraTS 2020 Challenge on Quantifying Uncertainty in Brain Tumor Segmentation - Analysis of Ranking Scores and Benchmarking Results. J Mach Learn Biomed Imaging 2022; 2022:https://www.melba-journal.org/papers/2022:026.html. [PMID: 36998700 PMCID: PMC10060060] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Subscribe] [Scholar Register] [Indexed: 04/01/2023]
Abstract
Deep learning (DL) models have provided state-of-the-art performance in various medical imaging benchmarking challenges, including the Brain Tumor Segmentation (BraTS) challenges. However, the task of focal pathology multi-compartment segmentation (e.g., tumor and lesion sub-regions) is particularly challenging, and potential errors hinder translating DL models into clinical workflows. Quantifying the reliability of DL model predictions in the form of uncertainties could enable clinical review of the most uncertain regions, thereby building trust and paving the way toward clinical translation. Several uncertainty estimation methods have recently been introduced for DL medical image segmentation tasks. Developing scores to evaluate and compare the performance of uncertainty measures will assist the end-user in making more informed decisions. In this study, we explore and evaluate a score developed during the BraTS 2019 and BraTS 2020 task on uncertainty quantification (QU-BraTS) and designed to assess and rank uncertainty estimates for brain tumor multi-compartment segmentation. This score (1) rewards uncertainty estimates that produce high confidence in correct assertions and those that assign low confidence levels at incorrect assertions, and (2) penalizes uncertainty measures that lead to a higher percentage of under-confident correct assertions. We further benchmark the segmentation uncertainties generated by 14 independent participating teams of QU-BraTS 2020, all of which also participated in the main BraTS segmentation task. Overall, our findings confirm the importance and complementary value that uncertainty estimates provide to segmentation algorithms, highlighting the need for uncertainty quantification in medical image analyses. Finally, in favor of transparency and reproducibility, our evaluation code is made publicly available at https://github.com/RagMeh11/QU-BraTS.
Collapse
Affiliation(s)
- Raghav Mehta
- Centre for Intelligent Machines (CIM), McGill University, Montreal, QC, Canada
| | - Angelos Filos
- Oxford Applied and Theoretical Machine Learning (OATML) Group, University of Oxford, Oxford, England
| | - Ujjwal Baid
- Center for Biomedical Image Computing and Analytics (CBICA), University of Pennsylvania, Philadelphia, PA, USA
- Department of Radiology, Perelman School of Medicine at the University of Pennsylvania, Philadelphia, PA, USA
- Department of Pathology and Laboratory Medicine, Perelman School of Medicine, University of Pennsylvania, Philadelphia, PA, USA
| | - Chiharu Sako
- Center for Biomedical Image Computing and Analytics (CBICA), University of Pennsylvania, Philadelphia, PA, USA
- Department of Radiology, Perelman School of Medicine at the University of Pennsylvania, Philadelphia, PA, USA
| | - Richard McKinley
- Support Center for Advanced Neuroimaging (SCAN), University Institute of Diagnostic and Interventional Neuroradiology, University of Bern, Inselspital, Bern University Hospital, Bern, Switzerland
| | - Michael Rebsamen
- Support Center for Advanced Neuroimaging (SCAN), University Institute of Diagnostic and Interventional Neuroradiology, University of Bern, Inselspital, Bern University Hospital, Bern, Switzerland
| | - Katrin Dätwyler
- Support Center for Advanced Neuroimaging (SCAN), University Institute of Diagnostic and Interventional Neuroradiology, University of Bern, Inselspital, Bern University Hospital, Bern, Switzerland
- Human Performance Lab, Schulthess Clinic, Zurich, Switzerland
| | | | - Piotr Radojewski
- Support Center for Advanced Neuroimaging (SCAN), University Institute of Diagnostic and Interventional Neuroradiology, University of Bern, Inselspital, Bern University Hospital, Bern, Switzerland
| | | | - Sahil Nalawade
- Department of Radiology, University of Texas Southwestern Medical Center, Dallas, TX, USA
| | - Chandan Ganesh
- Department of Radiology, University of Texas Southwestern Medical Center, Dallas, TX, USA
| | - Ben Wagner
- Department of Radiology, University of Texas Southwestern Medical Center, Dallas, TX, USA
| | - Fang F. Yu
- Department of Radiology, University of Texas Southwestern Medical Center, Dallas, TX, USA
| | - Baowei Fei
- Department of Bioengineering, University of Texas at Dallas, Texas, USA
| | - Ananth J. Madhuranthakam
- Department of Radiology, University of Texas Southwestern Medical Center, Dallas, TX, USA
- Advanced Imaging Research Center, University of Texas Southwestern Medical Center, Dallas, TX, USA
| | - Joseph A. Maldjian
- Department of Radiology, University of Texas Southwestern Medical Center, Dallas, TX, USA
- Advanced Imaging Research Center, University of Texas Southwestern Medical Center, Dallas, TX, USA
| | - Laura Daza
- Universidad de los Andes, Bogotá, Colombia
| | | | | | - Chengliang Dai
- Data Science Institute, Imperial College London, London, UK
| | - Shuo Wang
- Data Science Institute, Imperial College London, London, UK
| | | | - Yuanhan Mo
- Data Science Institute, Imperial College London, London, UK
| | - Elsa Angelini
- NIHR Imperial BRC, ITMAT Data Science Group, Imperial College London, London, UK
| | - Yike Guo
- Data Science Institute, Imperial College London, London, UK
| | - Wenjia Bai
- Data Science Institute, Imperial College London, London, UK
- Department of Brain Sciences, Imperial College London, London, UK
| | - Subhashis Banerjee
- Machine Intelligence Unit, Indian Statistical Institute, Kolkata, India
- Department of CSE, University of Calcutta, Kolkata, India
- Division of Visual Information and Interaction (Vi2), Department of Information Technology, Uppsala University, Uppsala, Sweden
| | - Linmin Pei
- Department of Diagnostic Radiology, The University of Pittsburgh Medical Center, Pittsburgh, PA, USA
| | - Murat AK
- Department of Diagnostic Radiology, The University of Pittsburgh Medical Center, Pittsburgh, PA, USA
| | | | - Ilyess Zemmoura
- UMR U1253 iBrain, Université de Tours, Inserm, Tours, France
- Neurosurgery department, CHRU de Tours, Tours, France
| | - Clovis Tauber
- UMR U1253 iBrain, Université de Tours, Inserm, Tours, France
| | - Minh H. Vu
- Department of Radiation Sciences, Umeå University, Umeå, Sweden
| | - Tufve Nyholm
- Department of Radiation Sciences, Umeå University, Umeå, Sweden
| | - Tommy Löfstedt
- Department of Computing Science, Umeå University, Umeå, Sweden
| | - Laura Mora Ballestar
- Signal Theory and Communications Department, Universitat Politècnica de Catalunya, BarcelonaTech, Barcelona, Spain
| | - Veronica Vilaplana
- Signal Theory and Communications Department, Universitat Politècnica de Catalunya, BarcelonaTech, Barcelona, Spain
| | - Hugh McHugh
- Faculty of Medical and Health Sciences, University of Auckland, Auckland, New Zealand
- Radiology Department, Auckland City Hospital, Auckland, New Zealand
| | | | - Alan Wang
- Faculty of Medical and Health Sciences, University of Auckland, Auckland, New Zealand
- Auckland Bioengineering Institute, University of Auckland, New Zealand
| | - Jay Patel
- Athinoula A. Martinos Center for Biomedical Imaging, Department of Radiology, Massachusetts General Hospital, Boston, MA, USA
- Massachusetts Institute of Technology, Cambridge, MA, USA
| | - Ken Chang
- Athinoula A. Martinos Center for Biomedical Imaging, Department of Radiology, Massachusetts General Hospital, Boston, MA, USA
- Massachusetts Institute of Technology, Cambridge, MA, USA
| | - Katharina Hoebel
- Athinoula A. Martinos Center for Biomedical Imaging, Department of Radiology, Massachusetts General Hospital, Boston, MA, USA
- Massachusetts Institute of Technology, Cambridge, MA, USA
| | - Mishka Gidwani
- Athinoula A. Martinos Center for Biomedical Imaging, Department of Radiology, Massachusetts General Hospital, Boston, MA, USA
| | - Nishanth Arun
- Athinoula A. Martinos Center for Biomedical Imaging, Department of Radiology, Massachusetts General Hospital, Boston, MA, USA
| | - Sharut Gupta
- Athinoula A. Martinos Center for Biomedical Imaging, Department of Radiology, Massachusetts General Hospital, Boston, MA, USA
| | - Mehak Aggarwal
- Athinoula A. Martinos Center for Biomedical Imaging, Department of Radiology, Massachusetts General Hospital, Boston, MA, USA
| | - Praveer Singh
- Athinoula A. Martinos Center for Biomedical Imaging, Department of Radiology, Massachusetts General Hospital, Boston, MA, USA
| | - Elizabeth R. Gerstner
- Athinoula A. Martinos Center for Biomedical Imaging, Department of Radiology, Massachusetts General Hospital, Boston, MA, USA
| | - Jayashree Kalpathy-Cramer
- Athinoula A. Martinos Center for Biomedical Imaging, Department of Radiology, Massachusetts General Hospital, Boston, MA, USA
| | - Nicolas Boutry
- EPITA Research and Development Laboratory (LRDE), France
| | - Alexis Huard
- EPITA Research and Development Laboratory (LRDE), France
| | - Lasitha Vidyaratne
- Vision Lab, Electrical and Computer Engineering, Old Dominion University, Norfolk, VA 23529, USA
| | - Md Monibor Rahman
- Vision Lab, Electrical and Computer Engineering, Old Dominion University, Norfolk, VA 23529, USA
| | - Khan M. Iftekharuddin
- Vision Lab, Electrical and Computer Engineering, Old Dominion University, Norfolk, VA 23529, USA
| | - Joseph Chazalon
- EPITA Research and Development Laboratory (LRDE), Le Kremlin-Biĉetre, France
| | - Elodie Puybareau
- EPITA Research and Development Laboratory (LRDE), Le Kremlin-Biĉetre, France
| | - Guillaume Tochon
- EPITA Research and Development Laboratory (LRDE), Le Kremlin-Biĉetre, France
| | - Jun Ma
- School of Science, Nanjing University of Science and Technology
| | - Mariano Cabezas
- Research Institute of Computer Vision and Robotics, University of Girona, Spain
| | - Xavier Llado
- Research Institute of Computer Vision and Robotics, University of Girona, Spain
| | - Arnau Oliver
- Research Institute of Computer Vision and Robotics, University of Girona, Spain
| | - Liliana Valencia
- Research Institute of Computer Vision and Robotics, University of Girona, Spain
| | - Sergi Valverde
- Research Institute of Computer Vision and Robotics, University of Girona, Spain
| | - Mehdi Amian
- Department of Electrical and Computer Engineering, University of Tehran, Iran
| | | | | | | | - Xue Feng
- Biomedical Engineering, University of Virginia, Charlottesville, USA
| | - Quan Dou
- Biomedical Engineering, University of Virginia, Charlottesville, USA
| | - Nicholas Tustison
- Radiology and Medical Imaging, University of Virginia, Charlottesville, USA
| | - Craig Meyer
- Biomedical Engineering, University of Virginia, Charlottesville, USA
- Radiology and Medical Imaging, University of Virginia, Charlottesville, USA
| | - Nisarg A. Shah
- Department of Electrical Engineering, Indian Institute of Technology - Jodhpur, Jodhpur, India
| | - Sanjay Talbar
- SGGS Institute of Engineering and Technology, Nanded, India
| | - Marc-André Weber
- Institute of Diagnostic and Interventional Radiology, Pediatric Radiology and Neuroradiology, University Medical Center Rostock, Rostock, Germany
| | - Abhishek Mahajan
- Tata Memorial Centre, Homi Bhabha National Institute, Mumbai, India
| | - Andras Jakab
- Center for MR-Research, University Children’s Hospital Zurich, Zurich, Switzerland
| | - Roland Wiest
- Support Center for Advanced Neuroimaging (SCAN), University Institute of Diagnostic and Interventional Neuroradiology, University of Bern, Inselspital, Bern University Hospital, Bern, Switzerland
- Institute for Surgical Technology and Biomechanics, University of Bern, Bern, Switzerland
| | | | - Arash Nazeri
- Department of Radiology, Washington University, St. Louis, MO, USA
| | - Mikhail Milchenko1
- Department of Radiology, Washington University, St. Louis, MO, USA
- Neuroimaging Informatics and Analysis Center, Washington University, St. Louis, MO, USA
| | - Daniel Marcus
- Department of Radiology, Washington University, St. Louis, MO, USA
- Neuroimaging Informatics and Analysis Center, Washington University, St. Louis, MO, USA
| | - Aikaterini Kotrotsou
- Department of Diagnostic Radiology, University of Texas MD Anderson Cancer Center, Houston, TX, USA
| | - Rivka Colen
- Department of Diagnostic Radiology, University of Texas MD Anderson Cancer Center, Houston, TX, USA
| | - John Freymann
- Leidos Biomedical Research, Inc, Frederick National Laboratory for Cancer Research, Frederick, MD, USA
- Cancer Imaging Program, National Cancer Institute, National Institutes of Health, Bethesda, MD, USA
| | - Justin Kirby
- Leidos Biomedical Research, Inc, Frederick National Laboratory for Cancer Research, Frederick, MD, USA
- Cancer Imaging Program, National Cancer Institute, National Institutes of Health, Bethesda, MD, USA
| | - Christos Davatzikos
- Center for Biomedical Image Computing and Analytics (CBICA), University of Pennsylvania, Philadelphia, PA, USA
- Department of Radiology, Perelman School of Medicine at the University of Pennsylvania, Philadelphia, PA, USA
| | - Bjoern Menze
- Department of Quantitative Biomedicine, University of Zurich, Zurich, Switzerland
- Department of Informatics, Technical University of Munich, Munich, Germany
| | - Spyridon Bakas
- Center for Biomedical Image Computing and Analytics (CBICA), University of Pennsylvania, Philadelphia, PA, USA
- Department of Radiology, Perelman School of Medicine at the University of Pennsylvania, Philadelphia, PA, USA
- Department of Pathology and Laboratory Medicine, Perelman School of Medicine, University of Pennsylvania, Philadelphia, PA, USA
| | - Yarin Gal
- Oxford Applied and Theoretical Machine Learning (OATML) Group, University of Oxford, Oxford, England
| | - Tal Arbel
- Centre for Intelligent Machines (CIM), McGill University, Montreal, QC, Canada
- MILA - Quebec Artificial Intelligence Institute, Montreal, QC, Canada
| |
Collapse
|
4
|
Xiong Z, Xia Q, Hu Z, Huang N, Bian C, Zheng Y, Vesal S, Ravikumar N, Maier A, Yang X, Heng PA, Ni D, Li C, Tong Q, Si W, Puybareau E, Khoudli Y, Géraud T, Chen C, Bai W, Rueckert D, Xu L, Zhuang X, Luo X, Jia S, Sermesant M, Liu Y, Wang K, Borra D, Masci A, Corsi C, de Vente C, Veta M, Karim R, Preetha CJ, Engelhardt S, Qiao M, Wang Y, Tao Q, Nuñez-Garcia M, Camara O, Savioli N, Lamata P, Zhao J. A global benchmark of algorithms for segmenting the left atrium from late gadolinium-enhanced cardiac magnetic resonance imaging. Med Image Anal 2021; 67:101832. [PMID: 33166776 DOI: 10.1016/j.media.2020.101832] [Citation(s) in RCA: 73] [Impact Index Per Article: 24.3] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/14/2020] [Revised: 09/21/2020] [Accepted: 09/23/2020] [Indexed: 12/29/2022]
Abstract
Segmentation of medical images, particularly late gadolinium-enhanced magnetic resonance imaging (LGE-MRI) used for visualizing diseased atrial structures, is a crucial first step for ablation treatment of atrial fibrillation. However, direct segmentation of LGE-MRIs is challenging due to the varying intensities caused by contrast agents. Since most clinical studies have relied on manual, labor-intensive approaches, automatic methods are of high interest, particularly optimized machine learning approaches. To address this, we organized the 2018 Left Atrium Segmentation Challenge using 154 3D LGE-MRIs, currently the world's largest atrial LGE-MRI dataset, and associated labels of the left atrium segmented by three medical experts, ultimately attracting the participation of 27 international teams. In this paper, extensive analysis of the submitted algorithms using technical and biological metrics was performed by undergoing subgroup analysis and conducting hyper-parameter analysis, offering an overall picture of the major design choices of convolutional neural networks (CNNs) and practical considerations for achieving state-of-the-art left atrium segmentation. Results show that the top method achieved a Dice score of 93.2% and a mean surface to surface distance of 0.7 mm, significantly outperforming prior state-of-the-art. Particularly, our analysis demonstrated that double sequentially used CNNs, in which a first CNN is used for automatic region-of-interest localization and a subsequent CNN is used for refined regional segmentation, achieved superior results than traditional methods and machine learning approaches containing single CNNs. This large-scale benchmarking study makes a significant step towards much-improved segmentation methods for atrial LGE-MRIs, and will serve as an important benchmark for evaluating and comparing the future works in the field. Furthermore, the findings from this study can potentially be extended to other imaging datasets and modalities, having an impact on the wider medical imaging community.
Collapse
Affiliation(s)
- Zhaohan Xiong
- Auckland Bioengineering Institute, University of Auckland, Auckland, New Zealand
| | - Qing Xia
- State Key Lab of Virtual Reality Technology and Systems, Beihang University, Beijing, China
| | - Zhiqiang Hu
- School of Electronics Engineering and Computer Science, Peking University, Beijing, China
| | | | - Cheng Bian
- Tencent Jarvis Laboratory, Shenzhen, China
| | | | - Sulaiman Vesal
- Pattern Recognition Lab, Friedrich-Alexander-Universität Erlangen-Nürnberg, Erlangen, Germany
| | - Nishant Ravikumar
- Pattern Recognition Lab, Friedrich-Alexander-Universität Erlangen-Nürnberg, Erlangen, Germany
| | - Andreas Maier
- Pattern Recognition Lab, Friedrich-Alexander-Universität Erlangen-Nürnberg, Erlangen, Germany
| | - Xin Yang
- Department of Computer Science and Engineering, The Chinese University of Hong Kong, Hong Kong
| | - Pheng-Ann Heng
- Department of Computer Science and Engineering, The Chinese University of Hong Kong, Hong Kong
| | - Dong Ni
- National-Regional Key Technology Engineering Laboratory for Medical Ultrasound, School of Biomedical Engineering, Health Science Center, Shenzhen University, Shenzhen, China
| | - Caizi Li
- School of Computer Science, Wuhan University, Wuhan, China
| | - Qianqian Tong
- School of Computer Science, Wuhan University, Wuhan, China
| | - Weixin Si
- Shenzhen Key Laboratory of Virtual Reality and Human Interaction Technology, Shenzhen Institutes of Advanced Technology, Chinese Academy of Sciences, Shenzhen, China
| | | | - Younes Khoudli
- EPITA Research and Development Laboratory, Paris, France
| | - Thierry Géraud
- EPITA Research and Development Laboratory, Paris, France
| | - Chen Chen
- Department of Computing, Imperial College London, London, United Kingdom
| | - Wenjia Bai
- Department of Computing, Imperial College London, London, United Kingdom
| | - Daniel Rueckert
- Department of Computing, Imperial College London, London, United Kingdom
| | - Lingchao Xu
- School of Naval Architecture, Ocean & Civil Engineering, Shanghai Jiao Tong University, Shanghai, China
| | - Xiahai Zhuang
- School of Data Science, Fudan University, Shanghai, China
| | - Xinzhe Luo
- School of Data Science, Fudan University, Shanghai, China
| | - Shuman Jia
- Inria, Université Côte d'Azur, Epione team, Sophia Antipolis, France
| | - Maxime Sermesant
- Inria, Université Côte d'Azur, Epione team, Sophia Antipolis, France
| | - Yashu Liu
- School of Computer Science and Technology, Harbin Institute of Technology, Harbin, China
| | - Kuanquan Wang
- School of Computer Science and Technology, Harbin Institute of Technology, Harbin, China
| | - Davide Borra
- Department of Electric, Electronic and Information Engineering, University of Bologna, Cesena, Italy
| | - Alessandro Masci
- Department of Electric, Electronic and Information Engineering, University of Bologna, Cesena, Italy
| | - Cristiana Corsi
- Department of Electric, Electronic and Information Engineering, University of Bologna, Cesena, Italy
| | - Coen de Vente
- Department of Biomedical Engineering, Eindhoven University of Technology, Eindhoven, the Netherlands
| | - Mitko Veta
- Department of Biomedical Engineering, Eindhoven University of Technology, Eindhoven, the Netherlands
| | - Rashed Karim
- School of Biomedical Engineering & Imaging Sciences, Kings College London, London, United Kingdom
| | | | - Sandy Engelhardt
- Department of Internal Medicine III, Heidelberg University Hospital, Heidelberg, Germany
| | - Menyun Qiao
- Biomedical Engineering Center, Fudan University, Shanghai, China
| | - Yuanyuan Wang
- Biomedical Engineering Center, Fudan University, Shanghai, China
| | - Qian Tao
- Department of Radiology, Leiden University Medical Center, Leiden, the Netherlands
| | - Marta Nuñez-Garcia
- Physense, Department of Information and Communication Technologies, Universitat Pompeu Fabra, Barcelona, Spain
| | - Oscar Camara
- Physense, Department of Information and Communication Technologies, Universitat Pompeu Fabra, Barcelona, Spain
| | - Nicolo Savioli
- Department of Bioengineering, Kings College London, London, United Kingdom
| | - Pablo Lamata
- Department of Bioengineering, Kings College London, London, United Kingdom
| | - Jichao Zhao
- Auckland Bioengineering Institute, University of Auckland, Auckland, New Zealand.
| |
Collapse
|
5
|
Kim YJ, Jang H, Lee K, Park S, Min SG, Hong C, Park JH, Lee K, Kim J, Hong W, Jung H, Liu Y, Rajkumar H, Khened M, Krishnamurthi G, Yang S, Wang X, Han CH, Kwak JT, Ma J, Tang Z, Marami B, Zeineh J, Zhao Z, Heng PA, Schmitz R, Madesta F, Rösch T, Werner R, Tian J, Puybareau E, Bovio M, Zhang X, Zhu Y, Chun SY, Jeong WK, Park P, Choi J. PAIP 2019: Liver cancer segmentation challenge. Med Image Anal 2020; 67:101854. [PMID: 33091742 DOI: 10.1016/j.media.2020.101854] [Citation(s) in RCA: 23] [Impact Index Per Article: 5.8] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/31/2019] [Revised: 07/06/2020] [Accepted: 09/03/2020] [Indexed: 01/22/2023]
Abstract
Pathology Artificial Intelligence Platform (PAIP) is a free research platform in support of pathological artificial intelligence (AI). The main goal of the platform is to construct a high-quality pathology learning data set that will allow greater accessibility. The PAIP Liver Cancer Segmentation Challenge, organized in conjunction with the Medical Image Computing and Computer Assisted Intervention Society (MICCAI 2019), is the first image analysis challenge to apply PAIP datasets. The goal of the challenge was to evaluate new and existing algorithms for automated detection of liver cancer in whole-slide images (WSIs). Additionally, the PAIP of this year attempted to address potential future problems of AI applicability in clinical settings. In the challenge, participants were asked to use analytical data and statistical metrics to evaluate the performance of automated algorithms in two different tasks. The participants were given the two different tasks: Task 1 involved investigating Liver Cancer Segmentation and Task 2 involved investigating Viable Tumor Burden Estimation. There was a strong correlation between high performance of teams on both tasks, in which teams that performed well on Task 1 also performed well on Task 2. After evaluation, we summarized the top 11 team's algorithms. We then gave pathological implications on the easily predicted images for cancer segmentation and the challenging images for viable tumor burden estimation. Out of the 231 participants of the PAIP challenge datasets, a total of 64 were submitted from 28 team participants. The submitted algorithms predicted the automatic segmentation on the liver cancer with WSIs to an accuracy of a score estimation of 0.78. The PAIP challenge was created in an effort to combat the lack of research that has been done to address Liver cancer using digital pathology. It remains unclear of how the applicability of AI algorithms created during the challenge can affect clinical diagnoses. However, the results of this dataset and evaluation metric provided has the potential to aid the development and benchmarking of cancer diagnosis and segmentation.
Collapse
Affiliation(s)
- Yoo Jung Kim
- Department of Biomedical Engineering, Seoul National University Hospital, Seoul, South Korea.
| | - Hyungjoon Jang
- School of Electrical and Computer Engineering, Ulsan National Institute of Science and Technology, Ulsan, South Korea.
| | - Kyoungbun Lee
- Department of Pathology, Seoul National University Hospital, Seoul, South Korea.
| | - Seongkeun Park
- Department of Biomedical Engineering, Seoul National University Hospital, Seoul, South Korea
| | - Sung-Gyu Min
- Department of Pathology, Seoul National University Hospital, Seoul, South Korea
| | - Choyeon Hong
- Department of Pathology, Seoul National University Hospital, Seoul, South Korea
| | - Jeong Hwan Park
- Department of Pathology, Seoul Metropolitan Government-Seoul National University Boramae Medical Center, Seoul, South Korea
| | - Kanggeun Lee
- School of Electrical and Computer Engineering, Ulsan National Institute of Science and Technology, Ulsan, South Korea
| | - Jisoo Kim
- School of Electrical and Computer Engineering, Ulsan National Institute of Science and Technology, Ulsan, South Korea
| | - Wonjae Hong
- School of Electrical and Computer Engineering, Ulsan National Institute of Science and Technology, Ulsan, South Korea
| | - Hyun Jung
- Frederick National Laboratory for Cancer Research, Frederick, Maryland, United States
| | - Yanling Liu
- Frederick National Laboratory for Cancer Research, Frederick, Maryland, United States
| | - Haran Rajkumar
- Department of Engineering Design, Indian Institute Of Technology Madras, Chennai, Tamil Nadu, India
| | - Mahendra Khened
- Department of Engineering Design, Indian Institute Of Technology Madras, Chennai, Tamil Nadu, India
| | - Ganapathy Krishnamurthi
- Department of Engineering Design, Indian Institute Of Technology Madras, Chennai, Tamil Nadu, India
| | - Sen Yang
- Sichuan University and Tencent AI Lab, Chengdu, Sichuan, China
| | - Xiyue Wang
- College of Computer Science, Sichuan University, Chengdu, Sichuan, China
| | - Chang Hee Han
- Department of Computer Science and Engineering, Sejong University, Seoul, South Korea
| | - Jin Tae Kwak
- Department of Computer Science and Engineering, Sejong University, Seoul, South Korea
| | | | | | - Bahram Marami
- The Center for Computational and Systems Pathology, Icahn School of Medicine at Mount Sinai, New York, NY, United States
| | - Jack Zeineh
- The Center for Computational and Systems Pathology, Icahn School of Medicine at Mount Sinai, New York, NY, United States
| | - Zixu Zhao
- Department of Computer Science and Engineering, The Chinese University of Hong Kong, Hong Kong
| | - Pheng-Ann Heng
- Department of Computer Science and Engineering, The Chinese University of Hong Kong, Hong Kong
| | - Rüdiger Schmitz
- Department for Interdisciplinary Endoscopy, University Medical Center Hamburg-Eppendorf, Hamburg, Germany; DAISYlabs, Forschungszentrum Medizintechnik Hamburg, Hamburg, Germany
| | - Frederic Madesta
- Instutute of Computational Neuroscience, University Medical Center Hamburg-Eppendorf, Hamburg, Germany; DAISYlabs, Forschungszentrum Medizintechnik Hamburg, Hamburg, Germany
| | - Thomas Rösch
- Department for Interdisciplinary Endoscopy, University Medical Center Hamburg-Eppendorf, Hamburg, Germany
| | - Rene Werner
- Instutute of Computational Neuroscience, University Medical Center Hamburg-Eppendorf, Hamburg, Germany; DAISYlabs, Forschungszentrum Medizintechnik Hamburg, Hamburg, Germany
| | - Jie Tian
- Shanghai JiaoTong University, Shanghai, China
| | | | | | | | - Yifeng Zhu
- University of Maine, Orono, ME, United States
| | - Se Young Chun
- School of Electrical and Computer Engineering, Ulsan National Institute of Science and Technology, Ulsan, South Korea.
| | - Won-Ki Jeong
- Department of Computer Science and Engineering, College of Informatics, Korea University, Seoul, 02841, Korea.
| | | | - Jinwook Choi
- Department of Biomedical Engineering, Seoul National University Hospital, Seoul, South Korea.
| |
Collapse
|
6
|
Kuijf HJ, Biesbroek JM, De Bresser J, Heinen R, Andermatt S, Bento M, Berseth M, Belyaev M, Cardoso MJ, Casamitjana A, Collins DL, Dadar M, Georgiou A, Ghafoorian M, Jin D, Khademi A, Knight J, Li H, Llado X, Luna M, Mahmood Q, McKinley R, Mehrtash A, Ourselin S, Park BY, Park H, Park SH, Pezold S, Puybareau E, Rittner L, Sudre CH, Valverde S, Vilaplana V, Wiest R, Xu Y, Xu Z, Zeng G, Zhang J, Zheng G, Chen C, van der Flier W, Barkhof F, Viergever MA, Biessels GJ. Standardized Assessment of Automatic Segmentation of White Matter Hyperintensities and Results of the WMH Segmentation Challenge. IEEE Trans Med Imaging 2019; 38:2556-2568. [PMID: 30908194 PMCID: PMC7590957 DOI: 10.1109/tmi.2019.2905770] [Citation(s) in RCA: 103] [Impact Index Per Article: 20.6] [Reference Citation Analysis] [What about the content of this article? (0)] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 05/23/2023]
Abstract
Quantification of cerebral white matter hyperintensities (WMH) of presumed vascular origin is of key importance in many neurological research studies. Currently, measurements are often still obtained from manual segmentations on brain MR images, which is a laborious procedure. The automatic WMH segmentation methods exist, but a standardized comparison of the performance of such methods is lacking. We organized a scientific challenge, in which developers could evaluate their methods on a standardized multi-center/-scanner image dataset, giving an objective comparison: the WMH Segmentation Challenge. Sixty T1 + FLAIR images from three MR scanners were released with the manual WMH segmentations for training. A test set of 110 images from five MR scanners was used for evaluation. The segmentation methods had to be containerized and submitted to the challenge organizers. Five evaluation metrics were used to rank the methods: 1) Dice similarity coefficient; 2) modified Hausdorff distance (95th percentile); 3) absolute log-transformed volume difference; 4) sensitivity for detecting individual lesions; and 5) F1-score for individual lesions. In addition, the methods were ranked on their inter-scanner robustness; 20 participants submitted their methods for evaluation. This paper provides a detailed analysis of the results. In brief, there is a cluster of four methods that rank significantly better than the other methods, with one clear winner. The inter-scanner robustness ranking shows that not all the methods generalize to unseen scanners. The challenge remains open for future submissions and provides a public platform for method evaluation.
Collapse
|
7
|
Wang L, Nie D, Li G, Puybareau E, Dolz J, Zhang Q, Wang F, Xia J, Wu Z, Chen J, Thung KH, Bui TD, Shin J, Zeng G, Zheng G, Fonov VS, Doyle A, Xu Y, Moeskops P, Pluim JPW, Desrosiers C, Ayed IB, Sanroma G, Benkarim OM, Casamitjana A, Vilaplana V, Lin W, Li G, Shen D. Benchmark on Automatic 6-month-old Infant Brain Segmentation Algorithms: The iSeg-2017 Challenge. IEEE Trans Med Imaging 2019; 38:2219-2230. [PMID: 30835215 PMCID: PMC6754324 DOI: 10.1109/tmi.2019.2901712] [Citation(s) in RCA: 63] [Impact Index Per Article: 12.6] [Reference Citation Analysis] [What about the content of this article? (0)] [Abstract] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 05/09/2023]
Abstract
Accurate segmentation of infant brain magnetic resonance (MR) images into white matter (WM), gray matter (GM), and cerebrospinal fluid (CSF) is an indispensable foundation for early studying of brain growth patterns and morphological changes in neurodevelopmental disorders. Nevertheless, in the isointense phase (approximately 6-9 months of age), due to inherent myelination and maturation process, WM and GM exhibit similar levels of intensity in both T1-weighted (T1w) and T2-weighted (T2w) MR images, making tissue segmentation very challenging. Despite many efforts were devoted to brain segmentation, only few studies have focused on the segmentation of 6-month infant brain images. With the idea of boosting methodological development in the community, iSeg-2017 challenge (http://iseg2017.web.unc.edu) provides a set of 6-month infant subjects with manual labels for training and testing the participating methods. Among the 21 automatic segmentation methods participating in iSeg-2017, we review the 8 top-ranked teams, in terms of Dice ratio, modified Hausdorff distance and average surface distance, and introduce their pipelines, implementations, as well as source codes. We further discuss limitations and possible future directions. We hope the dataset in iSeg-2017 and this review article could provide insights into methodological development for the community.
Collapse
|
8
|
Genest D, Puybareau E, Léonard M, Cousty J, De Crozé N, Talbot H. High throughput automated detection of axial malformations in Medaka embryo. Comput Biol Med 2019; 105:157-168. [PMID: 30654166 DOI: 10.1016/j.compbiomed.2018.12.016] [Citation(s) in RCA: 3] [Impact Index Per Article: 0.6] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/08/2018] [Revised: 12/18/2018] [Accepted: 12/29/2018] [Indexed: 11/18/2022]
Abstract
Fish embryo models are widely used as screening tools to assess the efficacy and/or toxicity of chemicals. This assessment involves the analysis of embryo morphological abnormalities. In this article, we propose a multi-scale pipeline to allow automated classification of fish embryos (Medaka: Oryzias latipes) based on the presence or absence of spine malformations. The proposed pipeline relies on the acquisition of fish embryo 2D images, on feature extraction based on mathematical morphology operators and on machine learning classification. After image acquisition, segmentation tools are used to detect the embryo before analysing several morphological features. An approach based on machine learning is then applied to these features to automatically classify embryos according to the presence of axial malformations. We built and validated our learning model on 1459 images with a 10-fold cross-validation by comparison with the gold standard of 3D observations performed under a microscope by a trained operator. Our pipeline results in correct classification in 85% of the cases included in the database. This percentage is similar to the percentage of success of a trained human operator working on 2D images. The key benefit of our approach is the low computational cost of our image analysis pipeline, which guarantees optimal throughput analysis.
Collapse
Affiliation(s)
- Diane Genest
- Université Paris-Est, LIGM (UMR 8049), CNRS, ENPC, ESIEE, UPEM, 2 Boulevard Blaise Pascal, 93162, Noisy-le Grand, France; L'OREAL Research & Innovation, 1 avenue Eugène Schueller, 93600, Aulnay sous Bois, France.
| | - Elodie Puybareau
- Université Paris-Est, LIGM (UMR 8049), CNRS, ENPC, ESIEE, UPEM, 2 Boulevard Blaise Pascal, 93162, Noisy-le Grand, France; EPITA Research and Development Laboratory (LRDE), 14-16 rue Voltaire, 94270, Le Kremlin-Bicêtre, France.
| | - Marc Léonard
- L'OREAL Research & Innovation, 1 avenue Eugène Schueller, 93600, Aulnay sous Bois, France
| | - Jean Cousty
- Université Paris-Est, LIGM (UMR 8049), CNRS, ENPC, ESIEE, UPEM, 2 Boulevard Blaise Pascal, 93162, Noisy-le Grand, France
| | - Noémie De Crozé
- L'OREAL Research & Innovation, 1 avenue Eugène Schueller, 93600, Aulnay sous Bois, France
| | - Hugues Talbot
- Université Paris-Est, LIGM (UMR 8049), CNRS, ENPC, ESIEE, UPEM, 2 Boulevard Blaise Pascal, 93162, Noisy-le Grand, France
| |
Collapse
|
9
|
Puybareau E, Genest D, Barbeau E, Léonard M, Talbot H. An automated assay for the assessment of cardiac arrest in fish embryo. Comput Biol Med 2017; 81:32-44. [DOI: 10.1016/j.compbiomed.2016.12.007] [Citation(s) in RCA: 4] [Impact Index Per Article: 0.6] [Reference Citation Analysis] [What about the content of this article? (0)] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/06/2016] [Revised: 12/09/2016] [Accepted: 12/11/2016] [Indexed: 10/20/2022]
|
10
|
Puybareau E, Talbot H, Pelle G, Louis B, Najman L, Coste A. Automatic detection of beating cilia with frequencies estimations. Cilia 2015. [PMCID: PMC4518898 DOI: 10.1186/2046-2530-4-s1-p85] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Track Full Text] [Download PDF] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/10/2022] Open
|