1
|
Antonelli M, Reinke A, Bakas S, Farahani K, Kopp-Schneider A, Landman BA, Litjens G, Menze B, Ronneberger O, Summers RM, van Ginneken B, Bilello M, Bilic P, Christ PF, Do RKG, Gollub MJ, Heckers SH, Huisman H, Jarnagin WR, McHugo MK, Napel S, Pernicka JSG, Rhode K, Tobon-Gomez C, Vorontsov E, Meakin JA, Ourselin S, Wiesenfarth M, Arbeláez P, Bae B, Chen S, Daza L, Feng J, He B, Isensee F, Ji Y, Jia F, Kim I, Maier-Hein K, Merhof D, Pai A, Park B, Perslev M, Rezaiifar R, Rippel O, Sarasua I, Shen W, Son J, Wachinger C, Wang L, Wang Y, Xia Y, Xu D, Xu Z, Zheng Y, Simpson AL, Maier-Hein L, Cardoso MJ. The Medical Segmentation Decathlon. Nat Commun 2022; 13:4128. [PMID: 35840566 PMCID: PMC9287542 DOI: 10.1038/s41467-022-30695-9] [Citation(s) in RCA: 115] [Impact Index Per Article: 57.5] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/16/2021] [Accepted: 05/13/2022] [Indexed: 02/05/2023] Open
Abstract
International challenges have become the de facto standard for comparative assessment of image analysis algorithms. Although segmentation is the most widely investigated medical image processing task, the various challenges have been organized to focus only on specific clinical tasks. We organized the Medical Segmentation Decathlon (MSD)-a biomedical image analysis challenge, in which algorithms compete in a multitude of both tasks and modalities to investigate the hypothesis that a method capable of performing well on multiple tasks will generalize well to a previously unseen task and potentially outperform a custom-designed solution. MSD results confirmed this hypothesis, moreover, MSD winner continued generalizing well to a wide range of other clinical problems for the next two years. Three main conclusions can be drawn from this study: (1) state-of-the-art image segmentation algorithms generalize well when retrained on unseen tasks; (2) consistent algorithmic performance across multiple tasks is a strong surrogate of algorithmic generalizability; (3) the training of accurate AI segmentation models is now commoditized to scientists that are not versed in AI model training.
Collapse
Affiliation(s)
- Michela Antonelli
- School of Biomedical Engineering & Imaging Sciences, King's College London, London, UK.
| | - Annika Reinke
- Div. Computer Assisted Medical Interventions, German Cancer Research Center (DKFZ), Heidelberg, Germany.,HI Helmholtz Imaging, German Cancer Research Center (DKFZ), Heidelberg, Germany.,Faculty of Mathematics and Computer Science, University of Heidelberg, Heidelberg, Germany
| | - Spyridon Bakas
- Center for Biomedical Image Computing and Analytics (CBICA), University of Pennsylvania, Philadelphia, PA, USA.,Department of Radiology, Perelman School of Medicine, University of Pennsylvania, Philadelphia, PA, USA.,Department of Pathology and Laboratory Medicine, Perelman School of Medicine, University of Pennsylvania, Philadelphia, PA, USA
| | - Keyvan Farahani
- Center for Biomedical Informatics and Information Technology, National Cancer Institute (NIH), Bethesda, MD, USA
| | | | - Bennett A Landman
- Electrical Engineering and Computer Science, Vanderbilt University, Nashville, TN, USA
| | - Geert Litjens
- Radboud University Medical Center, Radboud Institute for Health Sciences, Nijmegen, The Netherlands
| | - Bjoern Menze
- Quantitative Biomedicine, University of Zurich, Zurich, Switzerland
| | | | - Ronald M Summers
- Imaging Biomarkers and Computer-Aided Diagnosis Laboratory, Department of Radiology and Imaging Sciences, National Institutes of Health Clinical Center (NIH), Bethesda, MD, USA
| | - Bram van Ginneken
- Radboud University Medical Center, Radboud Institute for Health Sciences, Nijmegen, The Netherlands
| | - Michel Bilello
- Center for Biomedical Image Computing and Analytics (CBICA), University of Pennsylvania, Philadelphia, PA, USA
| | - Patrick Bilic
- Department of Informatics, Technische Universität München, München, Germany
| | - Patrick F Christ
- Department of Informatics, Technische Universität München, München, Germany
| | - Richard K G Do
- Department of Radiology, Memorial Sloan Kettering Cancer Center, New York, NY, USA
| | - Marc J Gollub
- Department of Radiology, Memorial Sloan Kettering Cancer Center, New York, NY, USA
| | - Stephan H Heckers
- Department of Psychiatry & Behavioral Sciences, Vanderbilt University Medical Center, Nashville, TN, USA
| | - Henkjan Huisman
- Radboud University Medical Center, Radboud Institute for Health Sciences, Nijmegen, The Netherlands
| | - William R Jarnagin
- Department of Surgery, Memorial Sloan Kettering Cancer Center, New York, NY, USA
| | - Maureen K McHugo
- Department of Psychiatry & Behavioral Sciences, Vanderbilt University Medical Center, Nashville, TN, USA
| | - Sandy Napel
- Department of Radiology, Stanford University, Stanford, CA, USA
| | | | - Kawal Rhode
- School of Biomedical Engineering & Imaging Sciences, King's College London, London, UK
| | - Catalina Tobon-Gomez
- School of Biomedical Engineering & Imaging Sciences, King's College London, London, UK
| | - Eugene Vorontsov
- Department of Computer Science and Software Engineering, École Polytechnique de Montréal, Montréal, QC, Canada
| | - James A Meakin
- Radboud University Medical Center, Radboud Institute for Health Sciences, Nijmegen, The Netherlands
| | - Sebastien Ourselin
- School of Biomedical Engineering & Imaging Sciences, King's College London, London, UK
| | - Manuel Wiesenfarth
- Div. Biostatistics, German Cancer Research Center (DKFZ), Heidelberg, Germany
| | | | | | | | - Laura Daza
- Universidad de los Andes, Bogota, Colombia
| | - Jianjiang Feng
- Department of Automation, Tsinghua University, Beijing, China
| | - Baochun He
- Shenzhen Institute of Advanced Technology, Chinese Academy of Sciences, Shenzhen, China
| | - Fabian Isensee
- HI Applied Computer Vision Lab, Division of Medical Image Computing, German Cancer Research Center (DKFZ), Heidelberg, Germany
| | - Yuanfeng Ji
- Department of Computer Science, Xiamen University, Xiamen, China
| | - Fucang Jia
- Shenzhen Institute of Advanced Technology, Chinese Academy of Sciences, Shenzhen, China
| | - Ildoo Kim
- Kakao Brain, Seongnam-si, Republic of Korea
| | - Klaus Maier-Hein
- Cerebriu A/S, Copenhagen, Denmark.,Pattern Analysis and Learning Group, Department of Radiation Oncology, Heidelberg University Hospital, Heidelberg, Germany
| | - Dorit Merhof
- Institute of Imaging & Computer Vision, RWTH Aachen University, Aachen, Germany.,Fraunhofer Institute for Digital Medicine MEVIS, Bremen, Germany
| | - Akshay Pai
- Cerebriu A/S, Copenhagen, Denmark.,Department of Computer Science, University of Copenhagen, Copenhagen, Denmark
| | | | - Mathias Perslev
- Department of Computer Science, University of Copenhagen, Copenhagen, Denmark
| | | | - Oliver Rippel
- Institute of Imaging & Computer Vision, RWTH Aachen University, Aachen, Germany
| | - Ignacio Sarasua
- Lab for Artificial Intelligence in Medical Imaging (AI-Med), Department of Child and Adolescent Psychiatry, University Hospital, LMU München, Germany
| | - Wei Shen
- MoE Key Lab of Artificial Intelligence, AI Institute, Shanghai Jiao Tong University, Shanghai, China
| | | | - Christian Wachinger
- Lab for Artificial Intelligence in Medical Imaging (AI-Med), Department of Child and Adolescent Psychiatry, University Hospital, LMU München, Germany
| | - Liansheng Wang
- Department of Computer Science, Xiamen University, Xiamen, China
| | - Yan Wang
- Shanghai Key Laboratory of Multidimensional Information Processing, East China Normal University, Shanghai, China
| | - Yingda Xia
- Johns Hopkins University, Baltimore, MD, USA
| | | | - Zhanwei Xu
- Department of Automation, Tsinghua University, Beijing, China
| | | | - Amber L Simpson
- School of Computing/Department of Biomedical and Molecular Sciences, Queen's University, Kingston, ON, Canada
| | - Lena Maier-Hein
- Div. Computer Assisted Medical Interventions, German Cancer Research Center (DKFZ), Heidelberg, Germany.,HI Helmholtz Imaging, German Cancer Research Center (DKFZ), Heidelberg, Germany.,Faculty of Mathematics and Computer Science, University of Heidelberg, Heidelberg, Germany.,Medical Faculty, University of Heidelberg, Heidelberg, Germany
| | - M Jorge Cardoso
- School of Biomedical Engineering & Imaging Sciences, King's College London, London, UK
| |
Collapse
|