3
|
Carass A, Roy S, Jog A, Cuzzocreo JL, Magrath E, Gherman A, Button J, Nguyen J, Prados F, Sudre CH, Jorge Cardoso M, Cawley N, Ciccarelli O, Wheeler-Kingshott CAM, Ourselin S, Catanese L, Deshpande H, Maurel P, Commowick O, Barillot C, Tomas-Fernandez X, Warfield SK, Vaidya S, Chunduru A, Muthuganapathy R, Krishnamurthi G, Jesson A, Arbel T, Maier O, Handels H, Iheme LO, Unay D, Jain S, Sima DM, Smeets D, Ghafoorian M, Platel B, Birenbaum A, Greenspan H, Bazin PL, Calabresi PA, Crainiceanu CM, Ellingsen LM, Reich DS, Prince JL, Pham DL. Longitudinal multiple sclerosis lesion segmentation: Resource and challenge. Neuroimage 2017; 148:77-102. [PMID: 28087490 PMCID: PMC5344762 DOI: 10.1016/j.neuroimage.2016.12.064] [Citation(s) in RCA: 131] [Impact Index Per Article: 18.7] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/19/2016] [Revised: 11/15/2016] [Accepted: 12/19/2016] [Indexed: 01/12/2023] Open
Abstract
In conjunction with the ISBI 2015 conference, we organized a longitudinal lesion segmentation challenge providing training and test data to registered participants. The training data consisted of five subjects with a mean of 4.4 time-points, and test data of fourteen subjects with a mean of 4.4 time-points. All 82 data sets had the white matter lesions associated with multiple sclerosis delineated by two human expert raters. Eleven teams submitted results using state-of-the-art lesion segmentation algorithms to the challenge, with ten teams presenting their results at the conference. We present a quantitative evaluation comparing the consistency of the two raters as well as exploring the performance of the eleven submitted results in addition to three other lesion segmentation algorithms. The challenge presented three unique opportunities: (1) the sharing of a rich data set; (2) collaboration and comparison of the various avenues of research being pursued in the community; and (3) a review and refinement of the evaluation metrics currently in use. We report on the performance of the challenge participants, as well as the construction and evaluation of a consensus delineation. The image data and manual delineations will continue to be available for download, through an evaluation website2 as a resource for future researchers in the area. This data resource provides a platform to compare existing methods in a fair and consistent manner to each other and multiple manual raters.
Collapse
Affiliation(s)
- Aaron Carass
- Department of Electrical and Computer Engineering, The Johns Hopkins University, Baltimore, MD 21218, USA; Department of Computer Science, The Johns Hopkins University, Baltimore, MD 21218, USA.
| | - Snehashis Roy
- CNRM, The Henry M. Jackson Foundation for the Advancement of Military Medicine, Bethesda, MD 20892, USA
| | - Amod Jog
- Department of Computer Science, The Johns Hopkins University, Baltimore, MD 21218, USA
| | - Jennifer L Cuzzocreo
- Department of Radiology, The Johns Hopkins School of Medicine, Baltimore, MD 21287, USA
| | - Elizabeth Magrath
- CNRM, The Henry M. Jackson Foundation for the Advancement of Military Medicine, Bethesda, MD 20892, USA
| | - Adrian Gherman
- Department of Biostatistics, The Johns Hopkins University, Baltimore, MD 21205, USA
| | - Julia Button
- Department of Radiology, The Johns Hopkins School of Medicine, Baltimore, MD 21287, USA
| | - James Nguyen
- Department of Radiology, The Johns Hopkins School of Medicine, Baltimore, MD 21287, USA
| | - Ferran Prados
- Translational Imaging Group, CMIC, UCL, NW1 2HE London, UK; NMR Research Unit, UCL Institute of Neurology, WC1N 3BG London, UK
| | - Carole H Sudre
- Translational Imaging Group, CMIC, UCL, NW1 2HE London, UK
| | - Manuel Jorge Cardoso
- Translational Imaging Group, CMIC, UCL, NW1 2HE London, UK; Dementia Research Centre, UCL Institute of Neurology, WC1N 3BG London, UK
| | - Niamh Cawley
- NMR Research Unit, UCL Institute of Neurology, WC1N 3BG London, UK
| | - Olga Ciccarelli
- NMR Research Unit, UCL Institute of Neurology, WC1N 3BG London, UK
| | | | - Sébastien Ourselin
- Translational Imaging Group, CMIC, UCL, NW1 2HE London, UK; Dementia Research Centre, UCL Institute of Neurology, WC1N 3BG London, UK
| | - Laurence Catanese
- VisAGeS: INSERM U746, CNRS UMR6074, INRIA, University of Rennes I, France
| | | | - Pierre Maurel
- VisAGeS: INSERM U746, CNRS UMR6074, INRIA, University of Rennes I, France
| | - Olivier Commowick
- VisAGeS: INSERM U746, CNRS UMR6074, INRIA, University of Rennes I, France
| | - Christian Barillot
- VisAGeS: INSERM U746, CNRS UMR6074, INRIA, University of Rennes I, France
| | - Xavier Tomas-Fernandez
- Computational Radiology Laboratory, Boston Childrens Hospital, Boston, MA 02115, USA; Harvard Medical School, Boston, MA 02115, USA
| | - Simon K Warfield
- Computational Radiology Laboratory, Boston Childrens Hospital, Boston, MA 02115, USA; Harvard Medical School, Boston, MA 02115, USA
| | - Suthirth Vaidya
- Biomedical Imaging Lab, Department of Engineering Design, Indian Institute of Technology, Chennai 600036, India
| | - Abhijith Chunduru
- Biomedical Imaging Lab, Department of Engineering Design, Indian Institute of Technology, Chennai 600036, India
| | - Ramanathan Muthuganapathy
- Biomedical Imaging Lab, Department of Engineering Design, Indian Institute of Technology, Chennai 600036, India
| | - Ganapathy Krishnamurthi
- Biomedical Imaging Lab, Department of Engineering Design, Indian Institute of Technology, Chennai 600036, India
| | - Andrew Jesson
- Centre For Intelligent Machines, McGill University, Montréal, QC H3A 0E9, Canada
| | - Tal Arbel
- Centre For Intelligent Machines, McGill University, Montréal, QC H3A 0E9, Canada
| | - Oskar Maier
- Institute of Medical Informatics, University of Lübeck, 23538 Lübeck, Germany
| | - Heinz Handels
- Institute of Medical Informatics, University of Lübeck, 23538 Lübeck, Germany
| | - Leonardo O Iheme
- Bahçeşehir University, Faculty of Engineering and Natural Sciences, 34349 Beşiktaş, Turkey
| | - Devrim Unay
- Bahçeşehir University, Faculty of Engineering and Natural Sciences, 34349 Beşiktaş, Turkey
| | | | | | | | - Mohsen Ghafoorian
- Institute for Computing and Information Sciences, Radboud University, 6525 HP Nijmegen, Netherlands
| | - Bram Platel
- Diagnostic Image Analysis Group, Radboud University Medical Center, 6525 GA Nijmegen, Netherlands
| | - Ariel Birenbaum
- Department of Electrical Engineering, Tel-Aviv University, Tel-Aviv 69978, Israel
| | - Hayit Greenspan
- Department of Biomedical Engineering, Tel-Aviv University, Tel-Aviv 69978, Israel
| | - Pierre-Louis Bazin
- Department of Neurophysics, Max Planck Institute, 04103 Leipzig, Germany
| | - Peter A Calabresi
- Department of Radiology, The Johns Hopkins School of Medicine, Baltimore, MD 21287, USA
| | | | - Lotta M Ellingsen
- Department of Electrical and Computer Engineering, The Johns Hopkins University, Baltimore, MD 21218, USA; Department of Electrical and Computer Engineering, University of Iceland, 107 Reykjavík, Iceland
| | - Daniel S Reich
- Department of Radiology, The Johns Hopkins School of Medicine, Baltimore, MD 21287, USA; Translational Neuroradiology Unit, National Institute of Neurological Disorders and Stroke, Bethesda, MD 20892, USA
| | - Jerry L Prince
- Department of Electrical and Computer Engineering, The Johns Hopkins University, Baltimore, MD 21218, USA; Department of Computer Science, The Johns Hopkins University, Baltimore, MD 21218, USA
| | - Dzung L Pham
- CNRM, The Henry M. Jackson Foundation for the Advancement of Military Medicine, Bethesda, MD 20892, USA
| |
Collapse
|
4
|
Kludas J, Arvas M, Castillo S, Pakula T, Oja M, Brouard C, Jäntti J, Penttilä M, Rousu J. Machine Learning of Protein Interactions in Fungal Secretory Pathways. PLoS One 2016; 11:e0159302. [PMID: 27441920 PMCID: PMC4956264 DOI: 10.1371/journal.pone.0159302] [Citation(s) in RCA: 6] [Impact Index Per Article: 0.8] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/15/2016] [Accepted: 06/30/2016] [Indexed: 12/18/2022] Open
Abstract
In this paper we apply machine learning methods for predicting protein interactions in fungal secretion pathways. We assume an inter-species transfer setting, where training data is obtained from a single species and the objective is to predict protein interactions in other, related species. In our methodology, we combine several state of the art machine learning approaches, namely, multiple kernel learning (MKL), pairwise kernels and kernelized structured output prediction in the supervised graph inference framework. For MKL, we apply recently proposed centered kernel alignment and p-norm path following approaches to integrate several feature sets describing the proteins, demonstrating improved performance. For graph inference, we apply input-output kernel regression (IOKR) in supervised and semi-supervised modes as well as output kernel trees (OK3). In our experiments simulating increasing genetic distance, Input-Output Kernel Regression proved to be the most robust prediction approach. We also show that the MKL approaches improve the predictions compared to uniform combination of the kernels. We evaluate the methods on the task of predicting protein-protein-interactions in the secretion pathways in fungi, S.cerevisiae, baker's yeast, being the source, T. reesei being the target of the inter-species transfer learning. We identify completely novel candidate secretion proteins conserved in filamentous fungi. These proteins could contribute to their unique secretion capabilities.
Collapse
Affiliation(s)
- Jana Kludas
- Helsinki Institute for Information Technology HIIT, Department of Computer Science, Aalto University, Espoo, Finland
| | - Mikko Arvas
- VTT Technical Research Centre of Finland, Espoo, Finland
| | | | - Tiina Pakula
- VTT Technical Research Centre of Finland, Espoo, Finland
| | - Merja Oja
- VTT Technical Research Centre of Finland, Espoo, Finland
| | - Céline Brouard
- Helsinki Institute for Information Technology HIIT, Department of Computer Science, Aalto University, Espoo, Finland
| | - Jussi Jäntti
- VTT Technical Research Centre of Finland, Espoo, Finland
| | - Merja Penttilä
- VTT Technical Research Centre of Finland, Espoo, Finland
| | - Juho Rousu
- Helsinki Institute for Information Technology HIIT, Department of Computer Science, Aalto University, Espoo, Finland
| |
Collapse
|