2
|
Maier-Hein KH, Neher PF, Houde JC, Côté MA, Garyfallidis E, Zhong J, Chamberland M, Yeh FC, Lin YC, Ji Q, Reddick WE, Glass JO, Chen DQ, Feng Y, Gao C, Wu Y, Ma J, He R, Li Q, Westin CF, Deslauriers-Gauthier S, González JOO, Paquette M, St-Jean S, Girard G, Rheault F, Sidhu J, Tax CMW, Guo F, Mesri HY, Dávid S, Froeling M, Heemskerk AM, Leemans A, Boré A, Pinsard B, Bedetti C, Desrosiers M, Brambati S, Doyon J, Sarica A, Vasta R, Cerasa A, Quattrone A, Yeatman J, Khan AR, Hodges W, Alexander S, Romascano D, Barakovic M, Auría A, Esteban O, Lemkaddem A, Thiran JP, Cetingul HE, Odry BL, Mailhe B, Nadar MS, Pizzagalli F, Prasad G, Villalon-Reina JE, Galvis J, Thompson PM, Requejo FDS, Laguna PL, Lacerda LM, Barrett R, Dell'Acqua F, Catani M, Petit L, Caruyer E, Daducci A, Dyrby TB, Holland-Letz T, Hilgetag CC, Stieltjes B, Descoteaux M. The challenge of mapping the human connectome based on diffusion tractography. Nat Commun 2017; 8:1349. [PMID: 29116093 PMCID: PMC5677006 DOI: 10.1038/s41467-017-01285-x] [Citation(s) in RCA: 700] [Impact Index Per Article: 100.0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/21/2016] [Accepted: 09/01/2017] [Indexed: 01/14/2023] Open
Abstract
Tractography based on non-invasive diffusion imaging is central to the study of human brain connectivity. To date, the approach has not been systematically validated in ground truth studies. Based on a simulated human brain data set with ground truth tracts, we organized an open international tractography challenge, which resulted in 96 distinct submissions from 20 research groups. Here, we report the encouraging finding that most state-of-the-art algorithms produce tractograms containing 90% of the ground truth bundles (to at least some extent). However, the same tractograms contain many more invalid than valid bundles, and half of these invalid bundles occur systematically across research groups. Taken together, our results demonstrate and confirm fundamental ambiguities inherent in tract reconstruction based on orientation information alone, which need to be considered when interpreting tractography and connectivity results. Our approach provides a novel framework for estimating reliability of tractography and encourages innovation to address its current limitations. Though tractography is widely used, it has not been systematically validated. Here, authors report results from 20 groups showing that many tractography algorithms produce both valid and invalid bundles.
Collapse
Affiliation(s)
- Klaus H Maier-Hein
- Division of Medical Image Computing, German Cancer Research Center (DKFZ), Heidelberg, 69120, Germany.
| | - Peter F Neher
- Division of Medical Image Computing, German Cancer Research Center (DKFZ), Heidelberg, 69120, Germany
| | - Jean-Christophe Houde
- Sherbrooke Connectivity Imaging Lab (SCIL), Université de Sherbrooke, Sherbrooke, QC J1K 0A5, QC, Canada
| | - Marc-Alexandre Côté
- Sherbrooke Connectivity Imaging Lab (SCIL), Université de Sherbrooke, Sherbrooke, QC J1K 0A5, QC, Canada
| | - Eleftherios Garyfallidis
- Sherbrooke Connectivity Imaging Lab (SCIL), Université de Sherbrooke, Sherbrooke, QC J1K 0A5, QC, Canada.,Department of Intelligent Systems Engineering, School of Informatics and Computing, Indiana University, Bloomington, IN 47408, USA
| | - Jidan Zhong
- Krembil Research Institute, University Health Network, Toronto, Canada, M5G 2C4
| | - Maxime Chamberland
- Sherbrooke Connectivity Imaging Lab (SCIL), Université de Sherbrooke, Sherbrooke, QC J1K 0A5, QC, Canada
| | - Fang-Cheng Yeh
- Department of Neurological Surgery, University of Pittsburgh School of Medicine, Pittsburgh, PA 15213, USA
| | - Ying-Chia Lin
- IMT-Institute for Advanced Studies, Lucca, 55100, Italy
| | - Qing Ji
- Department of Diagnostic Imaging, St. Jude Children's Research Hospital, Memphis, TN 38105, USA
| | - Wilburn E Reddick
- Department of Diagnostic Imaging, St. Jude Children's Research Hospital, Memphis, TN 38105, USA
| | - John O Glass
- Department of Diagnostic Imaging, St. Jude Children's Research Hospital, Memphis, TN 38105, USA
| | - David Qixiang Chen
- University of Toronto Institute of Medical Science, Toronto, Canada, M5S 1A8
| | - Yuanjing Feng
- Institute of Information Processing and Automation, Zhejiang University of Technology, Hangzhou, 310023, Zhejiang, China
| | - Chengfeng Gao
- Institute of Information Processing and Automation, Zhejiang University of Technology, Hangzhou, 310023, Zhejiang, China
| | - Ye Wu
- Institute of Information Processing and Automation, Zhejiang University of Technology, Hangzhou, 310023, Zhejiang, China
| | - Jieyan Ma
- United Imaging Healthcare Co., Shanghai, 201807, China
| | - Renjie He
- United Imaging Healthcare Co., Shanghai, 201807, China
| | - Qiang Li
- United Imaging Healthcare Co., Shanghai, 201807, China.,Shanghai Advanced Research Institute, Shanghai, 201210, China
| | - Carl-Fredrik Westin
- Laboratory of Mathematics in Imaging, Harvard Medical School, Boston, MA, 02215, USA
| | | | | | - Michael Paquette
- Sherbrooke Connectivity Imaging Lab (SCIL), Université de Sherbrooke, Sherbrooke, QC J1K 0A5, QC, Canada
| | - Samuel St-Jean
- Sherbrooke Connectivity Imaging Lab (SCIL), Université de Sherbrooke, Sherbrooke, QC J1K 0A5, QC, Canada
| | - Gabriel Girard
- Sherbrooke Connectivity Imaging Lab (SCIL), Université de Sherbrooke, Sherbrooke, QC J1K 0A5, QC, Canada
| | - François Rheault
- Sherbrooke Connectivity Imaging Lab (SCIL), Université de Sherbrooke, Sherbrooke, QC J1K 0A5, QC, Canada
| | - Jasmeen Sidhu
- Sherbrooke Connectivity Imaging Lab (SCIL), Université de Sherbrooke, Sherbrooke, QC J1K 0A5, QC, Canada
| | - Chantal M W Tax
- PROVIDI Lab, Image Sciences Institute, University Medical Center Utrecht, Utrecht, 3508, The Netherlands.,Cardiff University Brain Research Imaging Centre, School of Psychology, Cardiff University, Maindy Road, Cardiff, CF24 4HQ, UK
| | - Fenghua Guo
- PROVIDI Lab, Image Sciences Institute, University Medical Center Utrecht, Utrecht, 3508, The Netherlands
| | - Hamed Y Mesri
- PROVIDI Lab, Image Sciences Institute, University Medical Center Utrecht, Utrecht, 3508, The Netherlands
| | - Szabolcs Dávid
- PROVIDI Lab, Image Sciences Institute, University Medical Center Utrecht, Utrecht, 3508, The Netherlands
| | - Martijn Froeling
- Department of Radiology, University Medical Center Utrecht, Utrecht, 3508, The Netherlands
| | - Anneriet M Heemskerk
- PROVIDI Lab, Image Sciences Institute, University Medical Center Utrecht, Utrecht, 3508, The Netherlands
| | - Alexander Leemans
- PROVIDI Lab, Image Sciences Institute, University Medical Center Utrecht, Utrecht, 3508, The Netherlands
| | - Arnaud Boré
- Centre de recherche institut universitaire de geriatrie de Montreal (CRIUGM), Université de Montréal, Montreal, QC, Canada, H3W 1W5
| | - Basile Pinsard
- Centre de recherche institut universitaire de geriatrie de Montreal (CRIUGM), Université de Montréal, Montreal, QC, Canada, H3W 1W5.,Sorbonne Universités, UPMC Univ Paris 06, CNRS, INSERM, Laboratoire d'Imagerie Biomédicale (LIB), 75013, Paris, France
| | - Christophe Bedetti
- Centre de recherche institut universitaire de geriatrie de Montreal (CRIUGM), Université de Montréal, Montreal, QC, Canada, H3W 1W5.,Center for Advanced Research in Sleep Medicine, Hôpital du Sacré-Coeur de Montréal, Montreal, Canada, H4J 1C5
| | - Matthieu Desrosiers
- Centre de recherche institut universitaire de geriatrie de Montreal (CRIUGM), Université de Montréal, Montreal, QC, Canada, H3W 1W5
| | - Simona Brambati
- Centre de recherche institut universitaire de geriatrie de Montreal (CRIUGM), Université de Montréal, Montreal, QC, Canada, H3W 1W5
| | - Julien Doyon
- Centre de recherche institut universitaire de geriatrie de Montreal (CRIUGM), Université de Montréal, Montreal, QC, Canada, H3W 1W5
| | - Alessia Sarica
- Neuroimaging Unit, Institute of Bioimaging and Molecular Physiology (IBFM), National Research Council (CNR), Policlinico Magna Graecia, Germaneto, 88100, CZ, Italy
| | - Roberta Vasta
- Neuroimaging Unit, Institute of Bioimaging and Molecular Physiology (IBFM), National Research Council (CNR), Policlinico Magna Graecia, Germaneto, 88100, CZ, Italy
| | - Antonio Cerasa
- Neuroimaging Unit, Institute of Bioimaging and Molecular Physiology (IBFM), National Research Council (CNR), Policlinico Magna Graecia, Germaneto, 88100, CZ, Italy
| | - Aldo Quattrone
- Neuroimaging Unit, Institute of Bioimaging and Molecular Physiology (IBFM), National Research Council (CNR), Policlinico Magna Graecia, Germaneto, 88100, CZ, Italy.,Institute of Neurology, University Magna Graecia, Germaneto, 88100, CZ, Italy
| | - Jason Yeatman
- Institute for Learning & Brain Sciences and Department of Speech & Hearing Sciences, University of Washington, Seattle, WA, 98195, USA
| | - Ali R Khan
- Departments of Medical Biophysics & Medical Imaging, Schulich School of Medicine and Dentistry, Western University, 1151 Richmond St N, London, ON, Canada, N6A 5C1
| | - Wes Hodges
- Synaptive Medical Inc., MaRS Discovery District, 101 College Street, Suite 200, Toronto, ON, Canada, M5V 3B1
| | - Simon Alexander
- Synaptive Medical Inc., MaRS Discovery District, 101 College Street, Suite 200, Toronto, ON, Canada, M5V 3B1
| | - David Romascano
- Signal Processing Lab (LTS5), Ecole Polytechnique Federale de Lausanne, Lausanne, 1015, Switzerland
| | - Muhamed Barakovic
- Signal Processing Lab (LTS5), Ecole Polytechnique Federale de Lausanne, Lausanne, 1015, Switzerland
| | - Anna Auría
- Signal Processing Lab (LTS5), Ecole Polytechnique Federale de Lausanne, Lausanne, 1015, Switzerland
| | - Oscar Esteban
- Biomedical Image Technologies (BIT), ETSI Telecom., U. Politécnica de Madrid and CIBER-BBN, Madrid, 28040, Spain
| | - Alia Lemkaddem
- Signal Processing Lab (LTS5), Ecole Polytechnique Federale de Lausanne, Lausanne, 1015, Switzerland
| | - Jean-Philippe Thiran
- Signal Processing Lab (LTS5), Ecole Polytechnique Federale de Lausanne, Lausanne, 1015, Switzerland.,Department of Radiology, University Hospital Center (CHUV) and University of Lausanne (UNIL), Lausanne, 1011, Switzerland
| | - H Ertan Cetingul
- Medical Imaging Technologies, Siemens Healthcare, Princeton, NJ, 08540, USA
| | - Benjamin L Odry
- Medical Imaging Technologies, Siemens Healthcare, Princeton, NJ, 08540, USA
| | - Boris Mailhe
- Medical Imaging Technologies, Siemens Healthcare, Princeton, NJ, 08540, USA
| | - Mariappan S Nadar
- Medical Imaging Technologies, Siemens Healthcare, Princeton, NJ, 08540, USA
| | - Fabrizio Pizzagalli
- Imaging Genetics Center, Stevens Neuro Imaging and Informatics Institute, Keck School of Medicine of USC, Marina del Rey, CA, 90033, USA
| | - Gautam Prasad
- Imaging Genetics Center, Stevens Neuro Imaging and Informatics Institute, Keck School of Medicine of USC, Marina del Rey, CA, 90033, USA
| | - Julio E Villalon-Reina
- Imaging Genetics Center, Stevens Neuro Imaging and Informatics Institute, Keck School of Medicine of USC, Marina del Rey, CA, 90033, USA
| | - Justin Galvis
- Imaging Genetics Center, Stevens Neuro Imaging and Informatics Institute, Keck School of Medicine of USC, Marina del Rey, CA, 90033, USA
| | - Paul M Thompson
- Imaging Genetics Center, Stevens Neuro Imaging and Informatics Institute, Keck School of Medicine of USC, Marina del Rey, CA, 90033, USA
| | | | - Pedro Luque Laguna
- NatBrainLab, Institute of Psychiatry, Psychology & Neuroscience, King's College London, London, SE5 8AF, UK
| | - Luis Miguel Lacerda
- NatBrainLab, Institute of Psychiatry, Psychology & Neuroscience, King's College London, London, SE5 8AF, UK
| | - Rachel Barrett
- NatBrainLab, Institute of Psychiatry, Psychology & Neuroscience, King's College London, London, SE5 8AF, UK
| | - Flavio Dell'Acqua
- NatBrainLab, Institute of Psychiatry, Psychology & Neuroscience, King's College London, London, SE5 8AF, UK
| | - Marco Catani
- NatBrainLab, Institute of Psychiatry, Psychology & Neuroscience, King's College London, London, SE5 8AF, UK
| | - Laurent Petit
- Groupe d'imagerie Neurofonctionnelle-Institut des Maladies Neurodégénératives (GIN-IMN), UMR5293 CNRS, CEA, University of Bordeaux, Bordeaux, 33000, France
| | - Emmanuel Caruyer
- Centre national de la recherche scientifique (CNRS), Institute for Research in IT and Random Systems (IRISA), UMR 6074 VISAGES Project-Team, Rennes, 35042, France
| | - Alessandro Daducci
- Signal Processing Lab (LTS5), Ecole Polytechnique Federale de Lausanne, Lausanne, 1015, Switzerland.,Department of Radiology, University Hospital Center (CHUV) and University of Lausanne (UNIL), Lausanne, 1011, Switzerland
| | - Tim B Dyrby
- Danish Research Centre for Magnetic Resonance, Center for Functional and Diagnostic Imaging and Research, Copenhagen University Hospital Hvidovre, Hvidovre, 2650, Denmark.,Department of Applied Mathematics and Computer Science, Technical University of Denmark, Kongens Lyngby, 2800, Denmark
| | - Tim Holland-Letz
- Division of Biostatistics, German Cancer Research Center (DKFZ), Heidelberg, 69120, Germany
| | - Claus C Hilgetag
- Department of Computational Neuroscience, University Medical Center Eppendorf, Hamburg, 20246, Germany
| | - Bram Stieltjes
- University Hospital Basel, Radiology & Nuclear Medicine Clinic, Basel, 4031, Switzerland
| | - Maxime Descoteaux
- Sherbrooke Connectivity Imaging Lab (SCIL), Université de Sherbrooke, Sherbrooke, QC J1K 0A5, QC, Canada.
| |
Collapse
|