1
|
Zehtabian A, Fuchs J, Eickholt BJ, Ewers H. Automated Analysis of Neuronal Morphology in 2D Fluorescence Micrographs through an Unsupervised Semantic Segmentation of Neurons. Neuroscience 2024; 551:333-344. [PMID: 38838980 DOI: 10.1016/j.neuroscience.2024.05.024] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/19/2023] [Revised: 05/17/2024] [Accepted: 05/20/2024] [Indexed: 06/07/2024]
Abstract
Brain function emerges from a highly complex network of specialized cells that are interlinked by billions of synapses. The synaptic connectivity between neurons is established between the elongated processes of their axons and dendrites or, together, neurites. To establish these connections, cellular neurites have to grow in highly specialized, cell-type dependent patterns covering extensive distances and connecting with thousands of other neurons. The outgrowth and branching of neurites are tightly controlled during development and are a commonly used functional readout of imaging in the neurosciences. Manual analysis of neuronal morphology from microscopy images, however, is very time intensive and prone to bias. Most automated analyses of neurons rely on reconstruction of the neuron as a whole without a semantic analysis of each neurite. A fully-automated classification of all neurites still remains unavailable in open-source software. Here we present a standalone, GUI-based software for batch-quantification of neuronal morphology in two-dimensional fluorescence micrographs of cultured neurons with minimal requirements for user interaction. Single neurons are first reconstructed into binarized images using a Hessian-based segmentation algorithm to detect thin neurite structures combined with intensity- and shape-based reconstruction of the cell body. Neurites are then classified into axon, dendrites and their branches of increasing order using a geodesic distance transform of the cell skeleton. The software was benchmarked against a published dataset and reproduced the phenotype observed after manual annotation. Our tool promises accelerated and improved morphometric studies of neuronal morphology by allowing for consistent and automated analysis of large datasets.
Collapse
Affiliation(s)
- Amin Zehtabian
- Institute for Chemistry and Biochemistry, Freie Universität Berlin, Thielallee 63, 14195 Berlin, Germany.
| | - Joachim Fuchs
- Charité - Universitätsmedizin Berlin, Corporate Member of Freie Universität Berlin, Humboldt-Universität zu Berlin, and Berlin Institute of Health, Institute of Molecular Biology and Biochemistry, Virchowweg 6, 10117 Berlin, Germany
| | - Britta J Eickholt
- Charité - Universitätsmedizin Berlin, Corporate Member of Freie Universität Berlin, Humboldt-Universität zu Berlin, and Berlin Institute of Health, Institute of Molecular Biology and Biochemistry, Virchowweg 6, 10117 Berlin, Germany
| | - Helge Ewers
- Institute for Chemistry and Biochemistry, Freie Universität Berlin, Thielallee 63, 14195 Berlin, Germany
| |
Collapse
|
2
|
Liu M, Wu S, Chen R, Lin Z, Wang Y, Meijering E. Brain Image Segmentation for Ultrascale Neuron Reconstruction via an Adaptive Dual-Task Learning Network. IEEE TRANSACTIONS ON MEDICAL IMAGING 2024; 43:2574-2586. [PMID: 38373129 DOI: 10.1109/tmi.2024.3367384] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 02/21/2024]
Abstract
Accurate morphological reconstruction of neurons in whole brain images is critical for brain science research. However, due to the wide range of whole brain imaging, uneven staining, and optical system fluctuations, there are significant differences in image properties between different regions of the ultrascale brain image, such as dramatically varying voxel intensities and inhomogeneous distribution of background noise, posing an enormous challenge to neuron reconstruction from whole brain images. In this paper, we propose an adaptive dual-task learning network (ADTL-Net) to quickly and accurately extract neuronal structures from ultrascale brain images. Specifically, this framework includes an External Features Classifier (EFC) and a Parameter Adaptive Segmentation Decoder (PASD), which share the same Multi-Scale Feature Encoder (MSFE). MSFE introduces an attention module named Channel Space Fusion Module (CSFM) to extract structure and intensity distribution features of neurons at different scales for addressing the problem of anisotropy in 3D space. Then, EFC is designed to classify these feature maps based on external features, such as foreground intensity distributions and image smoothness, and select specific PASD parameters to decode them of different classes to obtain accurate segmentation results. PASD contains multiple sets of parameters trained by different representative complex signal-to-noise distribution image blocks to handle various images more robustly. Experimental results prove that compared with other advanced segmentation methods for neuron reconstruction, the proposed method achieves state-of-the-art results in the task of neuron reconstruction from ultrascale brain images, with an improvement of about 49% in speed and 12% in F1 score.
Collapse
|
3
|
Nourollah AM, Hassanpour H, Zehtabian A. Quantifying morphologies of developing neuronal cells using deep learning with imperfect annotations. IBRO Neurosci Rep 2024; 16:118-126. [PMID: 38282758 PMCID: PMC10820797 DOI: 10.1016/j.ibneur.2023.12.009] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/26/2023] [Accepted: 12/29/2023] [Indexed: 01/30/2024] Open
Abstract
The functionality of human intelligence relies on the interaction and health of neurons, hence, quantifying neuronal morphologies can be crucial for investigating the functionality of the human brain. This paper proposes a deep learning (DL) based method for segmenting and quantifying neuronal structures in fluorescence microscopy images of developing neuronal cells cultured in vitro. Compared to the majority of supervised DL-based segmentation methods that heavily rely on creating exact corresponding masks of neuronal structures for the preparation of training samples, the proposed approach allows for imperfect annotation of neurons, as it only requires tracing the centrelines of the neurites. This ability accelerates the preparation of training data by several folds. Our proposed framework is built on a modified version of PSPNet with an EfficientNet backbone pre-trained on the CityScapes dataset. To handle the imperfectness of training samples, we incorporated a weighted combination of two loss functions, namely the Dice loss and Lovász loss functions, into our network. We evaluated the proposed framework and several other state-of-the-art methods on a published dataset of approximately 900 manually quantified cultured mouse neurons. Our results indicate a close correlation between the proposed method and manual quantification in terms of neuron length and the number of branches while demonstrating improved analysis speed. Furthermore, the proposed method achieved high accuracy in neuron segmentation, as evidenced by the evaluation of the neurons' length and number of branches.
Collapse
Affiliation(s)
- Amir Masoud Nourollah
- Department of Computer Engineering and Information Technology, Shahrood University of Technology, Iran
| | - Hamid Hassanpour
- Department of Computer Engineering and Information Technology, Shahrood University of Technology, Iran
| | - Amin Zehtabian
- Institute for Chemistry and Biochemistry, Freie Universität Berlin, Germany
| |
Collapse
|
4
|
Zeng Y, Wang Y. Complete Neuron Reconstruction Based on Branch Confidence. Brain Sci 2024; 14:396. [PMID: 38672045 PMCID: PMC11047972 DOI: 10.3390/brainsci14040396] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/04/2024] [Revised: 04/04/2024] [Accepted: 04/09/2024] [Indexed: 04/28/2024] Open
Abstract
In the past few years, significant advancements in microscopic imaging technology have led to the production of numerous high-resolution images capturing brain neurons at the micrometer scale. The reconstructed structure of neurons from neuronal images can serve as a valuable reference for research in brain diseases and neuroscience. Currently, there lacks an accurate and efficient method for neuron reconstruction. Manual reconstruction remains the primary approach, offering high accuracy but requiring significant time investment. While some automatic reconstruction methods are faster, they often sacrifice accuracy and cannot be directly relied upon. Therefore, the primary goal of this paper is to develop a neuron reconstruction tool that is both efficient and accurate. The tool aids users in reconstructing complete neurons by calculating the confidence of branches during the reconstruction process. The method models the neuron reconstruction as multiple Markov chains, and calculates the confidence of the connections between branches by simulating the reconstruction artifacts in the results. Users iteratively modify low-confidence branches to ensure precise and efficient neuron reconstruction. Experiments on both the publicly accessible BigNeuron dataset and a self-created Whole-Brain dataset demonstrate that the tool achieves high accuracy similar to manual reconstruction, while significantly reducing reconstruction time.
Collapse
Affiliation(s)
- Ying Zeng
- School of Computer Science and Technology, Shanghai University, Shanghai 200444, China;
- Guangdong Institute of Intelligence Science and Technology, Zhuhai 519031, China
| | - Yimin Wang
- Guangdong Institute of Intelligence Science and Technology, Zhuhai 519031, China
| |
Collapse
|
5
|
Ding L, Zhao X, Guo S, Liu Y, Liu L, Wang Y, Peng H. SNAP: a structure-based neuron morphology reconstruction automatic pruning pipeline. Front Neuroinform 2023; 17:1174049. [PMID: 37388757 PMCID: PMC10303825 DOI: 10.3389/fninf.2023.1174049] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/25/2023] [Accepted: 05/22/2023] [Indexed: 07/01/2023] Open
Abstract
Background Neuron morphology analysis is an essential component of neuron cell-type definition. Morphology reconstruction represents a bottleneck in high-throughput morphology analysis workflow, and erroneous extra reconstruction owing to noise and entanglements in dense neuron regions restricts the usability of automated reconstruction results. We propose SNAP, a structure-based neuron morphology reconstruction pruning pipeline, to improve the usability of results by reducing erroneous extra reconstruction and splitting entangled neurons. Methods For the four different types of erroneous extra segments in reconstruction (caused by noise in the background, entanglement with dendrites of close-by neurons, entanglement with axons of other neurons, and entanglement within the same neuron), SNAP incorporates specific statistical structure information into rules for erroneous extra segment detection and achieves pruning and multiple dendrite splitting. Results Experimental results show that this pipeline accomplishes pruning with satisfactory precision and recall. It also demonstrates good multiple neuron-splitting performance. As an effective tool for post-processing reconstruction, SNAP can facilitate neuron morphology analysis.
Collapse
Affiliation(s)
- Liya Ding
- Institute for Brain and Intelligence, Southeast University, Nanjing, China
| | - Xuan Zhao
- Institute for Brain and Intelligence, Southeast University, Nanjing, China
| | - Shuxia Guo
- Institute for Brain and Intelligence, Southeast University, Nanjing, China
| | - Yufeng Liu
- Institute for Brain and Intelligence, Southeast University, Nanjing, China
| | - Lijuan Liu
- Institute for Brain and Intelligence, Southeast University, Nanjing, China
| | - Yimin Wang
- Institute for Brain and Intelligence, Southeast University, Nanjing, China
- Guangdong Institute of Intelligence Science and Technology, Zhuhai, China
| | - Hanchuan Peng
- Institute for Brain and Intelligence, Southeast University, Nanjing, China
| |
Collapse
|
6
|
Zhang Y, Miller JA, Park J, Lelieveldt BP, Long B, Abdelaal T, Aevermann BD, Biancalani T, Comiter C, Dzyubachyk O, Eggermont J, Langseth CM, Petukhov V, Scalia G, Vaishnav ED, Zhao Y, Lein ES, Scheuermann RH. Reference-based cell type matching of in situ image-based spatial transcriptomics data on primary visual cortex of mouse brain. Sci Rep 2023; 13:9567. [PMID: 37311768 DOI: 10.1038/s41598-023-36638-8] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/18/2022] [Accepted: 06/07/2023] [Indexed: 06/15/2023] Open
Abstract
With the advent of multiplex fluorescence in situ hybridization (FISH) and in situ RNA sequencing technologies, spatial transcriptomics analysis is advancing rapidly, providing spatial location and gene expression information about cells in tissue sections at single cell resolution. Cell type classification of these spatially-resolved cells can be inferred by matching the spatial transcriptomics data to reference atlases derived from single cell RNA-sequencing (scRNA-seq) in which cell types are defined by differences in their gene expression profiles. However, robust cell type matching of the spatially-resolved cells to reference scRNA-seq atlases is challenging due to the intrinsic differences in resolution between the spatial and scRNA-seq data. In this study, we systematically evaluated six computational algorithms for cell type matching across four image-based spatial transcriptomics experimental protocols (MERFISH, smFISH, BaristaSeq, and ExSeq) conducted on the same mouse primary visual cortex (VISp) brain region. We find that many cells are assigned as the same type by multiple cell type matching algorithms and are present in spatial patterns previously reported from scRNA-seq studies in VISp. Furthermore, by combining the results of individual matching strategies into consensus cell type assignments, we see even greater alignment with biological expectations. We present two ensemble meta-analysis strategies used in this study and share the consensus cell type matching results in the Cytosplore Viewer ( https://viewer.cytosplore.org ) for interactive visualization and data exploration. The consensus matching can also guide spatial data analysis using SSAM, allowing segmentation-free cell type assignment.
Collapse
Affiliation(s)
- Yun Zhang
- J. Craig Venter Institute, La Jolla, CA, USA
| | | | - Jeongbin Park
- School of Biomedical Convergence Engineering, Pusan National University, Busan, Korea
| | - Boudewijn P Lelieveldt
- LKEB, Department of Radiology, Leiden University Medical Center, Leiden, The Netherlands
- Pattern Recognition and Bioinformatics Group, Delft University of Technology, Delft, The Netherlands
| | - Brian Long
- Allen Institute for Brain Science, Seattle, WA, USA
| | - Tamim Abdelaal
- LKEB, Department of Radiology, Leiden University Medical Center, Leiden, The Netherlands
- Pattern Recognition and Bioinformatics Group, Delft University of Technology, Delft, The Netherlands
| | - Brian D Aevermann
- J. Craig Venter Institute, La Jolla, CA, USA
- Chan Zuckerberg Initiative, Redwood City, CA, USA
| | - Tommaso Biancalani
- Broad Institute of MIT and Harvard, Cambridge, MA, USA
- Genentech, South San Francisco, CA, USA
| | | | - Oleh Dzyubachyk
- LKEB, Department of Radiology, Leiden University Medical Center, Leiden, The Netherlands
| | - Jeroen Eggermont
- LKEB, Department of Radiology, Leiden University Medical Center, Leiden, The Netherlands
| | | | - Viktor Petukhov
- Biotech Research and Innovation Centre, Faculty of Health and Medical Sciences, University of Copenhagen, Copenhagen, Denmark
- Department of Biomedical Informatics, Harvard Medical School, Boston, MA, USA
| | - Gabriele Scalia
- Broad Institute of MIT and Harvard, Cambridge, MA, USA
- Genentech, South San Francisco, CA, USA
| | | | - Yilin Zhao
- Allen Institute for Brain Science, Seattle, WA, USA
| | - Ed S Lein
- Allen Institute for Brain Science, Seattle, WA, USA
| | - Richard H Scheuermann
- J. Craig Venter Institute, La Jolla, CA, USA.
- Department of Pathology, University of California, San Diego, CA, USA.
- Division of Vaccine Discovery, La Jolla Institute for Immunology, La Jolla, CA, USA.
| |
Collapse
|
7
|
Manubens-Gil L, Zhou Z, Chen H, Ramanathan A, Liu X, Liu Y, Bria A, Gillette T, Ruan Z, Yang J, Radojević M, Zhao T, Cheng L, Qu L, Liu S, Bouchard KE, Gu L, Cai W, Ji S, Roysam B, Wang CW, Yu H, Sironi A, Iascone DM, Zhou J, Bas E, Conde-Sousa E, Aguiar P, Li X, Li Y, Nanda S, Wang Y, Muresan L, Fua P, Ye B, He HY, Staiger JF, Peter M, Cox DN, Simonneau M, Oberlaender M, Jefferis G, Ito K, Gonzalez-Bellido P, Kim J, Rubel E, Cline HT, Zeng H, Nern A, Chiang AS, Yao J, Roskams J, Livesey R, Stevens J, Liu T, Dang C, Guo Y, Zhong N, Tourassi G, Hill S, Hawrylycz M, Koch C, Meijering E, Ascoli GA, Peng H. BigNeuron: a resource to benchmark and predict performance of algorithms for automated tracing of neurons in light microscopy datasets. Nat Methods 2023; 20:824-835. [PMID: 37069271 DOI: 10.1038/s41592-023-01848-5] [Citation(s) in RCA: 7] [Impact Index Per Article: 7.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/10/2022] [Accepted: 03/14/2023] [Indexed: 04/19/2023]
Abstract
BigNeuron is an open community bench-testing platform with the goal of setting open standards for accurate and fast automatic neuron tracing. We gathered a diverse set of image volumes across several species that is representative of the data obtained in many neuroscience laboratories interested in neuron tracing. Here, we report generated gold standard manual annotations for a subset of the available imaging datasets and quantified tracing quality for 35 automatic tracing algorithms. The goal of generating such a hand-curated diverse dataset is to advance the development of tracing algorithms and enable generalizable benchmarking. Together with image quality features, we pooled the data in an interactive web application that enables users and developers to perform principal component analysis, t-distributed stochastic neighbor embedding, correlation and clustering, visualization of imaging and tracing data, and benchmarking of automatic tracing algorithms in user-defined data subsets. The image quality metrics explain most of the variance in the data, followed by neuromorphological features related to neuron size. We observed that diverse algorithms can provide complementary information to obtain accurate results and developed a method to iteratively combine methods and generate consensus reconstructions. The consensus trees obtained provide estimates of the neuron structure ground truth that typically outperform single algorithms in noisy datasets. However, specific algorithms may outperform the consensus tree strategy in specific imaging conditions. Finally, to aid users in predicting the most accurate automatic tracing results without manual annotations for comparison, we used support vector machine regression to predict reconstruction quality given an image volume and a set of automatic tracings.
Collapse
Affiliation(s)
- Linus Manubens-Gil
- Institute for Brain and Intelligence, Southeast University, Nanjing, China
| | - Zhi Zhou
- Microsoft Corporation, Redmond, WA, USA
| | | | - Arvind Ramanathan
- Computing, Environment and Life Sciences Directorate, Argonne National Laboratory, Lemont, IL, USA
| | | | - Yufeng Liu
- Institute for Brain and Intelligence, Southeast University, Nanjing, China
| | | | - Todd Gillette
- Center for Neural Informatics, Structures and Plasticity, Krasnow Institute for Advanced Study, George Mason University, Fairfax, VA, USA
| | - Zongcai Ruan
- Institute for Brain and Intelligence, Southeast University, Nanjing, China
| | - Jian Yang
- Faculty of Information Technology, Beijing University of Technology, Beijing, China
- Beijing International Collaboration Base on Brain Informatics and Wisdom Services, Beijing, China
| | | | - Ting Zhao
- Janelia Research Campus, Howard Hughes Medical Institute, Ashburn, VA, USA
| | - Li Cheng
- Department of Electrical and Computer Engineering, University of Alberta, Edmonton, Alberta, Canada
| | - Lei Qu
- Institute for Brain and Intelligence, Southeast University, Nanjing, China
- Ministry of Education Key Laboratory of Intelligent Computation and Signal Processing, Anhui University, Hefei, China
| | | | - Kristofer E Bouchard
- Scientific Data Division and Biological Systems and Engineering Division, Lawrence Berkeley National Lab, Berkeley, CA, USA
- Helen Wills Neuroscience Institute and Redwood Center for Theoretical Neuroscience, UC Berkeley, Berkeley, CA, USA
| | - Lin Gu
- RIKEN AIP, Tokyo, Japan
- Research Center for Advanced Science and Technology (RCAST), The University of Tokyo, Tokyo, Japan
| | - Weidong Cai
- School of Computer Science, University of Sydney, Sydney, New South Wales, Australia
| | - Shuiwang Ji
- Texas A&M University, College Station, TX, USA
| | - Badrinath Roysam
- Cullen College of Engineering, University of Houston, Houston, TX, USA
| | - Ching-Wei Wang
- Graduate Institute of Biomedical Engineering, National Taiwan University of Science and Technology, Taipei, Taiwan
| | - Hongchuan Yu
- National Centre for Computer Animation, Bournemouth University, Poole, UK
| | | | - Daniel Maxim Iascone
- Department of Neuroscience, Columbia University, New York, NY, USA
- Mortimer B. Zuckerman Mind Brain Behavior Institute, Columbia University, New York, NY, USA
| | - Jie Zhou
- Department of Computer Science, Northern Illinois University, DeKalb, IL, USA
| | | | - Eduardo Conde-Sousa
- i3S, Instituto de Investigação E Inovação Em Saúde, Universidade Do Porto, Porto, Portugal
- INEB, Instituto de Engenharia Biomédica, Universidade Do Porto, Porto, Portugal
| | - Paulo Aguiar
- i3S, Instituto de Investigação E Inovação Em Saúde, Universidade Do Porto, Porto, Portugal
| | - Xiang Li
- Massachusetts General Hospital and Harvard Medical School, Boston, MA, USA
| | - Yujie Li
- Allen Institute for Brain Science, Seattle, WA, USA
- Cortical Architecture Imaging and Discovery Lab, Department of Computer Science and Bioimaging Research Center, The University of Georgia, Athens, GA, USA
| | - Sumit Nanda
- Center for Neural Informatics, Structures and Plasticity, Krasnow Institute for Advanced Study, George Mason University, Fairfax, VA, USA
| | - Yuan Wang
- Program in Neuroscience, Department of Biomedical Sciences, Florida State University College of Medicine, Tallahassee, FL, USA
| | - Leila Muresan
- Cambridge Advanced Imaging Centre, University of Cambridge, Cambridge, UK
| | - Pascal Fua
- Computer Vision Laboratory, EPFL, Lausanne, Switzerland
| | - Bing Ye
- Life Sciences Institute and Department of Cell and Developmental Biology, University of Michigan, Ann Arbor, MI, USA
| | - Hai-Yan He
- Department of Biology, Georgetown University, Washington, DC, USA
| | - Jochen F Staiger
- Institute for Neuroanatomy, University Medical Center Göttingen, Georg-August- University Göttingen, Goettingen, Germany
| | - Manuel Peter
- Department of Stem Cell and Regenerative Biology and Center for Brain Science, Harvard University, Cambridge, MA, USA
| | - Daniel N Cox
- Neuroscience Institute, Georgia State University, Atlanta, GA, USA
| | - Michel Simonneau
- 42 ENS Paris-Saclay, CNRS, CentraleSupélec, LuMIn, Université Paris-Saclay, Gif-sur-Yvette, France
| | - Marcel Oberlaender
- Max Planck Group: In Silico Brain Sciences, Max Planck Institute for Neurobiology of Behavior - caesar, Bonn, Germany
| | - Gregory Jefferis
- Janelia Research Campus, Howard Hughes Medical Institute, Ashburn, VA, USA
- Division of Neurobiology, MRC Laboratory of Molecular Biology, Cambridge, UK
- Department of Zoology, University of Cambridge, Cambridge, UK
| | - Kei Ito
- Janelia Research Campus, Howard Hughes Medical Institute, Ashburn, VA, USA
- Institute for Quantitative Biosciences, University of Tokyo, Tokyo, Japan
- Institute of Zoology, Biocenter Cologne, University of Cologne, Cologne, Germany
| | | | - Jinhyun Kim
- Brain Science Institute, Korea Institute of Science and Technology (KIST), Seoul, South Korea
| | - Edwin Rubel
- Virginia Merrill Bloedel Hearing Research Center, University of Washington, Seattle, WA, USA
| | | | - Hongkui Zeng
- Allen Institute for Brain Science, Seattle, WA, USA
| | - Aljoscha Nern
- Janelia Research Campus, Howard Hughes Medical Institute, Ashburn, VA, USA
| | - Ann-Shyn Chiang
- Brain Research Center, National Tsing Hua University, Hsinchu, Taiwan
| | | | - Jane Roskams
- Allen Institute for Brain Science, Seattle, WA, USA
- Department of Zoology, Life Sciences Institute, University of British Columbia, Vancouver, British Columbia, Canada
| | - Rick Livesey
- Zayed Centre for Rare Disease Research, UCL Great Ormond Street Institute of Child Health, London, UK
| | - Janine Stevens
- Janelia Research Campus, Howard Hughes Medical Institute, Ashburn, VA, USA
| | - Tianming Liu
- Cortical Architecture Imaging and Discovery Lab, Department of Computer Science and Bioimaging Research Center, The University of Georgia, Athens, GA, USA
| | - Chinh Dang
- Virginia Merrill Bloedel Hearing Research Center, University of Washington, Seattle, WA, USA
| | - Yike Guo
- Data Science Institute, Imperial College London, London, UK
| | - Ning Zhong
- Faculty of Information Technology, Beijing University of Technology, Beijing, China
- Beijing International Collaboration Base on Brain Informatics and Wisdom Services, Beijing, China
- Department of Life Science and Informatics, Maebashi Institute of Technology, Maebashi, Japan
| | | | - Sean Hill
- Campbell Family Mental Health Research Institute, Centre for Addiction and Mental Health, Toronto, Ontario, Canada
- Institute of Medical Science, University of Toronto, Toronto, Ontario, Canada
- Krembil Centre for Neuroinformatics, Centre for Addiction and Mental Health, Toronto, Ontario, Canada
- Department of Psychiatry, University of Toronto, Toronto, Ontario, Canada
| | | | | | - Erik Meijering
- School of Computer Science and Engineering, University of New South Wales, Sydney, New South Wales, Australia.
| | - Giorgio A Ascoli
- Center for Neural Informatics, Structures and Plasticity, Krasnow Institute for Advanced Study, George Mason University, Fairfax, VA, USA.
| | - Hanchuan Peng
- Institute for Brain and Intelligence, Southeast University, Nanjing, China.
| |
Collapse
|
8
|
Liu Y, Zhong Y, Zhao X, Liu L, Ding L, Peng H. Tracing weak neuron fibers. Bioinformatics 2022; 39:6960919. [PMID: 36571479 PMCID: PMC9848051 DOI: 10.1093/bioinformatics/btac816] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/31/2022] [Revised: 11/01/2022] [Accepted: 12/23/2022] [Indexed: 12/27/2022] Open
Abstract
MOTIVATION Precise reconstruction of neuronal arbors is important for circuitry mapping. Many auto-tracing algorithms have been developed toward full reconstruction. However, it is still challenging to trace the weak signals of neurite fibers that often correspond to axons. RESULTS We proposed a method, named the NeuMiner, for tracing weak fibers by combining two strategies: an online sample mining strategy and a modified gamma transformation. NeuMiner improved the recall of weak signals (voxel values <20) by a large margin, from 5.1 to 27.8%. This is prominent for axons, which increased by 6.4 times, compared to 2.0 times for dendrites. Both strategies were shown to be beneficial for weak fiber recognition, and they reduced the average axonal spatial distances to gold standards by 46 and 13%, respectively. The improvement was observed on two prevalent automatic tracing algorithms and can be applied to any other tracers and image types. AVAILABILITY AND IMPLEMENTATION Source codes of NeuMiner are freely available on GitHub (https://github.com/crazylyf/neuronet/tree/semantic_fnm). Image visualization, preprocessing and tracing are conducted on the Vaa3D platform, which is accessible at the Vaa3D GitHub repository (https://github.com/Vaa3D). All training and testing images are cropped from high-resolution fMOST mouse brains downloaded from the Brain Image Library (https://www.brainimagelibrary.org/), and the corresponding gold standards are available at https://doi.brainimagelibrary.org/doi/10.35077/g.25. SUPPLEMENTARY INFORMATION Supplementary data are available at Bioinformatics online.
Collapse
Affiliation(s)
- Yufeng Liu
- SEU-ALLEN Joint Center, Institute for Brain and Intelligence, Southeast University, Nanjing, Jiangsu 210096, China
| | - Ye Zhong
- SEU-ALLEN Joint Center, Institute for Brain and Intelligence, Southeast University, Nanjing, Jiangsu 210096, China
| | - Xuan Zhao
- SEU-ALLEN Joint Center, Institute for Brain and Intelligence, Southeast University, Nanjing, Jiangsu 210096, China
| | - Lijuan Liu
- SEU-ALLEN Joint Center, Institute for Brain and Intelligence, Southeast University, Nanjing, Jiangsu 210096, China
| | - Liya Ding
- SEU-ALLEN Joint Center, Institute for Brain and Intelligence, Southeast University, Nanjing, Jiangsu 210096, China
| | | |
Collapse
|
9
|
Liu Y, Wang G, Ascoli GA, Zhou J, Liu L. Neuron tracing from light microscopy images: automation, deep learning and bench testing. Bioinformatics 2022; 38:5329-5339. [PMID: 36303315 PMCID: PMC9750132 DOI: 10.1093/bioinformatics/btac712] [Citation(s) in RCA: 13] [Impact Index Per Article: 6.5] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/28/2022] [Revised: 10/19/2022] [Accepted: 10/26/2022] [Indexed: 12/24/2022] Open
Abstract
MOTIVATION Large-scale neuronal morphologies are essential to neuronal typing, connectivity characterization and brain modeling. It is widely accepted that automation is critical to the production of neuronal morphology. Despite previous survey papers about neuron tracing from light microscopy data in the last decade, thanks to the rapid development of the field, there is a need to update recent progress in a review focusing on new methods and remarkable applications. RESULTS This review outlines neuron tracing in various scenarios with the goal to help the community understand and navigate tools and resources. We describe the status, examples and accessibility of automatic neuron tracing. We survey recent advances of the increasingly popular deep-learning enhanced methods. We highlight the semi-automatic methods for single neuron tracing of mammalian whole brains as well as the resulting datasets, each containing thousands of full neuron morphologies. Finally, we exemplify the commonly used datasets and metrics for neuron tracing bench testing.
Collapse
Affiliation(s)
- Yufeng Liu
- School of Biological Science and Medical Engineering, Southeast University, Nanjing, China
| | - Gaoyu Wang
- School of Computer Science and Engineering, Southeast University, Nanjing, China
| | - Giorgio A Ascoli
- Center for Neural Informatics, Structures, & Plasticity, Krasnow Institute for Advanced Study, George Mason University, Fairfax, VA, USA
| | - Jiangning Zhou
- Institute of Brain Science, The First Affiliated Hospital of Anhui Medical University, Hefei, China
| | - Lijuan Liu
- School of Biological Science and Medical Engineering, Southeast University, Nanjing, China
| |
Collapse
|
10
|
Wang X, Liu M, Wang Y, Fan J, Meijering E. A 3D Tubular Flux Model for Centerline Extraction in Neuron Volumetric Images. IEEE TRANSACTIONS ON MEDICAL IMAGING 2022; 41:1069-1079. [PMID: 34826295 DOI: 10.1109/tmi.2021.3130987] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/13/2023]
Abstract
Digital morphology reconstruction from neuron volumetric images is essential for computational neuroscience. The centerline of the axonal and dendritic tree provides an effective shape representation and serves as a basis for further neuron reconstruction. However, it is still a challenge to directly extract the accurate centerline from the complex neuron structure with poor image quality. In this paper, we propose a neuron centerline extraction method based on a 3D tubular flux model via a two-stage CNN framework. In the first stage, a 3D CNN is used to learn the latent neuron structure features, namely flux features, from neuron images. In the second stage, a light-weight U-Net takes the learned flux features as input to extract the centerline with a spatial weighted average strategy to constrain the multi-voxel width response. Specifically, the labels of flux features in the first stage are generated by the 3D tubular model which calculates the geometric representations of the flux between each voxel in the tubular region and the nearest point on the centerline ground truth. Compared with self-learned features by networks, flux features, as a kind of prior knowledge, explicitly take advantage of the contextual distance and direction distribution information around the centerline, which is beneficial for the precise centerline extraction. Experiments on two challenging datasets demonstrate that the proposed method outperforms other state-of-the-art methods by 18% and 35.1% in F1-measurement and average distance scores at the most, and the extracted centerline is helpful to improve the neuron reconstruction performance.
Collapse
|
11
|
Hidden Markov modeling for maximum probability neuron reconstruction. Commun Biol 2022; 5:388. [PMID: 35468989 PMCID: PMC9038756 DOI: 10.1038/s42003-022-03320-0] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/27/2021] [Accepted: 03/24/2022] [Indexed: 11/08/2022] Open
Abstract
Recent advances in brain clearing and imaging have made it possible to image entire mammalian brains at sub-micron resolution. These images offer the potential to assemble brain-wide atlases of neuron morphology, but manual neuron reconstruction remains a bottleneck. Several automatic reconstruction algorithms exist, but most focus on single neuron images. In this paper, we present a probabilistic reconstruction method, ViterBrain, which combines a hidden Markov state process that encodes neuron geometry with a random field appearance model of neuron fluorescence. ViterBrain utilizes dynamic programming to compute the global maximizer of what we call the most probable neuron path. We applied our algorithm to imperfect image segmentations, and showed that it can follow axons in the presence of noise or nearby neurons. We also provide an interactive framework where users can trace neurons by fixing start and endpoints. ViterBrain is available in our open-source Python package brainlit. ViterBrain is an automated probabilistic reconstruction method that can reconstruct neuronal geometry and processes from microscopy images with code available in the open-source Python package, brainlit.
Collapse
|
12
|
Yang B, Liu M, Wang Y, Zhang K, Meijering E. Structure-Guided Segmentation for 3D Neuron Reconstruction. IEEE TRANSACTIONS ON MEDICAL IMAGING 2022; 41:903-914. [PMID: 34748483 DOI: 10.1109/tmi.2021.3125777] [Citation(s) in RCA: 10] [Impact Index Per Article: 5.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/13/2023]
Abstract
Digital reconstruction of neuronal morphologies in 3D microscopy images is critical in the field of neuroscience. However, most existing automatic tracing algorithms cannot obtain accurate neuron reconstruction when processing 3D neuron images contaminated by strong background noises or containing weak filament signals. In this paper, we present a 3D neuron segmentation network named Structure-Guided Segmentation Network (SGSNet) to enhance weak neuronal structures and remove background noises. The network contains a shared encoding path but utilizes two decoding paths called Main Segmentation Branch (MSB) and Structure-Detection Branch (SDB), respectively. MSB is trained on binary labels to acquire the 3D neuron image segmentation maps. However, the segmentation results in challenging datasets often contain structural errors, such as discontinued segments of the weak-signal neuronal structures and missing filaments due to low signal-to-noise ratio (SNR). Therefore, SDB is presented to detect the neuronal structures by regressing neuron distance transform maps. Furthermore, a Structure Attention Module (SAM) is designed to integrate the multi-scale feature maps of the two decoding paths, and provide contextual guidance of structural features from SDB to MSB to improve the final segmentation performance. In the experiments, we evaluate our model in two challenging 3D neuron image datasets, the BigNeuron dataset and the Extended Whole Mouse Brain Sub-image (EWMBS) dataset. When using different tracing methods on the segmented images produced by our method rather than other state-of-the-art segmentation methods, the distance scores gain 42.48% and 35.83% improvement in the BigNeuron dataset and 37.75% and 23.13% in the EWMBS dataset.
Collapse
|
13
|
Chen X, Zhang C, Zhao J, Xiong Z, Zha ZJ, Wu F. Weakly Supervised Neuron Reconstruction From Optical Microscopy Images With Morphological Priors. IEEE TRANSACTIONS ON MEDICAL IMAGING 2021; 40:3205-3216. [PMID: 33999814 DOI: 10.1109/tmi.2021.3080695] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/12/2023]
Abstract
Manually labeling neurons from high-resolution but noisy and low-contrast optical microscopy (OM) images is tedious. As a result, the lack of annotated data poses a key challenge when applying deep learning techniques for reconstructing neurons from noisy and low-contrast OM images. While traditional tracing methods provide a possible way to efficiently generate labels for supervised network training, the generated pseudo-labels contain many noisy and incorrect labels, which lead to severe performance degradation. On the other hand, the publicly available dataset, BigNeuron, provides a large number of single 3D neurons that are reconstructed using various imaging paradigms and tracing methods. Though the raw OM images are not fully available for these neurons, they convey essential morphological priors for complex 3D neuron structures. In this paper, we propose a new approach to exploit morphological priors from neurons that have been reconstructed for training a deep neural network to extract neuron signals from OM images. We integrate a deep segmentation network in a generative adversarial network (GAN), expecting the segmentation network to be weakly supervised by pseudo-labels at the pixel level while utilizing the supervision of previously reconstructed neurons at the morphology level. In our morphological-prior-guided neuron reconstruction GAN, named MP-NRGAN, the segmentation network extracts neuron signals from raw images, and the discriminator network encourages the extracted neurons to follow the morphology distribution of reconstructed neurons. Comprehensive experiments on the public VISoR-40 dataset and BigNeuron dataset demonstrate that our proposed MP-NRGAN outperforms state-of-the-art approaches with less training effort.
Collapse
|
14
|
He Y, Huang J, Wu G, Yang J. Exploring highly reliable substructures in auto-reconstructions of a neuron. Brain Inform 2021; 8:17. [PMID: 34431008 PMCID: PMC8384950 DOI: 10.1186/s40708-021-00137-1] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/20/2021] [Accepted: 07/27/2021] [Indexed: 11/10/2022] Open
Abstract
The digital reconstruction of a neuron is the most direct and effective way to investigate its morphology. Many automatic neuron tracing methods have been proposed, but without manual check it is difficult to know whether a reconstruction or which substructure in a reconstruction is accurate. For a neuron's reconstructions generated by multiple automatic tracing methods with different principles or models, their common substructures are highly reliable and named individual motifs. In this work, we propose a Vaa3D-based method called Lamotif to explore individual motifs in automatic reconstructions of a neuron. Lamotif utilizes the local alignment algorithm in BlastNeuron to extract local alignment pairs between a specified objective reconstruction and multiple reference reconstructions, and combines these pairs to generate individual motifs on the objective reconstruction. The proposed Lamotif is evaluated on reconstructions of 163 multiple species neurons, which are generated by four state-of-the-art tracing methods. Experimental results show that individual motifs are almost on corresponding gold standard reconstructions and have much higher precision rate than objective reconstructions themselves. Furthermore, an objective reconstruction is mostly quite accurate if its individual motifs have high recall rate. Individual motifs contain common geometry substructures in multiple reconstructions, and can be used to select some accurate substructures from a reconstruction or some accurate reconstructions from automatic reconstruction dataset of different neurons.
Collapse
Affiliation(s)
- Yishan He
- Faculty of Information Technology, Beijing University of Technology, 100 Pingleyuan, Chaoyang District, Beijing, 100124, China.,Beijing International Collaboration Base On Brain Informatics and Wisdom Services, 100 Pingleyuan, Chaoyang District, Beijing, 100124, China
| | - Jiajin Huang
- Faculty of Information Technology, Beijing University of Technology, 100 Pingleyuan, Chaoyang District, Beijing, 100124, China.,Beijing International Collaboration Base On Brain Informatics and Wisdom Services, 100 Pingleyuan, Chaoyang District, Beijing, 100124, China
| | - Gaowei Wu
- School of Artificial Intelligence, University of Chinese Academy of Sciences, 19(A) Yuquan Road, Shijingshan District, Beijing, 100049, China.,Institute of Automation, Chinese Academy of Sciences, Haidian District, 95 Zhongguancun East Road, Beijing, 100190, China
| | - Jian Yang
- Faculty of Information Technology, Beijing University of Technology, 100 Pingleyuan, Chaoyang District, Beijing, 100124, China. .,Beijing International Collaboration Base On Brain Informatics and Wisdom Services, 100 Pingleyuan, Chaoyang District, Beijing, 100124, China. .,School of Artificial Intelligence, University of Chinese Academy of Sciences, 19(A) Yuquan Road, Shijingshan District, Beijing, 100049, China.
| |
Collapse
|
15
|
Shih CT, Chen NY, Wang TY, He GW, Wang GT, Lin YJ, Lee TK, Chiang AS. NeuroRetriever: Automatic Neuron Segmentation for Connectome Assembly. Front Syst Neurosci 2021; 15:687182. [PMID: 34366800 PMCID: PMC8342815 DOI: 10.3389/fnsys.2021.687182] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/29/2021] [Accepted: 06/21/2021] [Indexed: 11/15/2022] Open
Abstract
Segmenting individual neurons from a large number of noisy raw images is the first step in building a comprehensive map of neuron-to-neuron connections for predicting information flow in the brain. Thousands of fluorescence-labeled brain neurons have been imaged. However, mapping a complete connectome remains challenging because imaged neurons are often entangled and manual segmentation of a large population of single neurons is laborious and prone to bias. In this study, we report an automatic algorithm, NeuroRetriever, for unbiased large-scale segmentation of confocal fluorescence images of single neurons in the adult Drosophila brain. NeuroRetriever uses a high-dynamic-range thresholding method to segment three-dimensional morphology of single neurons based on branch-specific structural features. Applying NeuroRetriever to automatically segment single neurons in 22,037 raw brain images, we successfully retrieved 28,125 individual neurons validated by human segmentation. Thus, automated NeuroRetriever will greatly accelerate 3D reconstruction of the single neurons for constructing the complete connectomes.
Collapse
Affiliation(s)
- Chi-Tin Shih
- Department of Applied Physics, Tunghai University, Taichung, Taiwan.,Brain Research Center, National Tsing Hua University, Hsinchu, Taiwan
| | - Nan-Yow Chen
- National Center for High-Performance Computing, National Applied Research Laboratories, Hsinchu, Taiwan
| | - Ting-Yuan Wang
- Institute of Biotechnology and Department of Life Science, National Tsing Hua University, Hsinchu, Taiwan
| | - Guan-Wei He
- Department of Computer Science, National Yang Ming Chiao Tung University, Hsinchu, Taiwan
| | - Guo-Tzau Wang
- National Center for High-Performance Computing, National Applied Research Laboratories, Hsinchu, Taiwan
| | - Yen-Jen Lin
- National Center for High-Performance Computing, National Applied Research Laboratories, Hsinchu, Taiwan
| | - Ting-Kuo Lee
- Institute of Physics, Academia Sinica, Taipei, Taiwan.,Department of Physics, National Sun Yat-sen University, Kaohsiung, Taiwan
| | - Ann-Shyn Chiang
- Department of Applied Physics, Tunghai University, Taichung, Taiwan.,Brain Research Center, National Tsing Hua University, Hsinchu, Taiwan.,Institute of Physics, Academia Sinica, Taipei, Taiwan.,Institute of Systems Neuroscience, National Tsing Hua University, Hsinchu, Taiwan.,Department of Biomedical Science and Environmental Biology, Kaohsiung Medical University, Kaohsiung, Taiwan.,Kavli Institute for Brain and Mind, University of California, San Diego, San Diego, CA, United States
| |
Collapse
|
16
|
Yang B, Chen W, Luo H, Tan Y, Liu M, Wang Y. Neuron Image Segmentation via Learning Deep Features and Enhancing Weak Neuronal Structures. IEEE J Biomed Health Inform 2021; 25:1634-1645. [PMID: 32809948 DOI: 10.1109/jbhi.2020.3017540] [Citation(s) in RCA: 9] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/07/2022]
Abstract
Neuron morphology reconstruction (tracing) in 3D volumetric images is critical for neuronal research. However, most existing neuron tracing methods are not applicable in challenging datasets where the neuron images are contaminated by noises or containing weak filament signals. In this paper, we present a two-stage 3D neuron segmentation approach via learning deep features and enhancing weak neuronal structures, to reduce the impact of image noise in the data and enhance the weak-signal neuronal structures. In the first stage, we train a voxel-wise multi-level fully convolutional network (FCN), which specializes in learning deep features, to obtain the 3D neuron image segmentation maps in an end-to-end manner. In the second stage, a ray-shooting model is employed to detect the discontinued segments in segmentation results of the first-stage, and the local neuron diameter of the broken point is estimated and direction of the filamentary fragment is detected by rayburst sampling algorithm. Then, a Hessian-repair model is built to repair the broken structures, by enhancing weak neuronal structures in a fibrous structure determined by the estimated local neuron diameter and the filamentary fragment direction. Experimental results demonstrate that our proposed segmentation approach achieves better segmentation performance than other state-of-the-art methods for 3D neuron segmentation. Compared with the neuron reconstruction results on the segmented images produced by other segmentation methods, the proposed approach gains 47.83% and 34.83% improvement in the average distance scores. The average Precision and Recall rates of the branch point detection with our proposed method are 38.74% and 22.53% higher than the detection results without segmentation.
Collapse
|
17
|
Jiang Y, Chen W, Liu M, Wang Y, Meijering E. 3D Neuron Microscopy Image Segmentation via the Ray-Shooting Model and a DC-BLSTM Network. IEEE TRANSACTIONS ON MEDICAL IMAGING 2021; 40:26-37. [PMID: 32881683 DOI: 10.1109/tmi.2020.3021493] [Citation(s) in RCA: 26] [Impact Index Per Article: 8.7] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/11/2023]
Abstract
The morphology reconstruction (tracing) of neurons in 3D microscopy images is important to neuroscience research. However, this task remains very challenging because of the low signal-to-noise ratio (SNR) and the discontinued segments of neurite patterns in the images. In this paper, we present a neuronal structure segmentation method based on the ray-shooting model and the Long Short-Term Memory (LSTM)-based network to enhance the weak-signal neuronal structures and remove background noise in 3D neuron microscopy images. Specifically, the ray-shooting model is used to extract the intensity distribution features within a local region of the image. And we design a neural network based on the dual channel bidirectional LSTM (DC-BLSTM) to detect the foreground voxels according to the voxel-intensity features and boundary-response features extracted by multiple ray-shooting models that are generated in the whole image. This way, we transform the 3D image segmentation task into multiple 1D ray/sequence segmentation tasks, which makes it much easier to label the training samples than many existing Convolutional Neural Network (CNN) based 3D neuron image segmentation methods. In the experiments, we evaluate the performance of our method on the challenging 3D neuron images from two datasets, the BigNeuron dataset and the Whole Mouse Brain Sub-image (WMBS) dataset. Compared with the neuron tracing results on the segmented images produced by other state-of-the-art neuron segmentation methods, our method improves the distance scores by about 32% and 27% in the BigNeuron dataset, and about 38% and 27% in the WMBS dataset.
Collapse
|
18
|
Zhao J, Chen X, Xiong Z, Liu D, Zeng J, Xie C, Zhang Y, Zha ZJ, Bi G, Wu F. Neuronal Population Reconstruction From Ultra-Scale Optical Microscopy Images via Progressive Learning. IEEE TRANSACTIONS ON MEDICAL IMAGING 2020; 39:4034-4046. [PMID: 32746145 DOI: 10.1109/tmi.2020.3009148] [Citation(s) in RCA: 9] [Impact Index Per Article: 2.3] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/11/2023]
Abstract
Reconstruction of neuronal populations from ultra-scale optical microscopy (OM) images is essential to investigate neuronal circuits and brain mechanisms. The noises, low contrast, huge memory requirement, and high computational cost pose significant challenges in the neuronal population reconstruction. Recently, many studies have been conducted to extract neuron signals using deep neural networks (DNNs). However, training such DNNs usually relies on a huge amount of voxel-wise annotations in OM images, which are expensive in terms of both finance and labor. In this paper, we propose a novel framework for dense neuronal population reconstruction from ultra-scale images. To solve the problem of high cost in obtaining manual annotations for training DNNs, we propose a progressive learning scheme for neuronal population reconstruction (PLNPR) which does not require any manual annotations. Our PLNPR scheme consists of a traditional neuron tracing module and a deep segmentation network that mutually complement and progressively promote each other. To reconstruct dense neuronal populations from a terabyte-sized ultra-scale image, we introduce an automatic framework which adaptively traces neurons block by block and fuses fragmented neurites in overlapped regions continuously and smoothly. We build a dataset "VISoR-40" which consists of 40 large-scale OM image blocks from cortical regions of a mouse. Extensive experimental results on our VISoR-40 dataset and the public BigNeuron dataset demonstrate the effectiveness and superiority of our method on neuronal population reconstruction and single neuron reconstruction. Furthermore, we successfully apply our method to reconstruct dense neuronal populations from an ultra-scale mouse brain slice. The proposed adaptive block propagation and fusion strategies greatly improve the completeness of neurites in dense neuronal population reconstruction.
Collapse
|
19
|
Velasco I, Toharia P, Benavides-Piccione R, Fernaud-Espinosa I, Brito JP, Mata S, DeFelipe J, Pastor L, Bayona S. Neuronize v2: Bridging the Gap Between Existing Proprietary Tools to Optimize Neuroscientific Workflows. Front Neuroanat 2020; 14:585793. [PMID: 33192345 PMCID: PMC7646287 DOI: 10.3389/fnana.2020.585793] [Citation(s) in RCA: 3] [Impact Index Per Article: 0.8] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/21/2020] [Accepted: 09/07/2020] [Indexed: 12/15/2022] Open
Abstract
Knowledge about neuron morphology is key to understanding brain structure and function. There are a variety of software tools that are used to segment and trace the neuron morphology. However, these tools usually utilize proprietary formats. This causes interoperability problems since the information extracted with one tool cannot be used in other tools. This article aims to improve neuronal reconstruction workflows by facilitating the interoperability between two of the most commonly used software tools—Neurolucida (NL) and Imaris (Filament Tracer). The new functionality has been included in an existing tool—Neuronize—giving rise to its second version. Neuronize v2 makes it possible to automatically use the data extracted with Imaris Filament Tracer to generate a tracing with dendritic spine information that can be read directly by NL. It also includes some other new features, such as the ability to unify and/or correct inaccurately-formed meshes (i.e., dendritic spines) and to calculate new metrics. This tool greatly facilitates the process of neuronal reconstruction, bridging the gap between existing proprietary tools to optimize neuroscientific workflows.
Collapse
Affiliation(s)
- Ivan Velasco
- Department of Computer Science, Universidad Rey Juan Carlos, Madrid, Spain
| | - Pablo Toharia
- DATSI, ETSIINF, Universidad Politécnica de Madrid, Spain.,Centro de Investigación Biomédica en Red sobre Enfermedades Neurodegenerativas, Madrid, Spain
| | - Ruth Benavides-Piccione
- Centro de Investigación Biomédica en Red sobre Enfermedades Neurodegenerativas, Madrid, Spain.,Laboratorio Cajal de Circuitos Corticales, Centro de Tecnología Biomédica, Universidad Politécnica de Madrid, Madrid, Spain.,Instituto Cajal (CSIC), Madrid, Spain
| | - Isabel Fernaud-Espinosa
- Centro de Investigación Biomédica en Red sobre Enfermedades Neurodegenerativas, Madrid, Spain.,Laboratorio Cajal de Circuitos Corticales, Centro de Tecnología Biomédica, Universidad Politécnica de Madrid, Madrid, Spain.,Instituto Cajal (CSIC), Madrid, Spain
| | - Juan P Brito
- DLSIIS, ETSIINF, Universidad Politécnica de Madrid, Madrid, Spain.,Center for Computational Simulation, Universidad Politécnica de Madrid, Madrid, Spain
| | - Susana Mata
- Department of Computer Science, Universidad Rey Juan Carlos, Madrid, Spain.,Center for Computational Simulation, Universidad Politécnica de Madrid, Madrid, Spain
| | - Javier DeFelipe
- Centro de Investigación Biomédica en Red sobre Enfermedades Neurodegenerativas, Madrid, Spain.,Laboratorio Cajal de Circuitos Corticales, Centro de Tecnología Biomédica, Universidad Politécnica de Madrid, Madrid, Spain.,Instituto Cajal (CSIC), Madrid, Spain
| | - Luis Pastor
- Department of Computer Science, Universidad Rey Juan Carlos, Madrid, Spain.,Center for Computational Simulation, Universidad Politécnica de Madrid, Madrid, Spain
| | - Sofia Bayona
- Department of Computer Science, Universidad Rey Juan Carlos, Madrid, Spain.,Center for Computational Simulation, Universidad Politécnica de Madrid, Madrid, Spain
| |
Collapse
|
20
|
Yang J, He Y, Liu X. Retrieving similar substructures on 3D neuron reconstructions. Brain Inform 2020; 7:14. [PMID: 33146802 PMCID: PMC7642183 DOI: 10.1186/s40708-020-00117-x] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/29/2020] [Accepted: 10/26/2020] [Indexed: 11/16/2022] Open
Abstract
Since manual tracing is time consuming and the performance of automatic tracing is unstable, it is still a challenging task to generate accurate neuron reconstruction efficiently and effectively. One strategy is generating a reconstruction automatically and then amending its inaccurate parts manually. Aiming at finding inaccurate substructures efficiently, we propose a pipeline to retrieve similar substructures on one or more neuron reconstructions, which are very similar to a marked problematic substructure. The pipeline consists of four steps: getting a marked substructure, constructing a query substructure, generating candidate substructures and retrieving most similar substructures. The retrieval procedure was tested on 163 gold standard reconstructions provided by the BigNeuron project and a reconstruction of a mouse’s large neuron. Experimental results showed that the implementation of the proposed methods is very efficient and all retrieved substructures are very similar to the marked one in numbers of nodes and branches, and degree of curvature.
Collapse
Affiliation(s)
- Jian Yang
- Faculty of Information Technology, Beijing University of Technology, Beijing, China. .,Beijing International Collaboration Base On Brain Informatics and Wisdom Services, Beijing, China. .,School of Artificial Intelligence, University of Chinese Academy of Sciences, Beijing, China.
| | - Yishan He
- Faculty of Information Technology, Beijing University of Technology, Beijing, China.,Beijing International Collaboration Base On Brain Informatics and Wisdom Services, Beijing, China
| | - Xuefeng Liu
- Faculty of Information Technology, Beijing University of Technology, Beijing, China.,Beijing International Collaboration Base On Brain Informatics and Wisdom Services, Beijing, China
| |
Collapse
|
21
|
Callara AL, Magliaro C, Ahluwalia A, Vanello N. A Smart Region-Growing Algorithm for Single-Neuron Segmentation From Confocal and 2-Photon Datasets. Front Neuroinform 2020; 14:9. [PMID: 32256332 PMCID: PMC7090132 DOI: 10.3389/fninf.2020.00009] [Citation(s) in RCA: 10] [Impact Index Per Article: 2.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/19/2019] [Accepted: 02/26/2020] [Indexed: 12/13/2022] Open
Abstract
Accurately digitizing the brain at the micro-scale is crucial for investigating brain structure-function relationships and documenting morphological alterations due to neuropathies. Here we present a new Smart Region Growing algorithm (SmRG) for the segmentation of single neurons in their intricate 3D arrangement within the brain. Its Region Growing procedure is based on a homogeneity predicate determined by describing the pixel intensity statistics of confocal acquisitions with a mixture model, enabling an accurate reconstruction of complex 3D cellular structures from high-resolution images of neural tissue. The algorithm's outcome is a 3D matrix of logical values identifying the voxels belonging to the segmented structure, thus providing additional useful volumetric information on neurons. To highlight the algorithm's full potential, we compared its performance in terms of accuracy, reproducibility, precision and robustness of 3D neuron reconstructions based on microscopic data from different brain locations and imaging protocols against both manual and state-of-the-art reconstruction tools.
Collapse
Affiliation(s)
| | - Chiara Magliaro
- Research Center “E. Piaggio” - University of Pisa, Pisa, Italy
| | - Arti Ahluwalia
- Research Center “E. Piaggio” - University of Pisa, Pisa, Italy
- Dipartimento di Ingegneria dell’Informazione, University of Pisa, Pisa, Italy
| | - Nicola Vanello
- Research Center “E. Piaggio” - University of Pisa, Pisa, Italy
- Dipartimento di Ingegneria dell’Informazione, University of Pisa, Pisa, Italy
| |
Collapse
|
22
|
Magliaro C, Callara AL, Vanello N, Ahluwalia A. Gotta Trace 'em All: A Mini-Review on Tools and Procedures for Segmenting Single Neurons Toward Deciphering the Structural Connectome. Front Bioeng Biotechnol 2019; 7:202. [PMID: 31555642 PMCID: PMC6727034 DOI: 10.3389/fbioe.2019.00202] [Citation(s) in RCA: 17] [Impact Index Per Article: 3.4] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/27/2019] [Accepted: 08/06/2019] [Indexed: 12/12/2022] Open
Abstract
Decoding the morphology and physical connections of all the neurons populating a brain is necessary for predicting and studying the relationships between its form and function, as well as for documenting structural abnormalities in neuropathies. Digitizing a complete and high-fidelity map of the mammalian brain at the micro-scale will allow neuroscientists to understand disease, consciousness, and ultimately what it is that makes us humans. The critical obstacle for reaching this goal is the lack of robust and accurate tools able to deal with 3D datasets representing dense-packed cells in their native arrangement within the brain. This obliges neuroscientist to manually identify the neurons populating an acquired digital image stack, a notably time-consuming procedure prone to human bias. Here we review the automatic and semi-automatic algorithms and software for neuron segmentation available in the literature, as well as the metrics purposely designed for their validation, highlighting their strengths and limitations. In this direction, we also briefly introduce the recent advances in tissue clarification that enable significant improvements in both optical access of neural tissue and image stack quality, and which could enable more efficient segmentation approaches. Finally, we discuss new methods and tools for processing tissues and acquiring images at sub-cellular scales, which will require new robust algorithms for identifying neurons and their sub-structures (e.g., spines, thin neurites). This will lead to a more detailed structural map of the brain, taking twenty-first century cellular neuroscience to the next level, i.e., the Structural Connectome.
Collapse
Affiliation(s)
- Chiara Magliaro
- Research Center "E. Piaggio", University of Pisa, Pisa, Italy
| | | | - Nicola Vanello
- Research Center "E. Piaggio", University of Pisa, Pisa, Italy.,Dipartimento di Ingegneria dell'Informazione, University of Pisa, Pisa, Italy
| | - Arti Ahluwalia
- Research Center "E. Piaggio", University of Pisa, Pisa, Italy.,Dipartimento di Ingegneria dell'Informazione, University of Pisa, Pisa, Italy
| |
Collapse
|
23
|
FMST: an Automatic Neuron Tracing Method Based on Fast Marching and Minimum Spanning Tree. Neuroinformatics 2019; 17:185-196. [PMID: 30039210 DOI: 10.1007/s12021-018-9392-y] [Citation(s) in RCA: 19] [Impact Index Per Article: 3.8] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/28/2022]
Abstract
Neuron reconstruction is an important technique in computational neuroscience. Although there are many reconstruction algorithms, few can generate robust results. In this paper, we propose a reconstruction algorithm called fast marching spanning tree (FMST). FMST is based on a minimum spanning tree method (MST) and improve its performance in two aspects: faster implementation and no loss of small branches. The contributions of the proposed method are as follows. Firstly, the Euclidean distance weight of edges in MST is improved to be a more reasonable value, which is related to the probability of the existence of an edge. Secondly, a strategy of pruning nodes is presented, which is based on the radius of a node's inscribed ball. Thirdly, separate branches of broken neuron reconstructions can be merged into a single tree. FMST and many other state of the art reconstruction methods were implemented on two datasets: 120 Drosophila neurons and 163 neurons with gold standard reconstructions. Qualitative and quantitative analysis on experimental results demonstrates that the performance of FMST is good compared with many existing methods. Especially, on the 91 fruitfly neurons with gold standard and evaluated by five metrics, FMST is one of two methods with best performance among all 27 state of the art reconstruction methods. FMST is a good and practicable neuron reconstruction algorithm, and can be implemented in Vaa3D platform as a neuron tracing plugin.
Collapse
|
24
|
Abdolhoseini M, Kluge MG, Walker FR, Johnson SJ. Segmentation, Tracing, and Quantification of Microglial Cells from 3D Image Stacks. Sci Rep 2019; 9:8557. [PMID: 31189918 PMCID: PMC6561929 DOI: 10.1038/s41598-019-44917-6] [Citation(s) in RCA: 14] [Impact Index Per Article: 2.8] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/23/2018] [Accepted: 03/27/2019] [Indexed: 12/27/2022] Open
Abstract
Microglia play a central role in modulating synaptic structure and physiology, learning and memory processes. They exhibit morphological changes to perform these roles, therefore the morphological study of microglia can help to understand their functionality. Many promising methods are proposed to automatically segment the blood vessels or reconstruct the neuronal morphology. However, they often fail to accurately capture microglia organizations due to the striking structural differences. This requires a more sophisticated approach of reconstruction taking into account the varying nature of branch structures and soma sizes. To this end, we propose an automated method to reconstruct microglia, and quantify their features from 2D/3D image datasets. We first employ multilevel thresholding to segment soma volumes(3D)/areas(2D) and recognize foreground voxels/pixels. Seed points sampled from the foreground, are connected to form the skeleton of the branches via the tracing process. The reconstructed data is quantified and written in SWC standard file format. We have applied our method to 3D image datasets of microglia, then evaluated the results using ground truth data, and compared them to those achieved via the state-of-the-art methods. Our method outperforms the others both in accuracy and computational time.
Collapse
Affiliation(s)
- Mahmoud Abdolhoseini
- The University of Newcastle, School of Electrical Engineering and Computing, Callaghan, NSW, 2308, Australia.
| | - Murielle G Kluge
- The University of Newcastle, School of Biomedical Sciences and Pharmacy, Callaghan, NSW, 2308, Australia.,The Hunter Medical Research Institute, New Lambton, NSW, 2305, Australia
| | - Frederick R Walker
- The University of Newcastle, School of Biomedical Sciences and Pharmacy, Callaghan, NSW, 2308, Australia.,The Hunter Medical Research Institute, New Lambton, NSW, 2305, Australia
| | - Sarah J Johnson
- The University of Newcastle, School of Electrical Engineering and Computing, Callaghan, NSW, 2308, Australia.,The Hunter Medical Research Institute, New Lambton, NSW, 2305, Australia
| |
Collapse
|
25
|
Skibbe H, Reisert M, Nakae K, Watakabe A, Hata J, Mizukami H, Okano H, Yamamori T, Ishii S. PAT-Probabilistic Axon Tracking for Densely Labeled Neurons in Large 3-D Micrographs. IEEE TRANSACTIONS ON MEDICAL IMAGING 2019; 38:69-78. [PMID: 30010551 DOI: 10.1109/tmi.2018.2855736] [Citation(s) in RCA: 8] [Impact Index Per Article: 1.6] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/08/2023]
Abstract
A major goal of contemporary neuroscience research is to map the structural connectivity of mammalian brain using microscopy imaging data. In this context, the reconstruction of densely labeled axons from two-photon microscopy images is a challenging and important task. The visually overlapping, crossing, and often strongly distorted images of the axons allow many ambiguous interpretations to be made. We address the problem of tracking axons in densely labeled samples of neurons in large image data sets acquired from marmoset brains. Our high-resolution images were acquired using two-photon microscopy and they provided whole brain coverage, occupying terabytes of memory. Both the image distortions and the large data set size frequently make it impractical to apply present-day neuron tracing algorithms to such data due to the optimization of such algorithms to the precise tracing of either single or sparse sets of neurons. Thus, new tracking techniques are needed. We propose a probabilistic axon tracking algorithm (PAT). PAT tackles the tracking of axons in two steps: locally (L-PAT) and globally (G-PAT). L-PAT is a probabilistic tracking algorithm that can tackle distorted, cluttered images of densely labeled axons. L-PAT divides a large micrograph into smaller image stacks. It then processes each image stack independently before mapping the axons in each image to a sparse model of axon trajectories. G-PAT merges the sparse L-PAT models into a single global model of axon trajectories by minimizing a global objective function using a probabilistic optimization method. We demonstrate the superior performance of PAT over standard approaches on synthetic data. Furthermore, we successfully apply PAT to densely labeled axons in large images acquired from marmoset brains.
Collapse
|
26
|
Liu S, Zhang D, Song Y, Peng H, Cai W. Automated 3-D Neuron Tracing With Precise Branch Erasing and Confidence Controlled Back Tracking. IEEE TRANSACTIONS ON MEDICAL IMAGING 2018; 37:2441-2452. [PMID: 29993997 DOI: 10.1109/tmi.2018.2833420] [Citation(s) in RCA: 14] [Impact Index Per Article: 2.3] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/08/2023]
Abstract
The automatic reconstruction of single neurons from microscopic images is essential to enable large-scale data-driven investigations in neuron morphology research. However, few previous methods were able to generate satisfactory results automatically from 3-D microscopic images without human intervention. In this paper, we developed a new algorithm for automatic 3-D neuron reconstruction. The main idea of the proposed algorithm is to iteratively track backward from the potential neuronal termini to the soma centre. An online confidence score is computed to decide if a tracing iteration should be stopped and discarded from the final reconstruction. The performance improvements comparing with the previous methods are mainly introduced by a more accurate estimation of the traced area and the confidence controlled back-tracking algorithm. The proposed algorithm supports large-scale batch-processing by requiring only one user specified parameter for background segmentation. We bench tested the proposed algorithm on the images obtained from both the DIADEM challenge and the BigNeuron challenge. Our proposed algorithm achieved the state-of-the-art results.
Collapse
|