1
|
Liu M, Wu S, Chen R, Lin Z, Wang Y, Meijering E. Brain Image Segmentation for Ultrascale Neuron Reconstruction via an Adaptive Dual-Task Learning Network. IEEE TRANSACTIONS ON MEDICAL IMAGING 2024; 43:2574-2586. [PMID: 38373129 DOI: 10.1109/tmi.2024.3367384] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 02/21/2024]
Abstract
Accurate morphological reconstruction of neurons in whole brain images is critical for brain science research. However, due to the wide range of whole brain imaging, uneven staining, and optical system fluctuations, there are significant differences in image properties between different regions of the ultrascale brain image, such as dramatically varying voxel intensities and inhomogeneous distribution of background noise, posing an enormous challenge to neuron reconstruction from whole brain images. In this paper, we propose an adaptive dual-task learning network (ADTL-Net) to quickly and accurately extract neuronal structures from ultrascale brain images. Specifically, this framework includes an External Features Classifier (EFC) and a Parameter Adaptive Segmentation Decoder (PASD), which share the same Multi-Scale Feature Encoder (MSFE). MSFE introduces an attention module named Channel Space Fusion Module (CSFM) to extract structure and intensity distribution features of neurons at different scales for addressing the problem of anisotropy in 3D space. Then, EFC is designed to classify these feature maps based on external features, such as foreground intensity distributions and image smoothness, and select specific PASD parameters to decode them of different classes to obtain accurate segmentation results. PASD contains multiple sets of parameters trained by different representative complex signal-to-noise distribution image blocks to handle various images more robustly. Experimental results prove that compared with other advanced segmentation methods for neuron reconstruction, the proposed method achieves state-of-the-art results in the task of neuron reconstruction from ultrascale brain images, with an improvement of about 49% in speed and 12% in F1 score.
Collapse
|
2
|
Hoffmann C, Cho E, Zalesky A, Di Biase MA. From pixels to connections: exploring in vitro neuron reconstruction software for network graph generation. Commun Biol 2024; 7:571. [PMID: 38750282 PMCID: PMC11096190 DOI: 10.1038/s42003-024-06264-9] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/10/2023] [Accepted: 04/29/2024] [Indexed: 05/18/2024] Open
Abstract
Digital reconstruction has been instrumental in deciphering how in vitro neuron architecture shapes information flow. Emerging approaches reconstruct neural systems as networks with the aim of understanding their organization through graph theory. Computational tools dedicated to this objective build models of nodes and edges based on key cellular features such as somata, axons, and dendrites. Fully automatic implementations of these tools are readily available, but they may also be purpose-built from specialized algorithms in the form of multi-step pipelines. Here we review software tools informing the construction of network models, spanning from noise reduction and segmentation to full network reconstruction. The scope and core specifications of each tool are explicitly defined to assist bench scientists in selecting the most suitable option for their microscopy dataset. Existing tools provide a foundation for complete network reconstruction, however more progress is needed in establishing morphological bases for directed/weighted connectivity and in software validation.
Collapse
Affiliation(s)
- Cassandra Hoffmann
- Systems Neuroscience Lab, Melbourne Neuropsychiatry Centre, Department of Psychiatry, The University of Melbourne, Parkville, Australia.
| | - Ellie Cho
- Biological Optical Microscopy Platform, University of Melbourne, Parkville, Australia
| | - Andrew Zalesky
- Systems Neuroscience Lab, Melbourne Neuropsychiatry Centre, Department of Psychiatry, The University of Melbourne, Parkville, Australia
- Department of Biomedical Engineering, The University of Melbourne, Parkville, Australia
| | - Maria A Di Biase
- Systems Neuroscience Lab, Melbourne Neuropsychiatry Centre, Department of Psychiatry, The University of Melbourne, Parkville, Australia
- Stem Cell Disease Modelling Lab, Department of Anatomy and Physiology, The University of Melbourne, Parkville, Australia
- Psychiatry Neuroimaging Laboratory, Department of Psychiatry, Brigham and Women's Hospital, Harvard Medical School, Boston, USA
| |
Collapse
|
3
|
Chen R, Liu M, Chen W, Wang Y, Meijering E. Deep learning in mesoscale brain image analysis: A review. Comput Biol Med 2023; 167:107617. [PMID: 37918261 DOI: 10.1016/j.compbiomed.2023.107617] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/01/2023] [Revised: 10/06/2023] [Accepted: 10/23/2023] [Indexed: 11/04/2023]
Abstract
Mesoscale microscopy images of the brain contain a wealth of information which can help us understand the working mechanisms of the brain. However, it is a challenging task to process and analyze these data because of the large size of the images, their high noise levels, the complex morphology of the brain from the cellular to the regional and anatomical levels, the inhomogeneous distribution of fluorescent labels in the cells and tissues, and imaging artifacts. Due to their impressive ability to extract relevant information from images, deep learning algorithms are widely applied to microscopy images of the brain to address these challenges and they perform superiorly in a wide range of microscopy image processing and analysis tasks. This article reviews the applications of deep learning algorithms in brain mesoscale microscopy image processing and analysis, including image synthesis, image segmentation, object detection, and neuron reconstruction and analysis. We also discuss the difficulties of each task and possible directions for further research.
Collapse
Affiliation(s)
- Runze Chen
- College of Electrical and Information Engineering, National Engineering Laboratory for Robot Visual Perception and Control Technology, Hunan University, Changsha, 410082, China
| | - Min Liu
- College of Electrical and Information Engineering, National Engineering Laboratory for Robot Visual Perception and Control Technology, Hunan University, Changsha, 410082, China; Research Institute of Hunan University in Chongqing, Chongqing, 401135, China.
| | - Weixun Chen
- College of Electrical and Information Engineering, National Engineering Laboratory for Robot Visual Perception and Control Technology, Hunan University, Changsha, 410082, China
| | - Yaonan Wang
- College of Electrical and Information Engineering, National Engineering Laboratory for Robot Visual Perception and Control Technology, Hunan University, Changsha, 410082, China
| | - Erik Meijering
- School of Computer Science and Engineering, University of New South Wales, Sydney 2052, New South Wales, Australia
| |
Collapse
|
4
|
Liu Y, Wang G, Ascoli GA, Zhou J, Liu L. Neuron tracing from light microscopy images: automation, deep learning and bench testing. Bioinformatics 2022; 38:5329-5339. [PMID: 36303315 PMCID: PMC9750132 DOI: 10.1093/bioinformatics/btac712] [Citation(s) in RCA: 13] [Impact Index Per Article: 6.5] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/28/2022] [Revised: 10/19/2022] [Accepted: 10/26/2022] [Indexed: 12/24/2022] Open
Abstract
MOTIVATION Large-scale neuronal morphologies are essential to neuronal typing, connectivity characterization and brain modeling. It is widely accepted that automation is critical to the production of neuronal morphology. Despite previous survey papers about neuron tracing from light microscopy data in the last decade, thanks to the rapid development of the field, there is a need to update recent progress in a review focusing on new methods and remarkable applications. RESULTS This review outlines neuron tracing in various scenarios with the goal to help the community understand and navigate tools and resources. We describe the status, examples and accessibility of automatic neuron tracing. We survey recent advances of the increasingly popular deep-learning enhanced methods. We highlight the semi-automatic methods for single neuron tracing of mammalian whole brains as well as the resulting datasets, each containing thousands of full neuron morphologies. Finally, we exemplify the commonly used datasets and metrics for neuron tracing bench testing.
Collapse
Affiliation(s)
- Yufeng Liu
- School of Biological Science and Medical Engineering, Southeast University, Nanjing, China
| | - Gaoyu Wang
- School of Computer Science and Engineering, Southeast University, Nanjing, China
| | - Giorgio A Ascoli
- Center for Neural Informatics, Structures, & Plasticity, Krasnow Institute for Advanced Study, George Mason University, Fairfax, VA, USA
| | - Jiangning Zhou
- Institute of Brain Science, The First Affiliated Hospital of Anhui Medical University, Hefei, China
| | - Lijuan Liu
- School of Biological Science and Medical Engineering, Southeast University, Nanjing, China
| |
Collapse
|
5
|
Garlick E, Thomas SG, Owen DM. Super-Resolution Imaging Approaches for Quantifying F-Actin in Immune Cells. Front Cell Dev Biol 2021; 9:676066. [PMID: 34490240 PMCID: PMC8416680 DOI: 10.3389/fcell.2021.676066] [Citation(s) in RCA: 4] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/04/2021] [Accepted: 05/20/2021] [Indexed: 11/21/2022] Open
Abstract
Immune cells comprise a diverse set of cells that undergo a complex array of biological processes that must be tightly regulated. A key component of cellular machinery that achieves this is the cytoskeleton. Therefore, imaging and quantitatively describing the architecture and dynamics of the cytoskeleton is an important research goal. Optical microscopy is well suited to this task. Here, we review the latest in the state-of-the-art methodology for labeling the cytoskeleton, fluorescence microscopy hardware suitable for such imaging and quantitative statistical analysis software applicable to describing cytoskeletal structures. We also highlight ongoing challenges and areas for future development.
Collapse
Affiliation(s)
- Evelyn Garlick
- Institute of Cardiovascular Sciences, College of Medical and Dental Science, University of Birmingham, Birmingham, United Kingdom.,Centre of Membrane Proteins and Receptors, University of Birmingham and University of Nottingham, Midlands, United Kingdom
| | - Steven G Thomas
- Institute of Cardiovascular Sciences, College of Medical and Dental Science, University of Birmingham, Birmingham, United Kingdom.,Centre of Membrane Proteins and Receptors, University of Birmingham and University of Nottingham, Midlands, United Kingdom
| | - Dylan M Owen
- Centre of Membrane Proteins and Receptors, University of Birmingham and University of Nottingham, Midlands, United Kingdom.,Institute for Immunology and Immunotherapy, College of Medical and Dental Science and School of Mathematics, College of Engineering and Physical Science, University of Birmingham, Birmingham, United Kingdom
| |
Collapse
|
6
|
Shih CT, Chen NY, Wang TY, He GW, Wang GT, Lin YJ, Lee TK, Chiang AS. NeuroRetriever: Automatic Neuron Segmentation for Connectome Assembly. Front Syst Neurosci 2021; 15:687182. [PMID: 34366800 PMCID: PMC8342815 DOI: 10.3389/fnsys.2021.687182] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/29/2021] [Accepted: 06/21/2021] [Indexed: 11/15/2022] Open
Abstract
Segmenting individual neurons from a large number of noisy raw images is the first step in building a comprehensive map of neuron-to-neuron connections for predicting information flow in the brain. Thousands of fluorescence-labeled brain neurons have been imaged. However, mapping a complete connectome remains challenging because imaged neurons are often entangled and manual segmentation of a large population of single neurons is laborious and prone to bias. In this study, we report an automatic algorithm, NeuroRetriever, for unbiased large-scale segmentation of confocal fluorescence images of single neurons in the adult Drosophila brain. NeuroRetriever uses a high-dynamic-range thresholding method to segment three-dimensional morphology of single neurons based on branch-specific structural features. Applying NeuroRetriever to automatically segment single neurons in 22,037 raw brain images, we successfully retrieved 28,125 individual neurons validated by human segmentation. Thus, automated NeuroRetriever will greatly accelerate 3D reconstruction of the single neurons for constructing the complete connectomes.
Collapse
Affiliation(s)
- Chi-Tin Shih
- Department of Applied Physics, Tunghai University, Taichung, Taiwan.,Brain Research Center, National Tsing Hua University, Hsinchu, Taiwan
| | - Nan-Yow Chen
- National Center for High-Performance Computing, National Applied Research Laboratories, Hsinchu, Taiwan
| | - Ting-Yuan Wang
- Institute of Biotechnology and Department of Life Science, National Tsing Hua University, Hsinchu, Taiwan
| | - Guan-Wei He
- Department of Computer Science, National Yang Ming Chiao Tung University, Hsinchu, Taiwan
| | - Guo-Tzau Wang
- National Center for High-Performance Computing, National Applied Research Laboratories, Hsinchu, Taiwan
| | - Yen-Jen Lin
- National Center for High-Performance Computing, National Applied Research Laboratories, Hsinchu, Taiwan
| | - Ting-Kuo Lee
- Institute of Physics, Academia Sinica, Taipei, Taiwan.,Department of Physics, National Sun Yat-sen University, Kaohsiung, Taiwan
| | - Ann-Shyn Chiang
- Department of Applied Physics, Tunghai University, Taichung, Taiwan.,Brain Research Center, National Tsing Hua University, Hsinchu, Taiwan.,Institute of Physics, Academia Sinica, Taipei, Taiwan.,Institute of Systems Neuroscience, National Tsing Hua University, Hsinchu, Taiwan.,Department of Biomedical Science and Environmental Biology, Kaohsiung Medical University, Kaohsiung, Taiwan.,Kavli Institute for Brain and Mind, University of California, San Diego, San Diego, CA, United States
| |
Collapse
|
7
|
Di Re J, Kayasandik C, Botello-Lins G, Labate D, Laezza F. Imaging of the Axon Initial Segment. ACTA ACUST UNITED AC 2020; 89:e78. [PMID: 31532918 DOI: 10.1002/cpns.78] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/10/2022]
Abstract
The axon initial segment (AIS) is the first 20- to 60-μm segment of the axon proximal to the soma of a neuron. This highly specialized subcellular domain is the initiation site of the action potential and contains a high concentration of voltage-gated ion channels held in place by a complex nexus of scaffolding and regulatory proteins that ensure proper electrical activity of the neuron. Studies have shown that dysfunction of many AIS channels and scaffolding proteins occurs in a variety of neuropsychiatric and neurodegenerative diseases, raising the need to develop accurate methods for visualization and quantification of the AIS and its protein content in models of normal and disease conditions. In this article, we describe methods for immunolabeling AIS proteins in cultured neurons and brain slices as well as methods for quantifying protein expression and pattern distribution using fluorescent labeling of these proteins. © 2019 by John Wiley & Sons, Inc.
Collapse
Affiliation(s)
- Jessica Di Re
- Neuroscience Graduate Program, University of Texas Medical Branch, Galveston, Texas.,Department of Pharmacology and Toxicology, University of Texas Medical Branch, Galveston, Texas
| | - Cihan Kayasandik
- Department of Computer Engineering, Istanbul Medipol University, Istanbul, Turkey
| | - Gonzalo Botello-Lins
- Biotechnology Program, Clear Falls High School, Clear Creek Independent School District, League City, Texas
| | - Demetrio Labate
- Department of Mathematics, University of Houston, Houston, Texas
| | - Fernanda Laezza
- Department of Pharmacology and Toxicology, University of Texas Medical Branch, Galveston, Texas
| |
Collapse
|
8
|
Janušonis S, Detering N, Metzler R, Vojta T. Serotonergic Axons as Fractional Brownian Motion Paths: Insights Into the Self-Organization of Regional Densities. Front Comput Neurosci 2020; 14:56. [PMID: 32670042 PMCID: PMC7328445 DOI: 10.3389/fncom.2020.00056] [Citation(s) in RCA: 15] [Impact Index Per Article: 3.8] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/27/2019] [Accepted: 05/19/2020] [Indexed: 01/03/2023] Open
Abstract
All vertebrate brains contain a dense matrix of thin fibers that release serotonin (5-hydroxytryptamine), a neurotransmitter that modulates a wide range of neural, glial, and vascular processes. Perturbations in the density of this matrix have been associated with a number of mental disorders, including autism and depression, but its self-organization and plasticity remain poorly understood. We introduce a model based on reflected Fractional Brownian Motion (FBM), a rigorously defined stochastic process, and show that it recapitulates some key features of regional serotonergic fiber densities. Specifically, we use supercomputing simulations to model fibers as FBM-paths in two-dimensional brain-like domains and demonstrate that the resultant steady state distributions approximate the fiber distributions in physical brain sections immunostained for the serotonin transporter (a marker for serotonergic axons in the adult brain). We suggest that this framework can support predictive descriptions and manipulations of the serotonergic matrix and that it can be further extended to incorporate the detailed physical properties of the fibers and their environment.
Collapse
Affiliation(s)
- Skirmantas Janušonis
- Department of Psychological and Brain Sciences, University of California, Santa Barbara, Santa Barbara, CA, United States
| | - Nils Detering
- Department of Statistics and Applied Probability, University of California, Santa Barbara, Santa Barbara, CA, United States
| | - Ralf Metzler
- Institute of Physics and Astronomy, University of Potsdam, Potsdam, Germany
| | - Thomas Vojta
- Department of Physics, Missouri University of Science and Technology, Rolla, MO, United States
| |
Collapse
|
9
|
Callara AL, Magliaro C, Ahluwalia A, Vanello N. A Smart Region-Growing Algorithm for Single-Neuron Segmentation From Confocal and 2-Photon Datasets. Front Neuroinform 2020; 14:9. [PMID: 32256332 PMCID: PMC7090132 DOI: 10.3389/fninf.2020.00009] [Citation(s) in RCA: 10] [Impact Index Per Article: 2.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/19/2019] [Accepted: 02/26/2020] [Indexed: 12/13/2022] Open
Abstract
Accurately digitizing the brain at the micro-scale is crucial for investigating brain structure-function relationships and documenting morphological alterations due to neuropathies. Here we present a new Smart Region Growing algorithm (SmRG) for the segmentation of single neurons in their intricate 3D arrangement within the brain. Its Region Growing procedure is based on a homogeneity predicate determined by describing the pixel intensity statistics of confocal acquisitions with a mixture model, enabling an accurate reconstruction of complex 3D cellular structures from high-resolution images of neural tissue. The algorithm's outcome is a 3D matrix of logical values identifying the voxels belonging to the segmented structure, thus providing additional useful volumetric information on neurons. To highlight the algorithm's full potential, we compared its performance in terms of accuracy, reproducibility, precision and robustness of 3D neuron reconstructions based on microscopic data from different brain locations and imaging protocols against both manual and state-of-the-art reconstruction tools.
Collapse
Affiliation(s)
| | - Chiara Magliaro
- Research Center “E. Piaggio” - University of Pisa, Pisa, Italy
| | - Arti Ahluwalia
- Research Center “E. Piaggio” - University of Pisa, Pisa, Italy
- Dipartimento di Ingegneria dell’Informazione, University of Pisa, Pisa, Italy
| | - Nicola Vanello
- Research Center “E. Piaggio” - University of Pisa, Pisa, Italy
- Dipartimento di Ingegneria dell’Informazione, University of Pisa, Pisa, Italy
| |
Collapse
|
10
|
Li AZ, Corey L, Zhu J. Random-Reaction-Seed Method for Automated Identification of Neurite Elongation and Branching. Sci Rep 2019; 9:2908. [PMID: 30814668 PMCID: PMC6393450 DOI: 10.1038/s41598-019-39962-0] [Citation(s) in RCA: 3] [Impact Index Per Article: 0.6] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/11/2018] [Accepted: 02/01/2019] [Indexed: 01/09/2023] Open
Abstract
Conventional deterministic algorithms (i.e., skeletonization and edge-detection) lack robustness and sensitivity to reliably detect the neurite elongation and branching of low signal-to-noise-ratio microscopy images. Neurite outgrowth experiments produce an enormous number of images that require automated measurement; however, the tracking of neurites is easily lost in the automated process due to the intrinsic variability of neurites (either axon or dendrite) under stimuli. We have developed a stochastic random-reaction-seed (RRS) method to identify neurite elongation and branching accurately and automatically. The random-seeding algorithm of RRS is based on the hidden-Markov-model (HMM) to offer a robust enough way for tracing arbitrary neurite structures, while the reaction-seeding algorithm of RRS secures the efficiency of random seeding. It is noteworthy that RRS is capable of tracing a whole neurite branch by only one initial seed, so that RRS is proficient at quantifying extensive amounts of neurite outgrowth images with noisy background in microfluidic devices of biomedical engineering fields. The method also showed notable performance for reconstructing of net-like structures, and thus is expected to be proficient for biomedical feature extractions in a wide range of applications, such as retinal vessel segmentation and cell membrane profiling in spurious-edge-tissues.
Collapse
Affiliation(s)
- Alvason Zhenhua Li
- Vaccine and Infectious Disease Division, Fred Hutchinson Cancer Research Center, Seattle, WA, 98109, USA.
| | - Lawrence Corey
- Vaccine and Infectious Disease Division, Fred Hutchinson Cancer Research Center, Seattle, WA, 98109, USA.,Clinical Research Division, Fred Hutchinson Cancer Research Center, Seattle, WA, 98109, USA.,Department of Laboratory Medicine, University of Washington, Seattle, WA, 98195, USA.,Department of Medicine, University of Washington, Seattle, WA, 98195, USA
| | - Jia Zhu
- Vaccine and Infectious Disease Division, Fred Hutchinson Cancer Research Center, Seattle, WA, 98109, USA.,Department of Laboratory Medicine, University of Washington, Seattle, WA, 98195, USA
| |
Collapse
|