Neuron tracing from light microscopy images: automation, deep learning and bench testing

Collaborative augmented reconstruction of 3D neuron morphology in mouse and human brains

Digital reconstruction of the intricate 3D morphology of individual neurons from microscopic images is a crucial challenge in both individual laboratories and large-scale projects focusing on cell types and brain anatomy. This task often fails in both conventional manual reconstruction and state-of-the-art artificial intelligence (AI)-based automatic reconstruction algorithms. It is also challenging to organize multiple neuroanatomists to generate and cross-validate biologically relevant and mutually agreed upon reconstructions in large-scale data production. Based on collaborative group intelligence augmented by AI, we developed a collaborative augmented reconstruction (CAR) platform for neuron reconstruction at scale. This platform allows for immersive interaction and efficient collaborative editing of neuron anatomy using a variety of devices, such as desktop workstations, virtual reality headsets and mobile phones, enabling users to contribute anytime and anywhere and to take advantage of several AI-based automation tools. We tested CAR's applicability for challenging mouse and human neurons toward scaled and faithful data production.

Accelerating the continuous community sharing of digital neuromorphology data

The tree-like morphology of neurons and glia is a key cellular determinant of circuit connectivity and metabolic function in the nervous system of essentially all animals. To elucidate the contribution of specific cell types to both physiological and pathological brain states, it is important to access detailed neuroanatomy data for quantitative analysis and computational modeling. NeuroMorpho.Org is the largest online collection of freely available digital neural reconstructions and related metadata and is continuously updated with new uploads. Earlier in the project, we released multiple datasets together yearly, but this process caused an average delay of several months in making the data public. Moreover, in the past 5 years, >80% of invited authors agreed to share their data with the community via NeuroMorpho.Org, up from <20% in the first 5 years of the project. In the same period, the average number of reconstructions per publication increased 600%, creating the need for automatic processing to release more reconstructions in less time. The progressive automation of our pipeline enabled the transition to agile releases of individual datasets as soon as they are ready. The overall time from data identification to public sharing decreased by 63.7%; 78% of the datasets are now released in less than 3 months with an average workflow duration below 40 days. Furthermore, the mean processing time per reconstruction dropped from 3 h to 2 min. With these continuous improvements, NeuroMorpho.Org strives to forge a positive culture of open data. Most importantly, the new, original research enabled through reuse of datasets across the world has a multiplicative effect on science discovery, benefiting both authors and users.

Computational Tools for Neuronal Morphometric Analysis: A Systematic Search and Review

Morphometry is fundamental for studying and correlating neuronal morphology with brain functions. With increasing computational power, it is possible to extract morphometric characteristics automatically, including features such as length, volume, and number of neuron branches. However, to the best of our knowledge, there is no mapping of morphometric tools yet. In this context, we conducted a systematic search and review to identify and analyze tools within the scope of neuron analysis. Thus, the work followed a well-defined protocol and sought to answer the following research questions: What open-source tools are available for neuronal morphometric analysis? What morphometric characteristics are extracted by these tools? For this, aiming for greater robustness and coverage, the study was based on the paper analysis as well as the study of documentation and tests with the tools available in repositories. We analyzed 1,586 papers and mapped 23 tools, where NeuroM, L-Measure, and NeuroMorphoVis extract the most features. Furthermore, we contribute to the body of knowledge with the unprecedented presentation of 150 unique morphometric features whose terminologies were categorized and standardized. Overall, the study contributes to advancing the understanding of the complex mechanisms underlying the brain.

Connecto-informatics at the mesoscale: current advances in image processing and analysis for mapping the brain connectivity

Mapping neural connections within the brain has been a fundamental goal in neuroscience to understand better its functions and changes that follow aging and diseases. Developments in imaging technology, such as microscopy and labeling tools, have allowed researchers to visualize this connectivity through high-resolution brain-wide imaging. With this, image processing and analysis have become more crucial. However, despite the wealth of neural images generated, access to an integrated image processing and analysis pipeline to process these data is challenging due to scattered information on available tools and methods. To map the neural connections, registration to atlases and feature extraction through segmentation and signal detection are necessary. In this review, our goal is to provide an updated overview of recent advances in these image-processing methods, with a particular focus on fluorescent images of the mouse brain. Our goal is to outline a pathway toward an integrated image-processing pipeline tailored for connecto-informatics. An integrated workflow of these image processing will facilitate researchers' approach to mapping brain connectivity to better understand complex brain networks and their underlying brain functions. By highlighting the image-processing tools available for fluroscent imaging of the mouse brain, this review will contribute to a deeper grasp of connecto-informatics, paving the way for better comprehension of brain connectivity and its implications.

Quantifying morphologies of developing neuronal cells using deep learning with imperfect annotations

The functionality of human intelligence relies on the interaction and health of neurons, hence, quantifying neuronal morphologies can be crucial for investigating the functionality of the human brain. This paper proposes a deep learning (DL) based method for segmenting and quantifying neuronal structures in fluorescence microscopy images of developing neuronal cells cultured in vitro. Compared to the majority of supervised DL-based segmentation methods that heavily rely on creating exact corresponding masks of neuronal structures for the preparation of training samples, the proposed approach allows for imperfect annotation of neurons, as it only requires tracing the centrelines of the neurites. This ability accelerates the preparation of training data by several folds. Our proposed framework is built on a modified version of PSPNet with an EfficientNet backbone pre-trained on the CityScapes dataset. To handle the imperfectness of training samples, we incorporated a weighted combination of two loss functions, namely the Dice loss and Lovász loss functions, into our network. We evaluated the proposed framework and several other state-of-the-art methods on a published dataset of approximately 900 manually quantified cultured mouse neurons. Our results indicate a close correlation between the proposed method and manual quantification in terms of neuron length and the number of branches while demonstrating improved analysis speed. Furthermore, the proposed method achieved high accuracy in neuron segmentation, as evidenced by the evaluation of the neurons' length and number of branches.

A modular framework for multi-scale tissue imaging and neuronal segmentation

The development of robust tools for segmenting cellular and sub-cellular neuronal structures lags behind the massive production of high-resolution 3D images of neurons in brain tissue. The challenges are principally related to high neuronal density and low signal-to-noise characteristics in thick samples, as well as the heterogeneity of data acquired with different imaging methods. To address this issue, we design a framework which includes sample preparation for high resolution imaging and image analysis. Specifically, we set up a method for labeling thick samples and develop SENPAI, a scalable algorithm for segmenting neurons at cellular and sub-cellular scales in conventional and super-resolution STimulated Emission Depletion (STED) microscopy images of brain tissues. Further, we propose a validation paradigm for testing segmentation performance when a manual ground-truth may not exhaustively describe neuronal arborization. We show that SENPAI provides accurate multi-scale segmentation, from entire neurons down to spines, outperforming state-of-the-art tools. The framework will empower image processing of complex neuronal circuitries.

From pixels to connections: exploring in vitro neuron reconstruction software for network graph generation

Digital reconstruction has been instrumental in deciphering how in vitro neuron architecture shapes information flow. Emerging approaches reconstruct neural systems as networks with the aim of understanding their organization through graph theory. Computational tools dedicated to this objective build models of nodes and edges based on key cellular features such as somata, axons, and dendrites. Fully automatic implementations of these tools are readily available, but they may also be purpose-built from specialized algorithms in the form of multi-step pipelines. Here we review software tools informing the construction of network models, spanning from noise reduction and segmentation to full network reconstruction. The scope and core specifications of each tool are explicitly defined to assist bench scientists in selecting the most suitable option for their microscopy dataset. Existing tools provide a foundation for complete network reconstruction, however more progress is needed in establishing morphological bases for directed/weighted connectivity and in software validation.

NIEND: neuronal image enhancement through noise disentanglement

MOTIVATION

The full automation of digital neuronal reconstruction from light microscopic images has long been impeded by noisy neuronal images. Previous endeavors to improve image quality can hardly get a good compromise between robustness and computational efficiency.

RESULTS

We present the image enhancement pipeline named Neuronal Image Enhancement through Noise Disentanglement (NIEND). Through extensive benchmarking on 863 mouse neuronal images with manually annotated gold standards, NIEND achieves remarkable improvements in image quality such as signal-background contrast (40-fold) and background uniformity (10-fold), compared to raw images. Furthermore, automatic reconstructions on NIEND-enhanced images have shown significant improvements compared to both raw images and images enhanced using other methods. Specifically, the average F1 score of NIEND-enhanced reconstructions is 0.88, surpassing the original 0.78 and the second-ranking method, which achieved 0.84. Up to 52% of reconstructions from NIEND-enhanced images outperform all other four methods in F1 scores. In addition, NIEND requires only 1.6 s on average for processing 256 × 256 × 256-sized images, and images after NIEND attain a substantial average compression rate of 1% by LZMA. NIEND improves image quality and neuron reconstruction, providing potential for significant advancements in automated neuron morphology reconstruction of petascale.

AVAILABILITY AND IMPLEMENTATION

The study is conducted based on Vaa3D and Python 3.10. Vaa3D is available on GitHub (https://github.com/Vaa3D). The proposed NIEND method is implemented in Python, and hosted on GitHub along with the testing code and data (https://github.com/zzhmark/NIEND). The raw neuronal images of mouse brains can be found at the BICCN's Brain Image Library (BIL) (https://www.brainimagelibrary.org). The detailed list and associated meta information are summarized in Supplementary Table S3.

Accelerating the continuous community sharing of digital neuromorphology data

The tree-like morphology of neurons and glia is a key cellular determinant of circuit connectivity and metabolic function in the nervous system of essentially all animals. To elucidate the contribution of specific cell types to both physiological and pathological brain states, it is important to access detailed neuroanatomy data for quantitative analysis and computational modeling. NeuroMorpho.Org is the largest online collection of freely available digital neural reconstructions and related metadata and is continuously updated with new uploads. Earlier in the project, we released multiple datasets together yearly, but this process caused an average delay of several months in making the data public. Moreover, in the past 5 years, >80% of invited authors agreed to share their data with the community via NeuroMorpho.Org, up from <20% in the first 5 years of the project. In the same period, the average number of reconstructions per publication increased 600%, creating the need for automatic processing to release more reconstructions in less time. The progressive automation of our pipeline enabled the transition to agile releases of individual datasets as soon as they are ready. The overall time from data identification to public sharing decreased by 63.7%; 78% of the datasets are now released in less than 3 months with an average workflow duration below 40 days. Furthermore, the mean processing time per reconstruction dropped from 3 hours to 2 minutes. With these continuous improvements, NeuroMorpho.Org strives to forge a positive culture of open data. Most importantly, the new, original research enabled through reuse of datasets across the world has a multiplicative effect on science discovery, benefiting both authors and users.

Spot Spine, a freely available ImageJ plugin for 3D detection and morphological analysis of dendritic spines

Background

Dendritic spines are tiny protrusions found along the dendrites of neurons, and their number is a measure of the density of synaptic connections. Altered density and morphology is observed in several pathologies, and spine formation as well as morphological changes correlate with learning and memory. The detection of spines in microscopy images and the analysis of their morphology is therefore a prerequisite for many studies. We have developed a new open-source, freely available, plugin for ImageJ/FIJI, called Spot Spine, that allows detection and morphological measurements of spines in three dimensional images.

Method

Local maxima are detected in spine heads, and the intensity distribution around the local maximum is computed to perform the segmentation of each spine head. Spine necks are then traced from the spine head to the dendrite. Several parameters can be set to optimize detection and segmentation, and manual correction gives further control over the result of the process.

Results

The plugin allows the analysis of images of dendrites obtained with various labeling and imaging methods. Quantitative measurements are retrieved including spine head volume and surface, and neck length.

Conclusion

The plugin and instructions for use are available at https://imagej.net/plugins/spot-spine.

Between neurons and networks: investigating mesoscale brain connectivity in neurological and psychiatric disorders

The study of brain connectivity has been a cornerstone in understanding the complexities of neurological and psychiatric disorders. It has provided invaluable insights into the functional architecture of the brain and how it is perturbed in disorders. However, a persistent challenge has been achieving the proper spatial resolution, and developing computational algorithms to address biological questions at the multi-cellular level, a scale often referred to as the mesoscale. Historically, neuroimaging studies of brain connectivity have predominantly focused on the macroscale, providing insights into inter-regional brain connections but often falling short of resolving the intricacies of neural circuitry at the cellular or mesoscale level. This limitation has hindered our ability to fully comprehend the underlying mechanisms of neurological and psychiatric disorders and to develop targeted interventions. In light of this issue, our review manuscript seeks to bridge this critical gap by delving into the domain of mesoscale neuroimaging. We aim to provide a comprehensive overview of conditions affected by aberrant neural connections, image acquisition techniques, feature extraction, and data analysis methods that are specifically tailored to the mesoscale. We further delineate the potential of brain connectivity research to elucidate complex biological questions, with a particular focus on schizophrenia and epilepsy. This review encompasses topics such as dendritic spine quantification, single neuron morphology, and brain region connectivity. We aim to showcase the applicability and significance of mesoscale neuroimaging techniques in the field of neuroscience, highlighting their potential for gaining insights into the complexities of neurological and psychiatric disorders.

Deep learning in mesoscale brain image analysis: A review

Mesoscale microscopy images of the brain contain a wealth of information which can help us understand the working mechanisms of the brain. However, it is a challenging task to process and analyze these data because of the large size of the images, their high noise levels, the complex morphology of the brain from the cellular to the regional and anatomical levels, the inhomogeneous distribution of fluorescent labels in the cells and tissues, and imaging artifacts. Due to their impressive ability to extract relevant information from images, deep learning algorithms are widely applied to microscopy images of the brain to address these challenges and they perform superiorly in a wide range of microscopy image processing and analysis tasks. This article reviews the applications of deep learning algorithms in brain mesoscale microscopy image processing and analysis, including image synthesis, image segmentation, object detection, and neuron reconstruction and analysis. We also discuss the difficulties of each task and possible directions for further research.

Bibliometric analysis of neuroscience publications quantifies the impact of data sharing

SUMMARY

Neural morphology, the branching geometry of brain cells, is an essential cellular substrate of nervous system function and pathology. Despite the accelerating production of digital reconstructions of neural morphology, the public accessibility of data remains a core issue in neuroscience. Deficiencies in the availability of existing data create redundancy of research efforts and limit synergy. We carried out a comprehensive bibliometric analysis of neural morphology publications to quantify the impact of data sharing in the neuroscience community. Our findings demonstrate that sharing digital reconstructions of neural morphology via NeuroMorpho.Org leads to a significant increase of citations to the original article, thus directly benefiting authors. The rate of data reusage remains constant for at least 16 years after sharing (the whole period analyzed), altogether nearly doubling the peer-reviewed discoveries in the field. Furthermore, the recent availability of larger and more numerous datasets fostered integrative applications, which accrue on average twice the citations of re-analyses of individual datasets. We also released an open-source citation tracking web-service allowing researchers to monitor reusage of their datasets in independent peer-reviewed reports. These results and tools can facilitate the recognition of shared data reuse for merit evaluations and funding decisions.

AVAILABILITY AND IMPLEMENTATION

The application is available at: http://cng-nmo-dev3.orc.gmu.edu:8181/. The source code at https://github.com/HerveEmissah/nmo-authors-app and https://github.com/HerveEmissah/nmo-bibliometric-analysis.

Online conversion of reconstructed neural morphologies into standardized SWC format

Digital reconstructions provide an accurate and reliable way to store, share, model, quantify, and analyze neural morphology. Continuous advances in cellular labeling, tissue processing, microscopic imaging, and automated tracing catalyzed a proliferation of software applications to reconstruct neural morphology. These computer programs typically encode the data in custom file formats. The resulting format heterogeneity severely hampers the interoperability and reusability of these valuable data. Among these many alternatives, the SWC file format has emerged as a popular community choice, coalescing a rich ecosystem of related neuroinformatics resources for tracing, visualization, analysis, and simulation. This report presents a standardized specification of the SWC file format. In addition, we introduce xyz2swc, a free online service that converts all 26 reconstruction formats (and 72 variations) described in the scientific literature into the SWC standard. The xyz2swc service is available open source through a user-friendly browser interface ( https://neuromorpho.org/xyz2swc/ui/ ) and an Application Programming Interface (API).

Combinatorial quantification of distinct neural projections from retrograde tracing

Comprehensive quantification of neuronal architectures underlying anatomical brain connectivity remains challenging. We introduce a method to identify distinct axonal projection patterns from a source to a set of target regions and the count of neurons with each pattern. A source region projecting to n targets could have 2n-1 theoretically possible projection types, although only a subset of these types typically exists. By injecting uniquely labeled retrograde tracers in k target regions (k < n), one can experimentally count the cells expressing different color combinations in the source region. The neuronal counts for different color combinations from n-choose-k experiments provide constraints for a model that is robustly solvable using evolutionary algorithms. Here, we demonstrate this method's reliability for 4 targets using simulated triple injection experiments. Furthermore, we illustrate the experimental application of this framework by quantifying the projections of male mouse primary motor cortex to the primary and secondary somatosensory and motor cortices.

Bibliometric analysis of neuroscience publications quantifies the impact of data sharing

Motivation

Neural morphology, the branching geometry of neurons and glia in the nervous system, is an essential cellular substrate of brain function and pathology. Despite the accelerating production of digital reconstructions of neural morphology in laboratories worldwide, the public accessibility of data remains a core issue in neuroscience. Deficiencies in the availability of existing data create redundancy of research efforts and prevent researchers from building on others' work. Data sharing complements the development of computational resources and literature mining tools to accelerate scientific discovery.

Results

We carried out a comprehensive bibliometric analysis of neural morphology publications to quantify the impact of data sharing in the neuroscience community. Our findings demonstrate that sharing digital reconstructions of neural morphology via the NeuroMorpho.Org online repository leads to a significant increase of citations to the original article, thus directly benefiting the authors. Moreover, the rate of data reusage remains constant for at least 16 years after sharing (the whole period analyzed), altogether nearly doubling the peer-reviewed discoveries in the field. Furthermore, the recent availability of larger and more numerous datasets fostered integrative meta-analysis applications, which accrue on average twice the citations of re-analyses of individual datasets. We also designed and deployed an open-source citation tracking web-service that allows researchers to monitor reusage of their datasets in independent peer-reviewed reports. These results and the released tool can facilitate the recognition of shared data reuse for promotion and tenure considerations, merit evaluations, and funding decisions.

Automatic identification of scientific publications describing digital reconstructions of neural morphology

The increasing number of peer-reviewed publications constitutes a challenge for biocuration. For example, NeuroMorpho.Org, a sharing platform for digital reconstructions of neural morphology, must evaluate more than 6000 potentially relevant articles per year to identify data of interest. Here, we describe a tool that uses natural language processing and deep learning to assess the likelihood of a publication to be relevant for the project. The tool automatically identifies articles describing digitally reconstructed neural morphologies with high accuracy. Its processing rate of 900 publications per hour is not only amply sufficient to autonomously track new research, but also allowed the successful evaluation of older publications backlogged due to limited human resources. The number of bio-entities found since launching the tool almost doubled while greatly reducing manual labor. The classification tool is open source, configurable, and simple to use, making it extensible to other biocuration projects.

Improved Workflow for Analysis of Vascular Myocyte Time-Series and Line-Scan Ca2+ Imaging Datasets

Intracellular Ca2+ signals are key for the regulation of cellular processes ranging from myocyte contraction, hormonal secretion, neural transmission, cellular metabolism, transcriptional regulation, and cell proliferation. Measurement of cellular Ca2+ is routinely performed using fluorescence microscopy with biological indicators. Analysis of deterministic signals is reasonably straightforward as relevant data can be discriminated based on the timing of cellular responses. However, analysis of stochastic, slower oscillatory events, as well as rapid subcellular Ca2+ responses, takes considerable time and effort which often includes visual analysis by trained investigators, especially when studying signals arising from cells embedded in complex tissues. The purpose of the current study was to determine if full-frame time-series and line-scan image analysis workflow of Fluo-4 generated Ca2+ fluorescence data from vascular myocytes could be automated without introducing errors. This evaluation was addressed by re-analyzing a published "gold standard" full-frame time-series dataset through visual analysis of Ca2+ signals from recordings made in pulmonary arterial myocytes of en face arterial preparations. We applied a combination of data driven and statistical approaches with comparisons to our published data to assess the fidelity of the various approaches. Regions of interest with Ca2+ oscillations were detected automatically post hoc using the LCPro plug-in for ImageJ. Oscillatory signals were separated based on event durations between 4 and 40 s. These data were filtered based on cutoffs obtained from multiple methods and compared to the published manually curated "gold standard" dataset. Subcellular focal and rapid Ca2+ "spark" events from line-scan recordings were examined using SparkLab 5.8, which is a custom automated detection and analysis program. After filtering, the number of true positives, false positives, and false negatives were calculated through comparisons to visually derived "gold standard" datasets. Positive predictive value, sensitivity, and false discovery rates were calculated. There were very few significant differences between the automated and manually curated results with respect to quality of the oscillatory and Ca2+ spark events, and there were no systematic biases in the data curation or filtering techniques. The lack of statistical difference in event quality between manual data curation and statistically derived critical cutoff techniques leads us to believe that automated analysis techniques can be reliably used to analyze spatial and temporal aspects to Ca2+ imaging data, which will improve experiment workflow.

NRRS: a re-tracing strategy to refine neuron reconstruction

It is crucial to develop accurate and reliable algorithms for fine reconstruction of neural morphology from whole-brain image datasets. Even though the involvement of human experts in the reconstruction process can help to ensure the quality and accuracy of the reconstructions, automated refinement algorithms are necessary to handle substantial deviations problems of reconstructed branches and bifurcation points from the large-scale and high-dimensional nature of the image data. Our proposed Neuron Reconstruction Refinement Strategy (NRRS) is a novel approach to address the problem of deviation errors in neuron morphology reconstruction. Our method partitions the reconstruction into fixed-size segments and resolves the deviation problems by re-tracing in two steps. We also validate the performance of our method using a synthetic dataset. Our results show that NRRS outperforms existing solutions and can handle most deviation errors. We apply our method to SEU-ALLEN/BICCN dataset containing 1741 complete neuron reconstructions and achieve remarkable improvements in the accuracy of the neuron skeleton representation, the task of radius estimation and axonal bouton detection. Our findings demonstrate the critical role of NRRS in refining neuron morphology reconstruction.

Availability and implementation

The proposed refinement method is implemented as a Vaa3D plugin and the source code are available under the repository of vaa3d_tools/hackathon/Levy/refinement. The original fMOST images of mouse brains can be found at the BICCN's Brain Image Library (BIL) (https://www.brainimagelibrary.org). The synthetic dataset is hosted on GitHub (https://github.com/Vaa3D/vaa3d_tools/tree/master/hackathon/Levy/refinement).

Supplementary information

Supplementary data are available at Bioinformatics Advances online.