Rapid reconstruction of 3D neuronal morphology from light microscopy images with augmented rayburst sampling

Brain Image Segmentation for Ultrascale Neuron Reconstruction via an Adaptive Dual-Task Learning Network

Accurate morphological reconstruction of neurons in whole brain images is critical for brain science research. However, due to the wide range of whole brain imaging, uneven staining, and optical system fluctuations, there are significant differences in image properties between different regions of the ultrascale brain image, such as dramatically varying voxel intensities and inhomogeneous distribution of background noise, posing an enormous challenge to neuron reconstruction from whole brain images. In this paper, we propose an adaptive dual-task learning network (ADTL-Net) to quickly and accurately extract neuronal structures from ultrascale brain images. Specifically, this framework includes an External Features Classifier (EFC) and a Parameter Adaptive Segmentation Decoder (PASD), which share the same Multi-Scale Feature Encoder (MSFE). MSFE introduces an attention module named Channel Space Fusion Module (CSFM) to extract structure and intensity distribution features of neurons at different scales for addressing the problem of anisotropy in 3D space. Then, EFC is designed to classify these feature maps based on external features, such as foreground intensity distributions and image smoothness, and select specific PASD parameters to decode them of different classes to obtain accurate segmentation results. PASD contains multiple sets of parameters trained by different representative complex signal-to-noise distribution image blocks to handle various images more robustly. Experimental results prove that compared with other advanced segmentation methods for neuron reconstruction, the proposed method achieves state-of-the-art results in the task of neuron reconstruction from ultrascale brain images, with an improvement of about 49% in speed and 12% in F1 score.

Connecto-informatics at the mesoscale: current advances in image processing and analysis for mapping the brain connectivity

Mapping neural connections within the brain has been a fundamental goal in neuroscience to understand better its functions and changes that follow aging and diseases. Developments in imaging technology, such as microscopy and labeling tools, have allowed researchers to visualize this connectivity through high-resolution brain-wide imaging. With this, image processing and analysis have become more crucial. However, despite the wealth of neural images generated, access to an integrated image processing and analysis pipeline to process these data is challenging due to scattered information on available tools and methods. To map the neural connections, registration to atlases and feature extraction through segmentation and signal detection are necessary. In this review, our goal is to provide an updated overview of recent advances in these image-processing methods, with a particular focus on fluorescent images of the mouse brain. Our goal is to outline a pathway toward an integrated image-processing pipeline tailored for connecto-informatics. An integrated workflow of these image processing will facilitate researchers' approach to mapping brain connectivity to better understand complex brain networks and their underlying brain functions. By highlighting the image-processing tools available for fluroscent imaging of the mouse brain, this review will contribute to a deeper grasp of connecto-informatics, paving the way for better comprehension of brain connectivity and its implications.

Complete Neuron Reconstruction Based on Branch Confidence

In the past few years, significant advancements in microscopic imaging technology have led to the production of numerous high-resolution images capturing brain neurons at the micrometer scale. The reconstructed structure of neurons from neuronal images can serve as a valuable reference for research in brain diseases and neuroscience. Currently, there lacks an accurate and efficient method for neuron reconstruction. Manual reconstruction remains the primary approach, offering high accuracy but requiring significant time investment. While some automatic reconstruction methods are faster, they often sacrifice accuracy and cannot be directly relied upon. Therefore, the primary goal of this paper is to develop a neuron reconstruction tool that is both efficient and accurate. The tool aids users in reconstructing complete neurons by calculating the confidence of branches during the reconstruction process. The method models the neuron reconstruction as multiple Markov chains, and calculates the confidence of the connections between branches by simulating the reconstruction artifacts in the results. Users iteratively modify low-confidence branches to ensure precise and efficient neuron reconstruction. Experiments on both the publicly accessible BigNeuron dataset and a self-created Whole-Brain dataset demonstrate that the tool achieves high accuracy similar to manual reconstruction, while significantly reducing reconstruction time.

Deep learning in mesoscale brain image analysis: A review

Mesoscale microscopy images of the brain contain a wealth of information which can help us understand the working mechanisms of the brain. However, it is a challenging task to process and analyze these data because of the large size of the images, their high noise levels, the complex morphology of the brain from the cellular to the regional and anatomical levels, the inhomogeneous distribution of fluorescent labels in the cells and tissues, and imaging artifacts. Due to their impressive ability to extract relevant information from images, deep learning algorithms are widely applied to microscopy images of the brain to address these challenges and they perform superiorly in a wide range of microscopy image processing and analysis tasks. This article reviews the applications of deep learning algorithms in brain mesoscale microscopy image processing and analysis, including image synthesis, image segmentation, object detection, and neuron reconstruction and analysis. We also discuss the difficulties of each task and possible directions for further research.

Deep self-learning enables fast, high-fidelity isotropic resolution restoration for volumetric fluorescence microscopy

One intrinsic yet critical issue that troubles the field of fluorescence microscopy ever since its introduction is the unmatched resolution in the lateral and axial directions (i.e., resolution anisotropy), which severely deteriorates the quality, reconstruction, and analysis of 3D volume images. By leveraging the natural anisotropy, we present a deep self-learning method termed Self-Net that significantly improves the resolution of axial images by using the lateral images from the same raw dataset as rational targets. By incorporating unsupervised learning for realistic anisotropic degradation and supervised learning for high-fidelity isotropic recovery, our method can effectively suppress the hallucination with substantially enhanced image quality compared to previously reported methods. In the experiments, we show that Self-Net can reconstruct high-fidelity isotropic 3D images from organelle to tissue levels via raw images from various microscopy platforms, e.g., wide-field, laser-scanning, or super-resolution microscopy. For the first time, Self-Net enables isotropic whole-brain imaging at a voxel resolution of 0.2 × 0.2 × 0.2 μm3, which addresses the last-mile problem of data quality in single-neuron morphology visualization and reconstruction with minimal effort and cost. Overall, Self-Net is a promising approach to overcoming the inherent resolution anisotropy for all classes of 3D fluorescence microscopy.

3D Soma Detection in Large-Scale Whole Brain Images via a Two-Stage Neural Network

3D soma detection in whole brain images is a critical step for neuron reconstruction. However, existing soma detection methods are not suitable for whole mouse brain images with large amounts of data and complex structure. In this paper, we propose a two-stage deep neural network to achieve fast and accurate soma detection in large-scale and high-resolution whole mouse brain images (more than 1TB). For the first stage, a lightweight Multi-level Cross Classification Network (MCC-Net) is proposed to filter out images without somas and generate coarse candidate images by combining the advantages of the multi convolution layer's feature extraction ability. It can speed up the detection of somas and reduce the computational complexity. For the second stage, to further obtain the accurate locations of somas in the whole mouse brain images, the Scale Fusion Segmentation Network (SFS-Net) is developed to segment soma regions from candidate images. Specifically, the SFS-Net captures multi-scale context information and establishes a complementary relationship between encoder and decoder by combining the encoder-decoder structure and a 3D Scale-Aware Pyramid Fusion (SAPF) module for better segmentation performance. The experimental results on three whole mouse brain images verify that the proposed method can achieve excellent performance and provide the reconstruction of neurons with beneficial information. Additionally, we have established a public dataset named WBMSD, including 798 high-resolution and representative images ( 256 ×256 ×256 voxels) from three whole mouse brain images, dedicated to the research of soma detection, which will be released along with this paper.

Neuron tracing from light microscopy images: automation, deep learning and bench testing

MOTIVATION

Large-scale neuronal morphologies are essential to neuronal typing, connectivity characterization and brain modeling. It is widely accepted that automation is critical to the production of neuronal morphology. Despite previous survey papers about neuron tracing from light microscopy data in the last decade, thanks to the rapid development of the field, there is a need to update recent progress in a review focusing on new methods and remarkable applications.

RESULTS

This review outlines neuron tracing in various scenarios with the goal to help the community understand and navigate tools and resources. We describe the status, examples and accessibility of automatic neuron tracing. We survey recent advances of the increasingly popular deep-learning enhanced methods. We highlight the semi-automatic methods for single neuron tracing of mammalian whole brains as well as the resulting datasets, each containing thousands of full neuron morphologies. Finally, we exemplify the commonly used datasets and metrics for neuron tracing bench testing.

Deep-Learning-Based Automated Neuron Reconstruction From 3D Microscopy Images Using Synthetic Training Images

Digital reconstruction of neuronal structures from 3D microscopy images is critical for the quantitative investigation of brain circuits and functions. It is a challenging task that would greatly benefit from automatic neuron reconstruction methods. In this paper, we propose a novel method called SPE-DNR that combines spherical-patches extraction (SPE) and deep-learning for neuron reconstruction (DNR). Based on 2D Convolutional Neural Networks (CNNs) and the intensity distribution features extracted by SPE, it determines the tracing directions and classifies voxels into foreground or background. This way, starting from a set of seed points, it automatically traces the neurite centerlines and determines when to stop tracing. To avoid errors caused by imperfect manual reconstructions, we develop an image synthesizing scheme to generate synthetic training images with exact reconstructions. This scheme simulates 3D microscopy imaging conditions as well as structural defects, such as gaps and abrupt radii changes, to improve the visual realism of the synthetic images. To demonstrate the applicability and generalizability of SPE-DNR, we test it on 67 real 3D neuron microscopy images from three datasets. The experimental results show that the proposed SPE-DNR method is robust and competitive compared with other state-of-the-art neuron reconstruction methods.

Structure-Guided Segmentation for 3D Neuron Reconstruction

Digital reconstruction of neuronal morphologies in 3D microscopy images is critical in the field of neuroscience. However, most existing automatic tracing algorithms cannot obtain accurate neuron reconstruction when processing 3D neuron images contaminated by strong background noises or containing weak filament signals. In this paper, we present a 3D neuron segmentation network named Structure-Guided Segmentation Network (SGSNet) to enhance weak neuronal structures and remove background noises. The network contains a shared encoding path but utilizes two decoding paths called Main Segmentation Branch (MSB) and Structure-Detection Branch (SDB), respectively. MSB is trained on binary labels to acquire the 3D neuron image segmentation maps. However, the segmentation results in challenging datasets often contain structural errors, such as discontinued segments of the weak-signal neuronal structures and missing filaments due to low signal-to-noise ratio (SNR). Therefore, SDB is presented to detect the neuronal structures by regressing neuron distance transform maps. Furthermore, a Structure Attention Module (SAM) is designed to integrate the multi-scale feature maps of the two decoding paths, and provide contextual guidance of structural features from SDB to MSB to improve the final segmentation performance. In the experiments, we evaluate our model in two challenging 3D neuron image datasets, the BigNeuron dataset and the Extended Whole Mouse Brain Sub-image (EWMBS) dataset. When using different tracing methods on the segmented images produced by our method rather than other state-of-the-art segmentation methods, the distance scores gain 42.48% and 35.83% improvement in the BigNeuron dataset and 37.75% and 23.13% in the EWMBS dataset.

Imaging functional neuroplasticity in human white matter tracts

Magnetic resonance imaging (MRI) studies are sensitive to biological mechanisms of neuroplasticity in white matter (WM). In particular, diffusion tensor imaging (DTI) has been used to investigate structural changes. Historically, functional MRI (fMRI) neuroplasticity studies have been restricted to gray matter, as fMRI studies have only recently expanded to WM. The current study evaluated WM neuroplasticity pre-post motor training in healthy adults, focusing on motor learning in the non-dominant hand. Neuroplasticity changes were evaluated in two established WM regions-of-interest: the internal capsule and the corpus callosum. Behavioral improvements following training were greater for the non-dominant hand, which corresponded with MRI-based neuroplasticity changes in the internal capsule for DTI fractional anisotropy, fMRI hemodynamic response functions, and low-frequency oscillations (LFOs). In the corpus callosum, MRI-based neuroplasticity changes were detected in LFOs, DTI, and functional correlation tensors (FCT). Taken together, the LFO results converged as significant amplitude reductions, implicating a common underlying mechanism of optimized transmission through altered myelination. The structural and functional neuroplasticity findings open new avenues for direct WM investigations into mapping connectomes and advancing MRI clinical applications.

DeepRayburst for Automatic Shape Analysis of Tree-Like Structures in Biomedical Images

Precise quantification of tree-like structures from biomedical images, such as neuronal shape reconstruction and retinal blood vessel caliber estimation, is increasingly important in understanding normal function and pathologic processes in biology. Some handcrafted methods have been proposed for this purpose in recent years. However, they are designed only for a specific application. In this paper, we propose a shape analysis algorithm, DeepRayburst, that can be applied to many different applications based on a Multi-Feature Rayburst Sampling (MFRS) and a Dual Channel Temporal Convolutional Network (DC-TCN). Specifically, we first generate a Rayburst Sampling (RS) core containing a set of multidirectional rays. Then the MFRS is designed by extending each ray of the RS to multiple parallel rays which extract a set of feature sequences. A Gaussian kernel is then used to fuse these feature sequences and outputs one feature sequence. Furthermore, we design a DC-TCN to make the rays terminate on the surface of tree-like structures according to the fused feature sequence. Finally, by analyzing the distribution patterns of the terminated rays, the algorithm can serve multiple shape analysis applications of tree-like structures. Experiments on three different applications, including soma shape reconstruction, neuronal shape reconstruction, and vessel caliber estimation, confirm that the proposed method outperforms other state-of-the-art shape analysis methods, which demonstrate its flexibility and robustness.

Exploring highly reliable substructures in auto-reconstructions of a neuron

The digital reconstruction of a neuron is the most direct and effective way to investigate its morphology. Many automatic neuron tracing methods have been proposed, but without manual check it is difficult to know whether a reconstruction or which substructure in a reconstruction is accurate. For a neuron's reconstructions generated by multiple automatic tracing methods with different principles or models, their common substructures are highly reliable and named individual motifs. In this work, we propose a Vaa3D-based method called Lamotif to explore individual motifs in automatic reconstructions of a neuron. Lamotif utilizes the local alignment algorithm in BlastNeuron to extract local alignment pairs between a specified objective reconstruction and multiple reference reconstructions, and combines these pairs to generate individual motifs on the objective reconstruction. The proposed Lamotif is evaluated on reconstructions of 163 multiple species neurons, which are generated by four state-of-the-art tracing methods. Experimental results show that individual motifs are almost on corresponding gold standard reconstructions and have much higher precision rate than objective reconstructions themselves. Furthermore, an objective reconstruction is mostly quite accurate if its individual motifs have high recall rate. Individual motifs contain common geometry substructures in multiple reconstructions, and can be used to select some accurate substructures from a reconstruction or some accurate reconstructions from automatic reconstruction dataset of different neurons.

Preclinical Western Blot in the Era of Digital Transformation and Reproducible Research, an Eastern Perspective

The current research is an interdisciplinary endeavor to develop a necessary tool in preclinical protein studies of diseases or disorders through western blotting. In the era of digital transformation and open access principles, an interactive cloud-based database called East-West Blot ( https://rancs-lab.shinyapps.io/WesternBlots ) is designed and developed. The online interactive subject-specific database built on the R shiny platform facilitates a systematic literature search on the specific subject matter, here set to western blot studies of protein regulation in the preclinical model of TBI. The tool summarizes the existing publicly available knowledge through a data visualization technique and easy access to the critical data elements and links to the study itself. The application compiled a relational database of PubMed-indexed western blot studies labeled under HHS public access, reporting downstream protein regulations presented by fluid percussion injury model of traumatic brain injury. The promises of the developed tool include progressing toward implementing the principles of 3Rs (replacement, reduction, and refinement) for humane experiments, cultivating the prerequisites of reproducible research in terms of reporting characteristics, paving the ways for a more collaborative experimental design in basic science, and rendering an up-to-date and summarized perspective of current publicly available knowledge.

Small dendritic synapses enhance temporal coding in a model of cochlear nucleus bushy cells

Spherical bushy cells (SBCs) in the anteroventral cochlear nucleus receive a single or very few powerful axosomatic inputs from the auditory nerve. However, SBCs are also contacted by small regular bouton synapses of the auditory nerve, located in their dendritic tree. The function of these small inputs is unknown. It was speculated that the interaction of axosomatic inputs with small dendritic inputs improved temporal precision, but direct evidence for this is missing. In a compartment model of spherical bushy cells with a stylized or realistic three-dimensional (3-D) representation of the bushy dendrite, we explored this hypothesis. Phase-locked dendritic inputs caused both tonic depolarization and a modulation of the model SBC membrane potential at the frequency of the stimulus. For plausible model parameters, dendritic inputs were subthreshold. Instead, the tonic depolarization increased the excitability of the SBC model and the modulation of the membrane potential caused a phase-dependent increase in the efficacy of the main axosomatic input. This improved response rate and entrainment for low-input frequencies and temporal precision of output at and above the characteristic frequency. A careful exploration of morphological and biophysical parameters of the bushy dendrite suggested a functional explanation for the peculiar shape of the bushy dendrite. Our model for the first time directly implied a role for the small excitatory dendritic inputs in auditory processing: they modulate the efficacy of the main input and are thus a plausible mechanism for the improvement of temporal precision and fidelity in these central auditory neurons.NEW & NOTEWORTHY We modeled dendritic inputs from the auditory nerve that spherical bushy cells of the cochlear nucleus receive. Dendritic inputs caused both tonic depolarization and modulation of the membrane potential at the input frequency. This improved the rate, entrainment, and temporal precision of output action potentials. Our simulations suggest a role for small dendritic inputs in auditory processing: they modulate the efficacy of the main input supporting temporal precision and fidelity in these central auditory neurons.

3D Neuron Microscopy Image Segmentation via the Ray-Shooting Model and a DC-BLSTM Network

The morphology reconstruction (tracing) of neurons in 3D microscopy images is important to neuroscience research. However, this task remains very challenging because of the low signal-to-noise ratio (SNR) and the discontinued segments of neurite patterns in the images. In this paper, we present a neuronal structure segmentation method based on the ray-shooting model and the Long Short-Term Memory (LSTM)-based network to enhance the weak-signal neuronal structures and remove background noise in 3D neuron microscopy images. Specifically, the ray-shooting model is used to extract the intensity distribution features within a local region of the image. And we design a neural network based on the dual channel bidirectional LSTM (DC-BLSTM) to detect the foreground voxels according to the voxel-intensity features and boundary-response features extracted by multiple ray-shooting models that are generated in the whole image. This way, we transform the 3D image segmentation task into multiple 1D ray/sequence segmentation tasks, which makes it much easier to label the training samples than many existing Convolutional Neural Network (CNN) based 3D neuron image segmentation methods. In the experiments, we evaluate the performance of our method on the challenging 3D neuron images from two datasets, the BigNeuron dataset and the Whole Mouse Brain Sub-image (WMBS) dataset. Compared with the neuron tracing results on the segmented images produced by other state-of-the-art neuron segmentation methods, our method improves the distance scores by about 32% and 27% in the BigNeuron dataset, and about 38% and 27% in the WMBS dataset.

Neuronize v2: Bridging the Gap Between Existing Proprietary Tools to Optimize Neuroscientific Workflows

Knowledge about neuron morphology is key to understanding brain structure and function. There are a variety of software tools that are used to segment and trace the neuron morphology. However, these tools usually utilize proprietary formats. This causes interoperability problems since the information extracted with one tool cannot be used in other tools. This article aims to improve neuronal reconstruction workflows by facilitating the interoperability between two of the most commonly used software tools—Neurolucida (NL) and Imaris (Filament Tracer). The new functionality has been included in an existing tool—Neuronize—giving rise to its second version. Neuronize v2 makes it possible to automatically use the data extracted with Imaris Filament Tracer to generate a tracing with dendritic spine information that can be read directly by NL. It also includes some other new features, such as the ability to unify and/or correct inaccurately-formed meshes (i.e., dendritic spines) and to calculate new metrics. This tool greatly facilitates the process of neuronal reconstruction, bridging the gap between existing proprietary tools to optimize neuroscientific workflows.

Retrieving similar substructures on 3D neuron reconstructions

Since manual tracing is time consuming and the performance of automatic tracing is unstable, it is still a challenging task to generate accurate neuron reconstruction efficiently and effectively. One strategy is generating a reconstruction automatically and then amending its inaccurate parts manually. Aiming at finding inaccurate substructures efficiently, we propose a pipeline to retrieve similar substructures on one or more neuron reconstructions, which are very similar to a marked problematic substructure. The pipeline consists of four steps: getting a marked substructure, constructing a query substructure, generating candidate substructures and retrieving most similar substructures. The retrieval procedure was tested on 163 gold standard reconstructions provided by the BigNeuron project and a reconstruction of a mouse’s large neuron. Experimental results showed that the implementation of the proposed methods is very efficient and all retrieved substructures are very similar to the marked one in numbers of nodes and branches, and degree of curvature.

Automated Neuron Reconstruction from 3D Fluorescence Microscopy Images Using Sequential Monte Carlo Estimation

Microscopic images of neuronal cells provide essential structural information about the key constituents of the brain and form the basis of many neuroscientific studies. Computational analyses of the morphological properties of the captured neurons require first converting the structural information into digital tree-like reconstructions. Many dedicated computational methods and corresponding software tools have been and are continuously being developed with the aim to automate this step while achieving human-comparable reconstruction accuracy. This pursuit is hampered by the immense diversity and intricacy of neuronal morphologies as well as the often low quality and ambiguity of the images. Here we present a novel method we developed in an effort to improve the robustness of digital reconstruction against these complicating factors. The method is based on probabilistic filtering by sequential Monte Carlo estimation and uses prediction and update models designed specifically for tracing neuronal branches in microscopic image stacks. Moreover, it uses multiple probabilistic traces to arrive at a more robust, ensemble reconstruction. The proposed method was evaluated on fluorescence microscopy image stacks of single neurons and dense neuronal networks with expert manual annotations serving as the gold standard, as well as on synthetic images with known ground truth. The results indicate that our method performs well under varying experimental conditions and compares favorably to state-of-the-art alternative methods.

Application of mild hypothermia successfully mitigates neural injury in a 3D in-vitro model of traumatic brain injury

Therapeutic hypothermia (TH) is an attractive target for mild traumatic brain injury (mTBI) treatment, yet significant gaps in our mechanistic understanding of TH, especially at the cellular level, remain and need to be addressed for significant forward progress to be made. Using a recently-established 3D in-vitro neural hydrogel model for mTBI we investigated the efficacy of TH after compressive impact injury and established critical treatment parameters including target cooling temperature, and time windows for application and maintenance of TH. Across four temperatures evaluated (31.5, 33, 35, and 37°C), 33°C was found to be most neuroprotective after 24 and 48 hours post-injury. Assessment of TH administration onset time and duration showed that TH should be administered within 4 hours post-injury and be maintained for at least 6 hours for achieving maximum viability. Cellular imaging showed TH reduced the percentage of cells positive for caspases 3/7 and increased the expression of calpastatin, an endogenous neuroprotectant. These findings provide significant new insight into the biological parameter space that renders TH effective in mitigating the deleterious effects of cellular mTBI and provides a quantitative foundation for the future development of animal and preclinical treatment protocols.

A Smart Region-Growing Algorithm for Single-Neuron Segmentation From Confocal and 2-Photon Datasets

Accurately digitizing the brain at the micro-scale is crucial for investigating brain structure-function relationships and documenting morphological alterations due to neuropathies. Here we present a new Smart Region Growing algorithm (SmRG) for the segmentation of single neurons in their intricate 3D arrangement within the brain. Its Region Growing procedure is based on a homogeneity predicate determined by describing the pixel intensity statistics of confocal acquisitions with a mixture model, enabling an accurate reconstruction of complex 3D cellular structures from high-resolution images of neural tissue. The algorithm's outcome is a 3D matrix of logical values identifying the voxels belonging to the segmented structure, thus providing additional useful volumetric information on neurons. To highlight the algorithm's full potential, we compared its performance in terms of accuracy, reproducibility, precision and robustness of 3D neuron reconstructions based on microscopic data from different brain locations and imaging protocols against both manual and state-of-the-art reconstruction tools.

Gotta Trace 'em All: A Mini-Review on Tools and Procedures for Segmenting Single Neurons Toward Deciphering the Structural Connectome

Decoding the morphology and physical connections of all the neurons populating a brain is necessary for predicting and studying the relationships between its form and function, as well as for documenting structural abnormalities in neuropathies. Digitizing a complete and high-fidelity map of the mammalian brain at the micro-scale will allow neuroscientists to understand disease, consciousness, and ultimately what it is that makes us humans. The critical obstacle for reaching this goal is the lack of robust and accurate tools able to deal with 3D datasets representing dense-packed cells in their native arrangement within the brain. This obliges neuroscientist to manually identify the neurons populating an acquired digital image stack, a notably time-consuming procedure prone to human bias. Here we review the automatic and semi-automatic algorithms and software for neuron segmentation available in the literature, as well as the metrics purposely designed for their validation, highlighting their strengths and limitations. In this direction, we also briefly introduce the recent advances in tissue clarification that enable significant improvements in both optical access of neural tissue and image stack quality, and which could enable more efficient segmentation approaches. Finally, we discuss new methods and tools for processing tissues and acquiring images at sub-cellular scales, which will require new robust algorithms for identifying neurons and their sub-structures (e.g., spines, thin neurites). This will lead to a more detailed structural map of the brain, taking twenty-first century cellular neuroscience to the next level, i.e., the Structural Connectome.

FMST: an Automatic Neuron Tracing Method Based on Fast Marching and Minimum Spanning Tree

Neuron reconstruction is an important technique in computational neuroscience. Although there are many reconstruction algorithms, few can generate robust results. In this paper, we propose a reconstruction algorithm called fast marching spanning tree (FMST). FMST is based on a minimum spanning tree method (MST) and improve its performance in two aspects: faster implementation and no loss of small branches. The contributions of the proposed method are as follows. Firstly, the Euclidean distance weight of edges in MST is improved to be a more reasonable value, which is related to the probability of the existence of an edge. Secondly, a strategy of pruning nodes is presented, which is based on the radius of a node's inscribed ball. Thirdly, separate branches of broken neuron reconstructions can be merged into a single tree. FMST and many other state of the art reconstruction methods were implemented on two datasets: 120 Drosophila neurons and 163 neurons with gold standard reconstructions. Qualitative and quantitative analysis on experimental results demonstrates that the performance of FMST is good compared with many existing methods. Especially, on the 91 fruitfly neurons with gold standard and evaluated by five metrics, FMST is one of two methods with best performance among all 27 state of the art reconstruction methods. FMST is a good and practicable neuron reconstruction algorithm, and can be implemented in Vaa3D platform as a neuron tracing plugin.

A Multiscale Ray-Shooting Model for Termination Detection of Tree-Like Structures in Biomedical Images

Digital reconstruction (tracing) of tree-like structures, such as neurons, retinal blood vessels, and bronchi, from volumetric images and 2D images is very important to biomedical research. Many existing reconstruction algorithms rely on a set of good seed points. The 2D or 3D terminations are good candidates for such seed points. In this paper, we propose an automatic method to detect terminations for tree-like structures based on a multiscale ray-shooting model and a termination visual prior. The multiscale ray-shooting model detects 2D terminations by extracting and analyzing the multiscale intensity distribution features around a termination candidate. The range of scale is adaptively determined according to the local neurite diameter estimated by the Rayburst sampling algorithm in combination with the gray-weighted distance transform. The termination visual prior is based on a key observation-when observing a 3D termination from three orthogonal directions without occlusion, we can recognize it in at least two views. Using this prior with the multiscale ray-shooting model, we can detect 3D terminations with high accuracies. Experiments on 3D neuron image stacks, 2D neuron images, 3D bronchus image stacks, and 2D retinal blood vessel images exhibit average precision and recall rates of 87.50% and 90.54%. The experimental results confirm that the proposed method outperforms other the state-of-the-art termination detection methods.

Automated 3D Soma Segmentation with Morphological Surface Evolution for Neuron Reconstruction

The automatic neuron reconstruction is important since it accelerates the collection of 3D neuron models for the neuronal morphological studies. The majority of the previous neuron reconstruction methods only focused on tracing neuron fibres without considering the somatic surface. Thus, topological errors often present around the soma area in the results obtained by these tracing methods. Segmentation of the soma structures can be embedded in the existing neuron tracing methods to reduce such topological errors. In this paper, we present a novel method to segment the soma structures with complex geometry. It can be applied along with the existing methods in a fully automated pipeline. An approximate bounding block is firstly estimated based on a geodesic distance transform. Then the soma segmentation is obtained by evolving the surface with a set of morphological operators inside the initial bounding region. By evaluating the methods against the challenging images released by the BigNeuron project, we showed that the proposed method can outperform the existing soma segmentation methods regarding the accuracy. We also showed that the soma segmentation can be used for enhancing the results of existing neuron tracing methods.

Tracing Tubular Structures from Teravoxel-Sized Microscope Images

Tracing vasculature and neurites from teravoxel sized light-microscopy data-sets is a challenge impeding the availability of processed data to the research community. This is because (1) Holding terabytes of data during run-time is not easy for a regular PC. (2) Processing all the data at once would be slow and inefficient. In this paper, we propose a way to mitigate this challenge by Divide Conquer and Combine (DCC) method. We first split the volume into many smaller and manageable sub-volumes before tracing. These sub-volumes can then be traced individually in parallel (or otherwise). We propose an algorithm to stitch together the traced data from these sub-volumes. This algorithm is robust and handles challenging scenarios like (1) sub-optimal tracing at edges (2) densely packed structures and (3) different depths of trace termination. We validate our results using whole mouse brain vasculature data-set obtained from the Knife-Edge Scanning Microscopy (KESM) based automated tissue scanner.

Ensemble Neuron Tracer for 3D Neuron Reconstruction

Tracing of neuron paths is important in neuroscience. Recent studies have shown that it is possible to segment and reconstruct three-dimensional morphology of axons and dendrites using fully automatic neuron tracing methods. A specific tracer may be better than others for a specific dataset, but another tracer could perform better for some other datasets. Ensemble of learners is an effective way to improve learning accuracy in machine learning. We developed automatic ensemble neuron tracers, which consistently perform well on 57 datasets of 5 species collected from 7 laboratories worldwide. Quantitative evaluation based on the data generated by human annotators shows that the proposed ensemble tracers are valuable for 3D neuron tracing and can be widely applied to different datasets.

SparseTracer: the Reconstruction of Discontinuous Neuronal Morphology in Noisy Images

Digital reconstruction of a single neuron occupies an important position in computational neuroscience. Although many novel methods have been proposed, recent advances in molecular labeling and imaging systems allow for the production of large and complicated neuronal datasets, which pose many challenges for neuron reconstruction, especially when discontinuous neuronal morphology appears in a strong noise environment. Here, we develop a new pipeline to address this challenge. Our pipeline is based on two methods, one is the region-to-region connection (RRC) method for detecting the initial part of a neurite, which can effectively gather local cues, i.e., avoid the whole image analysis, and thus boosts the efficacy of computation; the other is constrained principal curves method for completing the neurite reconstruction, which uses the past reconstruction information of a neurite for current reconstruction and thus can be suitable for tracing discontinuous neurites. We investigate the reconstruction performances of our pipeline and some of the best state-of-the-art algorithms on the experimental datasets, indicating the superiority of our method in reconstructing sparsely distributed neurons with discontinuous neuronal morphologies in noisy environment. We show the strong ability of our pipeline in dealing with the large-scale image dataset. We validate the effectiveness in dealing with various kinds of image stacks including those from the DIADEM challenge and BigNeuron project.

Rivulet: 3D Neuron Morphology Tracing with Iterative Back-Tracking

The digital reconstruction of single neurons from 3D confocal microscopic images is an important tool for understanding the neuron morphology and function. However the accurate automatic neuron reconstruction remains a challenging task due to the varying image quality and the complexity in the neuronal arborisation. Targeting the common challenges of neuron tracing, we propose a novel automatic 3D neuron reconstruction algorithm, named Rivulet, which is based on the multi-stencils fast-marching and iterative back-tracking. The proposed Rivulet algorithm is capable of tracing discontinuous areas without being interrupted by densely distributed noises. By evaluating the proposed pipeline with the data provided by the Diadem challenge and the recent BigNeuron project, Rivulet is shown to be robust to challenging microscopic imagestacks. We discussed the algorithm design in technical details regarding the relationships between the proposed algorithm and the other state-of-the-art neuron tracing algorithms.

Precise Cerebral Vascular Atlas in Stereotaxic Coordinates of Whole Mouse Brain

Understanding amazingly complex brain functions and pathologies requires a complete cerebral vascular atlas in stereotaxic coordinates. Making a precise atlas for cerebral arteries and veins has been a century-old objective in neuroscience and neuropathology. Using micro-optical sectioning tomography (MOST) with a modified Nissl staining method, we acquired five mouse brain data sets containing arteries, veins, and microvessels. Based on the brain-wide vascular spatial structures and brain regions indicated by cytoarchitecture in one and the same mouse brain, we reconstructed and annotated the vascular system atlas of both arteries and veins of the whole mouse brain for the first time. The distributing patterns of the vascular system within the brain regions were acquired and our results show that the patterns of individual vessels are different from each other. Reconstruction and statistical analysis of the microvascular network, including derivation of quantitative vascular densities, indicate significant differences mainly in vessels with diameters less than 8 μm and large than 20 μm across different brain regions. Our precise cerebral vascular atlas provides an important resource and approach for quantitative studies of brain functions and diseases.

Touching Soma Segmentation Based on the Rayburst Sampling Algorithm

Neuronal soma segmentation is essential for morphology quantification analysis. Rapid advances in light microscope imaging techniques have generated such massive amounts of data that time-consuming manual methods cannot meet requirements for high throughput. However, touching soma segmentation is still a challenge for automatic segmentation methods. In this paper, we propose a soma segmentation method that combines the Rayburst sampling algorithm and ellipsoid fitting. The improved Rayburst sampling algorithm is used to detect the soma surface; the ellipsoid fitting method then refines jagged sampled soma surface to generate smooth ellipsoidal shapes for efficient analysis. In experiments, we validated the proposed method by applying it to datasets from the fluorescence micro-optical sectioning tomography (fMOST) system. The results indicate that the proposed method is comparable to the manual segmented gold standard with accurate soma segmentation at a relatively high speed. The proposed method can be extended to large-scale image stacks in the future.

TReMAP: Automatic 3D Neuron Reconstruction Based on Tracing, Reverse Mapping and Assembling of 2D Projections

Efficient and accurate digital reconstruction of neurons from large-scale 3D microscopic images remains a challenge in neuroscience. We propose a new automatic 3D neuron reconstruction algorithm, TReMAP, which utilizes 3D Virtual Finger (a reverse-mapping technique) to detect 3D neuron structures based on tracing results on 2D projection planes. Our fully automatic tracing strategy achieves close performance with the state-of-the-art neuron tracing algorithms, with the crucial advantage of efficient computation (much less memory consumption and parallel computation) for large-scale images.

BlastNeuron for Automated Comparison, Retrieval and Clustering of 3D Neuron Morphologies

Characterizing the identity and types of neurons in the brain, as well as their associated function, requires a means of quantifying and comparing 3D neuron morphology. Presently, neuron comparison methods are based on statistics from neuronal morphology such as size and number of branches, which are not fully suitable for detecting local similarities and differences in the detailed structure. We developed BlastNeuron to compare neurons in terms of their global appearance, detailed arborization patterns, and topological similarity. BlastNeuron first compares and clusters 3D neuron reconstructions based on global morphology features and moment invariants, independent of their orientations, sizes, level of reconstruction and other variations. Subsequently, BlastNeuron performs local alignment between any pair of retrieved neurons via a tree-topology driven dynamic programming method. A 3D correspondence map can thus be generated at the resolution of single reconstruction nodes. We applied BlastNeuron to three datasets: (1) 10,000+ neuron reconstructions from a public morphology database, (2) 681 newly and manually reconstructed neurons, and (3) neurons reconstructions produced using several independent reconstruction methods. Our approach was able to accurately and efficiently retrieve morphologically and functionally similar neuron structures from large morphology database, identify the local common structures, and find clusters of neurons that share similarities in both morphology and molecular profiles.

Progress Towards Mammalian Whole-Brain Cellular Connectomics

Neurons are the fundamental structural units of the nervous system-i.e., the Neuron Doctrine-as the pioneering work of Santiago Ramón y Cajal in the 1880's clearly demonstrated through careful observation of Golgi-stained neuronal morphologies. However, at that time sample preparation, imaging methods and computational tools were either nonexistent or insufficiently developed to permit the precise mapping of an entire brain with all of its neurons and their connections. Some measure of the "mesoscopic" connectional organization of the mammalian brain has been obtained over the past decade by alignment of sparse subsets of labeled neurons onto a reference atlas or via MRI-based diffusion tensor imaging. Neither method, however, provides data on the complete connectivity of all neurons comprising an individual brain. Fortunately, whole-brain cellular connectomics now appears within reach due to recent advances in whole-brain sample preparation and high-throughput electron microscopy (EM), though substantial obstacles remain with respect to large volume electron microscopic acquisitions and automated neurite reconstructions. This perspective examines the current status and problems associated with generating a mammalian whole-brain cellular connectome and argues that the time is right to launch a concerted connectomic attack on a small mammalian whole-brain.

Automatic Morphological Reconstruction of Neurons from Multiphoton and Confocal Microscopy Images Using 3D Tubular Models

The challenges faced in analyzing optical imaging data from neurons include a low signal-to-noise ratio of the acquired images and the multiscale nature of the tubular structures that range in size from hundreds of microns to hundreds of nanometers. In this paper, we address these challenges and present a computational framework for an automatic, three-dimensional (3D) morphological reconstruction of live nerve cells. The key aspects of this approach are: (i) detection of neuronal dendrites through learning 3D tubular models, and (ii) skeletonization by a new algorithm using a morphology-guided deformable model for extracting the dendritic centerline. To represent the neuron morphology, we introduce a novel representation, the Minimum Shape-Cost (MSC) Tree that approximates the dendrite centerline with sub-voxel accuracy and demonstrate the uniqueness of such a shape representation as well as its computational efficiency. We present extensive quantitative and qualitative results that demonstrate the accuracy and robustness of our method.

Adaptive and Background-Aware GAL4 Expression Enhancement of Co-registered Confocal Microscopy Images

GAL4 gene expression imaging using confocal microscopy is a common and powerful technique used to study the nervous system of a model organism such as Drosophila melanogaster. Recent research projects focused on high throughput screenings of thousands of different driver lines, resulting in large image databases. The amount of data generated makes manual assessment tedious or even impossible. The first and most important step in any automatic image processing and data extraction pipeline is to enhance areas with relevant signal. However, data acquired via high throughput imaging tends to be less then ideal for this task, often showing high amounts of background signal. Furthermore, neuronal structures and in particular thin and elongated projections with a weak staining signal are easily lost. In this paper we present a method for enhancing the relevant signal by utilizing a Hessian-based filter to augment thin and weak tube-like structures in the image. To get optimal results, we present a novel adaptive background-aware enhancement filter parametrized with the local background intensity, which is estimated based on a common background model. We also integrate recent research on adaptive image enhancement into our approach, allowing us to propose an effective solution for known problems present in confocal microscopy images. We provide an evaluation based on annotated image data and compare our results against current state-of-the-art algorithms. The results show that our algorithm clearly outperforms the existing solutions.

NeuroGPS-Tree: automatic reconstruction of large-scale neuronal populations with dense neurites

The reconstruction of neuronal populations, a key step in understanding neural circuits, remains a challenge in the presence of densely packed neurites. Here we achieved automatic reconstruction of neuronal populations by partially mimicking human strategies to separate individual neurons. For populations not resolvable by other methods, we obtained recall and precision rates of approximately 80%. We also demonstrate the reconstruction of 960 neurons within 3 h.

Whole-body tissue stabilization and selective extractions via tissue-hydrogel hybrids for high-resolution intact circuit mapping and phenotyping

To facilitate fine-scale phenotyping of whole specimens, we describe here a set of tissue fixation-embedding, detergent-clearing and staining protocols that can be used to transform excised organs and whole organisms into optically transparent samples within 1-2 weeks without compromising their cellular architecture or endogenous fluorescence. PACT (passive CLARITY technique) and PARS (perfusion-assisted agent release in situ) use tissue-hydrogel hybrids to stabilize tissue biomolecules during selective lipid extraction, resulting in enhanced clearing efficiency and sample integrity. Furthermore, the macromolecule permeability of PACT- and PARS-processed tissue hybrids supports the diffusion of immunolabels throughout intact tissue, whereas RIMS (refractive index matching solution) grants high-resolution imaging at depth by further reducing light scattering in cleared and uncleared samples alike. These methods are adaptable to difficult-to-image tissues, such as bone (PACT-deCAL), and to magnified single-cell visualization (ePACT). Together, these protocols and solutions enable phenotyping of subcellular components and tracing cellular connectivity in intact biological networks.

Neuron anatomy structure reconstruction based on a sliding filter

BACKGROUND

Reconstruction of neuron anatomy structure is a challenging and important task in neuroscience. However, few algorithms can automatically reconstruct the full structure well without manual assistance, making it essential to develop new methods for this task.

METHODS

This paper introduces a new pipeline for reconstructing neuron anatomy structure from 3-D microscopy image stacks. This pipeline is initialized with a set of seeds that were detected by our proposed Sliding Volume Filter (SVF), given a non-circular cross-section of a neuron cell. Then, an improved open curve snake model combined with a SVF external force is applied to trace the full skeleton of the neuron cell. A radius estimation method based on a 2D sliding band filter is developed to fit the real edge of the cross-section of the neuron cell. Finally, a surface reconstruction method based on non-parallel curve networks is used to generate the neuron cell surface to finish this pipeline.

RESULTS

The proposed pipeline has been evaluated using publicly available datasets. The results show that the proposed method achieves promising results in some datasets from the DIgital reconstruction of Axonal and DEndritic Morphology (DIADEM) challenge and new BigNeuron project.

CONCLUSION

The new pipeline works well in neuron tracing and reconstruction. It can achieve higher efficiency, stability and robustness in neuron skeleton tracing. Furthermore, the proposed radius estimation method and applied surface reconstruction method can obtain more accurate neuron anatomy structures.