Semi-automated neuron boundary detection and nonbranching process segmentation in electron microscopy images

Membrane mechanics dictate axonal morphology and function

Axons are thought to be ultrathin membrane cables of a relatively uniform diameter, designed to conduct electrical signals, or action potentials. Here, we demonstrate that unmyelinated axons are not simple cylindrical tubes. Rather, axons have nanoscopic boutons repeatedly along their length interspersed with a thin cable with a diameter of ∼60 nm like pearls-on-a-string. These boutons are only ∼200 nm in diameter and do not have synaptic contacts or a cluster of synaptic vesicles, hence non-synaptic. Our in silico modeling suggests that axon pearling can be explained by the mechanical properties of the membrane including the bending modulus and tension. Consistent with modeling predictions, treatments that disrupt these parameters like hyper- or hypo-tonic solutions, cholesterol removal, and non-muscle myosin II inhibition all alter the degree of axon pearling, suggesting that axon morphology is indeed determined by the membrane mechanics. Intriguingly, neuronal activity modulates the cholesterol level of plasma membrane, leading to shrinkage of axon pearls. Consequently, the conduction velocity of action potentials becomes slower. These data reveal that biophysical forces dictate axon morphology and function and that modulation of membrane mechanics likely underlies plasticity of unmyelinated axons.

Loon: Using Exemplars to Visualize Large-Scale Microscopy Data

Which drug is most promising for a cancer patient? A new microscopy-based approach for measuring the mass of individual cancer cells treated with different drugs promises to answer this question in only a few hours. However, the analysis pipeline for extracting data from these images is still far from complete automation: human intervention is necessary for quality control for preprocessing steps such as segmentation, adjusting filters, removing noise, and analyzing the result. To address this workflow, we developed Loon, a visualization tool for analyzing drug screening data based on quantitative phase microscopy imaging. Loon visualizes both derived data such as growth rates and imaging data. Since the images are collected automatically at a large scale, manual inspection of images and segmentations is infeasible. However, reviewing representative samples of cells is essential, both for quality control and for data analysis. We introduce a new approach for choosing and visualizing representative exemplar cells that retain a close connection to the low-level data. By tightly integrating the derived data visualization capabilities with the novel exemplar visualization and providing selection and filtering capabilities, Loon is well suited for making decisions about which drugs are suitable for a specific patient.

ScanEV - A neural network-based tool for the automated detection of extracellular vesicles in TEM images

Transmission electron microscopy (TEM) is the most widely accepted method for visualization of extracellular vesicles (EVs), and particularly, exosomes. TEM images provide us with information about the size and morphology of the EVs. We have developed an online tool ScanEV (Scanner for the Extracellular Vesicles, available at https://bioeng.ru/scanev), for the rapid and automated processing of such images. ScanEV is based on a convolutional neural network; it detects the «cup-shaped» particles in the images and calculates their morphometric parameters. This tool will be useful for researchers who study EVs and use TEM for their characterization.

Automated segmentation of brain cells for clonal analyses in fluorescence microscopy images

The understanding of how cell diversity within and across distinct brain regions is ontogenetically achieved is a pivotal topic in neuroscience. Clonal analyses based on multicolor cell labeling represent a powerful tool to tackle this issue and disclose lineage relationships, but produce enormous sets of fluorescence images, leading to time consuming analyses that may be biased by the operator's subjectivity. Thus, time-efficient automated software are needed to analyze images easily, accurately and without subjective bias. In this paper, we present a fully automated method, named FAST ('Fluorescent cell Analysis Segmentation Tool'), for the segmentation of neural cells labeled by multicolor combinations of fluorophores and for their classification into clones. The proposed method was tested on 77 high-magnification fluorescence images of adult mouse cerebellar tissues acquired using a confocal microscope. Automatic results were compared with manual annotations and two open-source software designed for cell detection in microscopic imaging. The algorithm showed very good performance in the cellular detection and in the assignment of the clonal identity. To the best of our knowledge, FAST is the first fully automated technique for the analysis of cellular clones based on combinatorial expression of fluorescent proteins. The proposed approach allows to perform clonal analyses easily, accurately and objectively, overcoming those biases and errors that may result from manual annotations. Moreover, it can be broadly applied to the quantification and colocalization within cells of fluorescent markers, therefore representing a versatile and powerful tool for automated quantitative analyses in fluorescence microscopy.

NeuroBlocks--Visual Tracking of Segmentation and Proofreading for Large Connectomics Projects

In the field of connectomics, neuroscientists acquire electron microscopy volumes at nanometer resolution in order to reconstruct a detailed wiring diagram of the neurons in the brain. The resulting image volumes, which often are hundreds of terabytes in size, need to be segmented to identify cell boundaries, synapses, and important cell organelles. However, the segmentation process of a single volume is very complex, time-intensive, and usually performed using a diverse set of tools and many users. To tackle the associated challenges, this paper presents NeuroBlocks, which is a novel visualization system for tracking the state, progress, and evolution of very large volumetric segmentation data in neuroscience. NeuroBlocks is a multi-user web-based application that seamlessly integrates the diverse set of tools that neuroscientists currently use for manual and semi-automatic segmentation, proofreading, visualization, and analysis. NeuroBlocks is the first system that integrates this heterogeneous tool set, providing crucial support for the management, provenance, accountability, and auditing of large-scale segmentations. We describe the design of NeuroBlocks, starting with an analysis of the domain-specific tasks, their inherent challenges, and our subsequent task abstraction and visual representation. We demonstrate the utility of our design based on two case studies that focus on different user roles and their respective requirements for performing and tracking the progress of segmentation and proofreading in a large real-world connectomics project.

An efficient conditional random field approach for automatic and interactive neuron segmentation

We present a new graphical-model-based method for automatic and interactive segmentation of neuron structures from electron microscopy (EM) images. For automated reconstruction, our learning based model selects a collection of nodes from a hierarchical merging tree as the proposed segmentation. More specifically, this is achieved by training a conditional random field (CRF) whose underlying graph is the watershed merging tree. The maximum a posteriori (MAP) prediction of the CRF is the output segmentation. Our results are comparable to the results of state-of-the-art methods. Furthermore, both the inference and the training are very efficient as the graph is tree-structured. The problem of neuron segmentation requires extremely high segmentation quality. Therefore, proofreading, namely, interactively correcting mistakes of the automatic method, is a necessary module in the pipeline. Based on our efficient tree-structured inference algorithm, we develop an interactive segmentation framework which only selects locations where the model is uncertain for a user to proofread. The uncertainty is measured by the marginals of the graphical model. Only giving a limited number of choices makes the user interaction very efficient. Based on user corrections, our framework modifies the merging tree and thus improves the segmentation globally.

VolRoverN: enhancing surface and volumetric reconstruction for realistic dynamical simulation of cellular and subcellular function

Establishing meaningful relationships between cellular structure and function requires accurate morphological reconstructions. In particular, there is an unmet need for high quality surface reconstructions to model subcellular and synaptic interactions among neurons and glia at nanometer resolution. We address this need with VolRoverN, a software package that produces accurate, efficient, and automated 3D surface reconstructions from stacked 2D contour tracings. While many techniques and tools have been developed in the past for 3D visualization of cellular structure, the reconstructions from VolRoverN meet specific quality criteria that are important for dynamical simulations. These criteria include manifoldness, water-tightness, lack of self- and object-object-intersections, and geometric accuracy. These enhanced surface reconstructions are readily extensible to any cell type and are used here on spiny dendrites with complex morphology and axons from mature rat hippocampal area CA1. Both spatially realistic surface reconstructions and reduced skeletonizations are produced and formatted by VolRoverN for easy input into analysis software packages for neurophysiological simulations at multiple spatial and temporal scales ranging from ion electro-diffusion to electrical cable models.

Design and Evaluation of Interactive Proofreading Tools for Connectomics

Proofreading refers to the manual correction of automatic segmentations of image data. In connectomics, electron microscopy data is acquired at nanometer-scale resolution and results in very large image volumes of brain tissue that require fully automatic segmentation algorithms to identify cell boundaries. However, these algorithms require hundreds of corrections per cubic micron of tissue. Even though this task is time consuming, it is fairly easy for humans to perform corrections through splitting, merging, and adjusting segments during proofreading. In this paper we present the design and implementation of Mojo, a fully-featured single-user desktop application for proofreading, and Dojo, a multi-user web-based application for collaborative proofreading. We evaluate the accuracy and speed of Mojo, Dojo, and Raveler, a proofreading tool from Janelia Farm, through a quantitative user study. We designed a between-subjects experiment and asked non-experts to proofread neurons in a publicly available connectomics dataset. Our results show a significant improvement of corrections using web-based Dojo, when given the same amount of time. In addition, all participants using Dojo reported better usability. We discuss our findings and provide an analysis of requirements for designing visual proofreading software.

Segmentation of neuronal structures using SARSA (λ)-based boundary amendment with reinforced gradient-descent curve shape fitting

The segmentation of structures in electron microscopy (EM) images is very important for neurobiological research. The low resolution neuronal EM images contain noise and generally few features are available for segmentation, therefore application of the conventional approaches to identify the neuron structure from EM images is not successful. We therefore present a multi-scale fused structure boundary detection algorithm in this study. In the algorithm, we generate an EM image Gaussian pyramid first, then at each level of the pyramid, we utilize Laplacian of Gaussian function (LoG) to attain structure boundary, we finally assemble the detected boundaries by using fusion algorithm to attain a combined neuron structure image. Since the obtained neuron structures usually have gaps, we put forward a reinforcement learning-based boundary amendment method to connect the gaps in the detected boundaries. We use a SARSA (λ)-based curve traveling and amendment approach derived from reinforcement learning to repair the incomplete curves. Using this algorithm, a moving point starts from one end of the incomplete curve and walks through the image where the decisions are supervised by the approximated curve model, with the aim of minimizing the connection cost until the gap is closed. Our approach provided stable and efficient structure segmentation. The test results using 30 EM images from ISBI 2012 indicated that both of our approaches, i.e., with or without boundary amendment, performed better than six conventional boundary detection approaches. In particular, after amendment, the Rand error and warping error, which are the most important performance measurements during structure segmentation, were reduced to very low values. The comparison with the benchmark method of ISBI 2012 and the recent developed methods also indicates that our method performs better for the accurate identification of substructures in EM images and therefore useful for the identification of imaging features related to brain diseases.

Automated detection of synapses in serial section transmission electron microscopy image stacks

We describe a method for fully automated detection of chemical synapses in serial electron microscopy images with highly anisotropic axial and lateral resolution, such as images taken on transmission electron microscopes. Our pipeline starts from classification of the pixels based on 3D pixel features, which is followed by segmentation with an Ising model MRF and another classification step, based on object-level features. Classifiers are learned on sparse user labels; a fully annotated data subvolume is not required for training. The algorithm was validated on a set of 238 synapses in 20 serial 7197×7351 pixel images (4.5×4.5×45 nm resolution) of mouse visual cortex, manually labeled by three independent human annotators and additionally re-verified by an expert neuroscientist. The error rate of the algorithm (12% false negative, 7% false positive detections) is better than state-of-the-art, even though, unlike the state-of-the-art method, our algorithm does not require a prior segmentation of the image volume into cells. The software is based on the ilastik learning and segmentation toolkit and the vigra image processing library and is freely available on our website, along with the test data and gold standard annotations (http://www.ilastik.org/synapse-detection/sstem).

Machine learning of hierarchical clustering to segment 2D and 3D images

We aim to improve segmentation through the use of machine learning tools during region agglomeration. We propose an active learning approach for performing hierarchical agglomerative segmentation from superpixels. Our method combines multiple features at all scales of the agglomerative process, works for data with an arbitrary number of dimensions, and scales to very large datasets. We advocate the use of variation of information to measure segmentation accuracy, particularly in 3D electron microscopy (EM) images of neural tissue, and using this metric demonstrate an improvement over competing algorithms in EM and natural images.

Biomedical Visual Computing: Case Studies and Challenges

Advances in computational geometric modeling, imaging, and simulation let researchers build and test models of increasing complexity, generating unprecedented amounts of data. As recent research in biomedical applications illustrates, visualization will be critical in making this vast amount of data usable; it's also fundamental to understanding models of complex phenomena.