Geometrical consistent 3D tracing of neuronal processes in ssTEM data

Large-scale automatic reconstruction of neuronal processes from electron microscopy images

Automated sample preparation and electron microscopy enables acquisition of very large image data sets. These technical advances are of special importance to the field of neuroanatomy, as 3D reconstructions of neuronal processes at the nm scale can provide new insight into the fine grained structure of the brain. Segmentation of large-scale electron microscopy data is the main bottleneck in the analysis of these data sets. In this paper we present a pipeline that provides state-of-the art reconstruction performance while scaling to data sets in the GB-TB range. First, we train a random forest classifier on interactive sparse user annotations. The classifier output is combined with an anisotropic smoothing prior in a Conditional Random Field framework to generate multiple segmentation hypotheses per image. These segmentations are then combined into geometrically consistent 3D objects by segmentation fusion. We provide qualitative and quantitative evaluation of the automatic segmentation and demonstrate large-scale 3D reconstructions of neuronal processes from a 27,000 μm(3) volume of brain tissue over a cube of 30 μm in each dimension corresponding to 1000 consecutive image sections. We also introduce Mojo, a proofreading tool including semi-automated correction of merge errors based on sparse user scribbles.

A modular hierarchical approach to 3D electron microscopy image segmentation

The study of neural circuit reconstruction, i.e., connectomics, is a challenging problem in neuroscience. Automated and semi-automated electron microscopy (EM) image analysis can be tremendously helpful for connectomics research. In this paper, we propose a fully automatic approach for intra-section segmentation and inter-section reconstruction of neurons using EM images. A hierarchical merge tree structure is built to represent multiple region hypotheses and supervised classification techniques are used to evaluate their potentials, based on which we resolve the merge tree with consistency constraints to acquire final intra-section segmentation. Then, we use a supervised learning based linking procedure for the inter-section neuron reconstruction. Also, we develop a semi-automatic method that utilizes the intermediate outputs of our automatic algorithm and achieves intra-segmentation with minimal user intervention. The experimental results show that our automatic method can achieve close-to-human intra-segmentation accuracy and state-of-the-art inter-section reconstruction accuracy. We also show that our semi-automatic method can further improve the intra-segmentation accuracy.

Using transmission electron microscopy and 3View to determine collagen fibril size and three-dimensional organization

Collagen fibrils are the major tensile element in vertebrate tissues, in which they occur as ordered bundles in the extracellular matrix. Abnormal fibril assembly and organization results in scarring, fibrosis, poor wound healing and connective tissue diseases. Transmission electron microscopy (TEM) is used to assess the formation of the fibrils, predominantly by measuring fibril diameter. Here we describe a protocol for measuring fibril diameter as well as fibril volume fraction, mean fibril length, fibril cross-sectional shape and fibril 3D organization, all of which are major determinants of tissue function. Serial-section TEM (ssTEM) has been used to visualize fibril 3D organization in vivo. However, serial block face-scanning electron microscopy (SBF-SEM) has emerged as a time-efficient alternative to ssTEM. The protocol described below is suitable for preparing tissues for TEM and SBF-SEM (by 3View). We describe how to use 3View for studying collagen fibril organization in vivo and show how to find and track individual fibrils. The overall time scale is ~8 d from isolating the tissue to having a 3D image stack.

The Open Connectome Project Data Cluster: Scalable Analysis and Vision for High-Throughput Neuroscience

We describe a scalable database cluster for the spatial analysis and annotation of high-throughput brain imaging data, initially for 3-d electron microscopy image stacks, but for time-series and multi-channel data as well. The system was designed primarily for workloads that build connectomes- neural connectivity maps of the brain-using the parallel execution of computer vision algorithms on high-performance compute clusters. These services and open-science data sets are publicly available at openconnecto.me. The system design inherits much from NoSQL scale-out and data-intensive computing architectures. We distribute data to cluster nodes by partitioning a spatial index. We direct I/O to different systems-reads to parallel disk arrays and writes to solid-state storage-to avoid I/O interference and maximize throughput. All programming interfaces are RESTful Web services, which are simple and stateless, improving scalability and usability. We include a performance evaluation of the production system, highlighting the effec-tiveness of spatial data organization.

Fiji: an open-source platform for biological-image analysis

Fiji is a distribution of the popular open-source software ImageJ focused on biological-image analysis. Fiji uses modern software engineering practices to combine powerful software libraries with a broad range of scripting languages to enable rapid prototyping of image-processing algorithms. Fiji facilitates the transformation of new algorithms into ImageJ plugins that can be shared with end users through an integrated update system. We propose Fiji as a platform for productive collaboration between computer science and biology research communities.

Scalable tracing of electron micrographs by fusing top down and bottom up cues using hypergraph diffusion

A novel framework for robust 3D tracing in electron micrographs is presented. The proposed framework is built using ideas from hypergraph diffusion, and achieves two main objectives. Firstly, the approach scales to trace hundreds of targets without noticeable increase in runtime complexity. Secondly, the framework yields flexibility to fuse top down (global cues as hyperedges) and bottom up (local superpixels as nodes) information. Subsequently, a procedure for auto-seeding to initialize the tracing procedure is proposed. The paper concludes with experimental validation on a challenging large scale tracing problem for simultaneously tracing 95 structures, illustrating applicability of the proposed algorithm.

Anisotropic ssTEM image segmentation using dense correspondence across sections

Connectomics based on high resolution ssTEM imagery requires reconstruction of the neuron geometry from histological slides. We present an approach for the automatic membrane segmentation in anisotropic stacks of electron microscopy brain tissue sections. The ambiguities in neuronal segmentation of a section are resolved by using the context from the neighboring sections. We find the global dense correspondence between the sections by SIFT Flow algorithm, evaluate the features of the corresponding pixels and use them to perform the segmentation. Our method is 3.6 and 6.4% more accurate in two different accuracy metrics than the algorithm with no context from other sections.

Multiple structure tracing in 3D electron micrographs

Automatic interpretation of Transmission Electron Micrograph (TEM) volumes is central to advancing current understanding of neural circuitry. In the context of TEM image analysis, tracing 3D neuronal structures is a significant problem. This work proposes a new model using the conditional random field (CRF) framework with higher order potentials for tracing multiple neuronal structures in 3D. The model consists of two key features. First, the higher order CRF cost is designed to enforce label smoothness in 3D and capture rich textures inherent in the data. Second, a technique based on semi-supervised edge learning is used to propagate high confidence structural edges during the tracing process. In contrast to predominantly edge based methods in the TEM tracing literature, this work simultaneously combines regional texture and learnt edge features into a single framework. Experimental results show that the proposed method outperforms more traditional models in tracing neuronal structures from TEM stacks.