Automated detection of synapses in serial section transmission electron microscopy image stacks

Creating ground truth for nanocrystal morphology: a fully automated pipeline for unbiased transmission electron microscopy analysis

Control over colloidal nanocrystal morphology (size, size distribution, and shape) is important for tailoring the functionality of individual nanocrystals and their ensemble behavior. Despite this, traditional methods to quantify nanocrystal morphology are laborious. New developments in automated morphology classification will accelerate these analyses but the assessment of machine learning models is limited by human accuracy for ground truth, causing even unsupervised machine learning models to have inherent bias. Herein, we introduce synthetic image rendering to solve the ground truth problem of nanocrystal morphology classification. By simulating 2D images of nanocrystal shapes via a function of high-dimensional parameter space, we trained a convolutional neural network to link unique morphologies to their simulated parameters, defining nanocrystal morphology quantitatively rather than qualitatively. An automated pipeline then processes, quantitatively defines, and classifies nanocrystal morphology from experimental transmission electron microscopy (TEM) images. Using improved computer vision techniques, 42 650 nanocrystals were identified, assessed, and labeled with quantitative parameters, offering a 600-fold improvement in efficiency over best-practice manual measurements. A classification algorithm was trained with a prediction accuracy of 99.5%, which can successfully analyze a range of concave, convex, and irregular nanocrystal shapes. The resulting pipeline was applied to differentiating two syntheses of nominally cuboidal CsPbBr₃ nanocrystals and uniquely classifying binary nickel sulfide nanocrystal phase based on morphology. This pipeline provides a simple, efficient, and unbiased method to quantify nanocrystal morphology and represents a practical route to construct large datasets with an absolute ground truth for training unbiased morphology-based machine learning algorithms.

How innovations in methodology offer new prospects for volume electron microscopy

Detailed knowledge of biological structure has been key in understanding biology at several levels of organisation, from organs to cells and proteins. Volume electron microscopy (volume EM) provides high resolution 3D structural information about tissues on the nanometre scale. However, the throughput rate of conventional electron microscopes has limited the volume size and number of samples that can be imaged. Recent improvements in methodology are currently driving a revolution in volume EM, making possible the structural imaging of whole organs and small organisms. In turn, these recent developments in image acquisition have created or stressed bottlenecks in other parts of the pipeline, like sample preparation, image analysis and data management. While the progress in image analysis is stunning due to the advent of automatic segmentation and server-based annotation tools, several challenges remain. Here we discuss recent trends in volume EM, emerging methods for increasing throughput and implications for sample preparation, image analysis and data management.

Automatic classification and neurotransmitter prediction of synapses in electron microscopy

This paper presents a deep-learning-based workflow to detect synapses and predict their neurotransmitter type in the primitive chordate Ciona intestinalis (Ciona) electron microscopic (EM) images. Identifying synapses from EM images to build a full map of connections between neurons is a labor-intensive process and requires significant domain expertise. Automation of synapse classification would hasten the generation and analysis of connectomes. Furthermore, inferences concerning neuron type and function from synapse features are in many cases difficult to make. Finding the connection between synapse structure and function is an important step in fully understanding a connectome. Class Activation Maps derived from the convolutional neural network provide insights on important features of synapses based on cell type and function. The main contribution of this work is in the differentiation of synapses by neurotransmitter type through the structural information in their EM images. This enables the prediction of neurotransmitter types for neurons in Ciona, which were previously unknown. The prediction model with code is available on GitHub.

Contour, a semi-automated segmentation and quantitation tool for cryo-soft-X-ray tomography

Cryo-soft-X-ray tomography is being increasingly used in biological research to study the morphology of cellular compartments and how they change in response to different stimuli, such as viral infections. Segmentation of these compartments is limited by time-consuming manual tools or machine learning algorithms that require extensive time and effort to train. Here we describe Contour, a new, easy-to-use, highly automated segmentation tool that enables accelerated segmentation of tomograms to delineate distinct cellular compartments. Using Contour, cellular structures can be segmented based on their projection intensity and geometrical width by applying a threshold range to the image and excluding noise smaller in width than the cellular compartments of interest. This method is less laborious and less prone to errors from human judgement than current tools that require features to be manually traced, and does not require training datasets as would machine-learning driven segmentation. We show that high-contrast compartments such as mitochondria, lipid droplets, and features at the cell surface can be easily segmented with this technique in the context of investigating herpes simplex virus 1 infection. Contour can extract geometric measurements from 3D segmented volumes, providing a new method to quantitate cryo-soft-X-ray tomography data. Contour can be freely downloaded at github.com/kamallouisnahas/Contour.

Automated Synapse Detection Method for Cerebellar Connectomics

The connectomic analyses of large-scale volumetric electron microscope (EM) images enable the discovery of hidden neural connectivity. While the technologies for neuronal reconstruction of EM images are under rapid progress, the technologies for synapse detection are lagging behind. Here, we propose a method that automatically detects the synapses in the 3D EM images, specifically for the mouse cerebellar molecular layer (CML). The method aims to accurately detect the synapses between the reconstructed neuronal fragments whose types can be identified. It extracts the contacts between the reconstructed neuronal fragments and classifies them as synaptic or non-synaptic with the help of type information and two deep learning artificial intelligences (AIs). The method can also assign the pre- and postsynaptic sides of a synapse and determine excitatory and inhibitory synapse types. The accuracy of this method is estimated to be 0.955 in F1-score for a test volume of CML containing 508 synapses. To demonstrate the usability, we measured the size and number of the synapses in the volume and investigated the subcellular connectivity between the CML neuronal fragments. The basic idea of the method to exploit tissue-specific properties can be extended to other brain regions.

Cellular connectomes as arbiters of local circuit models in the cerebral cortex

With the availability of cellular-resolution connectivity maps, connectomes, from the mammalian nervous system, it is in question how informative such massive connectomic data can be for the distinction of local circuit models in the mammalian cerebral cortex. Here, we investigated whether cellular-resolution connectomic data can in principle allow model discrimination for local circuit modules in layer 4 of mouse primary somatosensory cortex. We used approximate Bayesian model selection based on a set of simple connectome statistics to compute the posterior probability over proposed models given a to-be-measured connectome. We find that the distinction of the investigated local cortical models is faithfully possible based on purely structural connectomic data with an accuracy of more than 90%, and that such distinction is stable against substantial errors in the connectome measurement. Furthermore, mapping a fraction of only 10% of the local connectome is sufficient for connectome-based model distinction under realistic experimental constraints. Together, these results show for a concrete local circuit example that connectomic data allows model selection in the cerebral cortex and define the experimental strategy for obtaining such connectomic data.

Two Stream Active Query Suggestion for Active Learning in Connectomics

For large-scale vision tasks in biomedical images, the labeled data is often limited to train effective deep models. Active learning is a common solution, where a query suggestion method selects representative unlabeled samples for annotation, and the new labels are used to improve the base model. However, most query suggestion models optimize their learnable parameters only on the limited labeled data and consequently become less effective for the more challenging unlabeled data. To tackle this, we propose a two-stream active query suggestion approach. In addition to the supervised feature extractor, we introduce an unsupervised one optimized on all raw images to capture diverse image features, which can later be improved by fine-tuning on new labels. As a use case, we build an end-to-end active learning framework with our query suggestion method for 3D synapse detection and mitochondria segmentation in connectomics. With the framework, we curate, to our best knowledge, the largest connectomics dataset with dense synapses and mitochondria annotation. On this new dataset, our method outperforms previous state-of-the-art methods by 3.1% for synapse and 3.8% for mitochondria in terms of region-of-interest proposal accuracy. We also apply our method to image classification, where it outperforms previous approaches on CIFAR-10 under the same limited annotation budget. The project page is https://zudi-lin.github.io/projects/#two_stream_active.

Multiscale Imaging Approach for Studying the Central Nervous System: Methodology and Perspective

Non-invasive imaging methods have become essential tools for understanding the central nervous system (CNS) in health and disease. In particular, magnetic resonance imaging (MRI) techniques provide information about the anatomy, microstructure, and function of the brain and spinal cord in vivo non-invasively. However, MRI is limited by its spatial resolution and signal specificity. In order to mitigate these shortcomings, it is crucial to validate MRI with an array of ancillary ex vivo imaging techniques. These techniques include histological methods, such as light and electron microscopy (EM), which can provide specific information on the tissue structure in healthy and diseased brain and spinal cord, at cellular and subcellular level. However, these conventional histological techniques are intrinsically two-dimensional (2D) and, as a result of sectioning, lack volumetric information of the tissue. This limitation can be overcome with genuine three-dimensional (3D) imaging approaches of the tissue. 3D highly resolved information of the CNS achievable by means of other imaging techniques can complement and improve the interpretation of MRI measurements. In this article, we provide an overview of different 3D imaging techniques that can be used to validate MRI. As an example, we introduce an approach of how to combine diffusion MRI and synchrotron X-ray phase contrast tomography (SXRPCT) data. Our approach paves the way for a new multiscale assessment of the CNS allowing to validate and to improve our understanding of in vivo imaging (such as MRI).

Multifaceted Changes in Synaptic Composition and Astrocytic Involvement in a Mouse Model of Fragile X Syndrome

Fragile X Syndrome (FXS), a common inheritable form of intellectual disability, is known to alter neocortical circuits. However, its impact on the diverse synapse types comprising these circuits, or on the involvement of astrocytes, is not well known. We used immunofluorescent array tomography to quantify different synaptic populations and their association with astrocytes in layers 1 through 4 of the adult somatosensory cortex of a FXS mouse model, the FMR1 knockout mouse. The collected multi-channel data contained approximately 1.6 million synapses which were analyzed using a probabilistic synapse detector. Our study reveals complex, synapse-type and layer specific changes in the neocortical circuitry of FMR1 knockout mice. We report an increase of small glutamatergic VGluT1 synapses in layer 4 accompanied by a decrease in large VGluT1 synapses in layers 1 and 4. VGluT2 synapses show a rather consistent decrease in density in layers 1 and 2/3. In all layers, we observe the loss of large inhibitory synapses. Lastly, astrocytic association of excitatory synapses decreases. The ability to dissect the circuit deficits by synapse type and astrocytic involvement will be crucial for understanding how these changes affect circuit function, and ultimately defining targets for therapeutic intervention.

Learning cellular morphology with neural networks

Reconstruction and annotation of volume electron microscopy data sets of brain tissue is challenging but can reveal invaluable information about neuronal circuits. Significant progress has recently been made in automated neuron reconstruction as well as automated detection of synapses. However, methods for automating the morphological analysis of nanometer-resolution reconstructions are less established, despite the diversity of possible applications. Here, we introduce cellular morphology neural networks (CMNs), based on multi-view projections sampled from automatically reconstructed cellular fragments of arbitrary size and shape. Using unsupervised training, we infer morphology embeddings (Neuron2vec) of neuron reconstructions and train CMNs to identify glia cells in a supervised classification paradigm, which are then used to resolve neuron reconstruction errors. Finally, we demonstrate that CMNs can be used to identify subcellular compartments and the cell types of neuron reconstructions.

Software tools for automated transmission electron microscopy

The demand for high-throughput data collection in electron microscopy is increasing for applications in structural and cellular biology. Here we present a combination of software tools that enable automated acquisition guided by image analysis for a variety of transmission electron microscopy acquisition schemes. SerialEM controls microscopes and detectors and can trigger automated tasks at multiple positions with high flexibility. Py-EM interfaces with SerialEM to enact specimen-specific image-analysis pipelines that enable feedback microscopy. As example applications, we demonstrate dose reduction in cryo-electron microscopy experiments, fully automated acquisition of every cell in a plastic section and automated targeting on serial sections for 3D volume imaging across multiple grids.

Convolutional nets for reconstructing neural circuits from brain images acquired by serial section electron microscopy

Neural circuits can be reconstructed from brain images acquired by serial section electron microscopy. Image analysis has been performed by manual labor for half a century, and efforts at automation date back almost as far. Convolutional nets were first applied to neuronal boundary detection a dozen years ago, and have now achieved impressive accuracy on clean images. Robust handling of image defects is a major outstanding challenge. Convolutional nets are also being employed for other tasks in neural circuit reconstruction: finding synapses and identifying synaptic partners, extending or pruning neuronal reconstructions, and aligning serial section images to create a 3D image stack. Computational systems are being engineered to handle petavoxel images of cubic millimeter brain volumes.

Reprogramming the brain with synthetic neurobiology

The mammalian brain is among the most complex organs known in biology. Historically, neuroscience techniques have consisted primarily of low-throughput microscopy and electrophysiological approaches. While these methods will continue to serve the community, the emerging field of synthetic neurobiology may be better equipped to scale with systems neuroscience. By using genetic techniques to achieve cell-type specificity, a map of the connectome, neural activation and recording, and ultimately to program neural development itself, we can begin to build a better framework with which to understand the brain's mechanisms.

Navigating the Murine Brain: Toward Best Practices for Determining and Documenting Neuroanatomical Locations in Experimental Studies

In experimental neuroscientific research, anatomical location is a key attribute of experimental observations and critical for interpretation of results, replication of findings, and comparison of data across studies. With steadily rising numbers of publications reporting basic experimental results, there is an increasing need for integration and synthesis of data. Since comparison of data relies on consistently defined anatomical locations, it is a major concern that practices and precision in the reporting of location of observations from different types of experimental studies seem to vary considerably. To elucidate and possibly meet this challenge, we have evaluated and compared current practices for interpreting and documenting the anatomical location of measurements acquired from murine brains with different experimental methods. Our observations show substantial differences in approach, interpretation and reproducibility of anatomical locations among reports of different categories of experimental research, and strongly indicate that ambiguous reports of anatomical location can be attributed to missing descriptions. Based on these findings, we suggest a set of minimum requirements for documentation of anatomical location in experimental murine brain research. We furthermore demonstrate how these requirements have been applied in the EU Human Brain Project to optimize workflows for integration of heterogeneous data in common reference atlases. We propose broad adoption of some straightforward steps for improving the precision of location metadata and thereby facilitating interpretation, reuse and integration of data.

Fully-Automatic Synapse Prediction and Validation on a Large Data Set

Extracting a connectome from an electron microscopy (EM) data set requires identification of neurons and determination of connections (synapses) between neurons. As manual extraction of this information is very time-consuming, there has been extensive research efforts to automatically segment the neurons to help guide and eventually replace manual tracing. Until recently, there has been comparatively little research on automatic detection of the actual synapses between neurons. This discrepancy can, in part, be attributed to several factors: obtaining neuronal shapes is a prerequisite for the first step in extracting a connectome, manual tracing is much more time-consuming than annotating synapses, and neuronal contact area can be used as a proxy for synapses in determining connections. However, recent research has demonstrated that contact area alone is not a sufficient predictor of a synaptic connection. Moreover, as segmentation improved, we observed that synapse annotation consumes a more significant fraction of overall reconstruction time (upwards of 50% of total effort). This ratio will only get worse as segmentation improves, gating the overall possible speed-up. Therefore, we address this problem by developing algorithms that automatically detect presynaptic neurons and their postsynaptic partners. In particular, presynaptic structures are detected using a U-Net convolutional neural network (CNN), and postsynaptic partners are detected using a multilayer perceptron (MLP) with features conditioned on the local segmentation. This work is novel because it requires minimal amount of training, leverages advances in image segmentation directly, and provides a complete solution for polyadic synapse detection. We further introduce novel metrics to evaluate our algorithm on connectomes of meaningful size. When applied to the output of our method on EM data from Drosphila, these metrics demonstrate that a completely automatic prediction can be used to effectively characterize most of the connectivity correctly.

A Computational Synaptic Antibody Characterization Tool for Array Tomography

Application-specific validation of antibodies is a critical prerequisite for their successful use. Here we introduce an automated framework for characterization and screening of antibodies against synaptic molecules for high-resolution immunofluorescence array tomography (AT). The proposed Synaptic Antibody Characterization Tool (SACT) is designed to provide an automatic, robust, flexible, and efficient tool for antibody characterization at scale. SACT automatically detects puncta of immunofluorescence labeling from candidate antibodies and determines whether a punctum belongs to a synapse. The molecular composition and size of the target synapses expected to contain the antigen is determined by the user, based on biological knowledge. Operationally, the presence of a synapse is defined by the colocalization or adjacency of the candidate antibody punctum to one or more reference antibody puncta. The outputs of SACT are automatically computed measurements such as target synapse density and target specificity ratio that reflect the sensitivity and specificity of immunolabeling with a given candidate antibody. These measurements provide an objective way to characterize and compare the performance of different antibodies against the same target, and can be used to objectively select the antibodies best suited for AT and potentially for other immunolabeling applications.

Effective automated pipeline for 3D reconstruction of synapses based on deep learning

Background

The locations and shapes of synapses are important in reconstructing connectomes and analyzing synaptic plasticity. However, current synapse detection and segmentation methods are still not adequate for accurately acquiring the synaptic connectivity, and they cannot effectively alleviate the burden of synapse validation.

Results

We propose a fully automated method that relies on deep learning to realize the 3D reconstruction of synapses in electron microscopy (EM) images. The proposed method consists of three main parts: (1) training and employing the faster region convolutional neural networks (R-CNN) algorithm to detect synapses, (2) using the z-continuity of synapses to reduce false positives, and (3) combining the Dijkstra algorithm with the GrabCut algorithm to obtain the segmentation of synaptic clefts. Experimental results were validated by manual tracking, and the effectiveness of our proposed method was demonstrated. The experimental results in anisotropic and isotropic EM volumes demonstrate the effectiveness of our algorithm, and the average precision of our detection (92.8% in anisotropy, 93.5% in isotropy) and segmentation (88.6% in anisotropy, 93.0% in isotropy) suggests that our method achieves state-of-the-art results.

Conclusions

Our fully automated approach contributes to the development of neuroscience, providing neurologists with a rapid approach for obtaining rich synaptic statistics.

Electronic supplementary material

The online version of this article (10.1186/s12859-018-2232-0) contains supplementary material, which is available to authorized users.

Microscopy in Infectious Disease Research-Imaging Across Scales

A comprehensive understanding of host-pathogen interactions requires quantitative assessment of molecular events across a wide range of spatiotemporal scales and organizational complexities. Due to recent technical developments, this is currently only achievable with microscopy. This article is providing a general perspective on the importance of microscopy in infectious disease research, with a focus on new imaging modalities that promise to have a major impact in biomedical research in the years to come. Every major technological breakthrough in light microscopy depends on, and is supported by, advancements in computing and information technologies. Bioimage acquisition and analysis based on machine learning will pave the way toward more robust, automated and objective implementation of new imaging modalities and in biomedical research in general. The combination of novel imaging technologies with machine learning and near-physiological model systems promises to accelerate discoveries and breakthroughs in our understanding of infectious diseases, from basic research all the way to clinical applications.

Data integration through brain atlasing: Human Brain Project tools and strategies

The Human Brain Project (HBP), an EU Flagship Initiative, is currently building an infrastructure that will allow integration of large amounts of heterogeneous neuroscience data. The ultimate goal of the project is to develop a unified multi-level understanding of the brain and its diseases, and beyond this to emulate the computational capabilities of the brain. Reference atlases of the brain are one of the key components in this infrastructure. Based on a new generation of three-dimensional (3D) reference atlases, new solutions for analyzing and integrating brain data are being developed. HBP will build services for spatial query and analysis of brain data comparable to current online services for geospatial data. The services will provide interactive access to a wide range of data types that have information about anatomical location tied to them. The 3D volumetric nature of the brain, however, introduces a new level of complexity that requires a range of tools for making use of and interacting with the atlases. With such new tools, neuroscience research groups will be able to connect their data to atlas space, share their data through online data systems, and search and find other relevant data through the same systems. This new approach partly replaces earlier attempts to organize research data based only on a set of semantic terminologies describing the brain and its subdivisions.

Comparison of 3D cellular imaging techniques based on scanned electron probes: Serial block face SEM vs. Axial bright-field STEM tomography

Microscopies based on focused electron probes allow the cell biologist to image the 3D ultrastructure of eukaryotic cells and tissues extending over large volumes, thus providing new insight into the relationship between cellular architecture and function of organelles. Here we compare two such techniques: electron tomography in conjunction with axial bright-field scanning transmission electron microscopy (BF-STEM), and serial block face scanning electron microscopy (SBF-SEM). The advantages and limitations of each technique are illustrated by their application to determining the 3D ultrastructure of human blood platelets, by considering specimen geometry, specimen preparation, beam damage and image processing methods. Many features of the complex membranes composing the platelet organelles can be determined from both approaches, although STEM tomography offers a higher ∼3 nm isotropic pixel size, compared with ∼5 nm for SBF-SEM in the plane of the block face and ∼30 nm in the perpendicular direction. In this regard, we demonstrate that STEM tomography is advantageous for visualizing the platelet canalicular system, which consists of an interconnected network of narrow (∼50-100 nm) membranous cisternae. In contrast, SBF-SEM enables visualization of complete platelets, each of which extends ∼2 µm in minimum dimension, whereas BF-STEM tomography can typically only visualize approximately half of the platelet volume due to a rapid non-linear loss of signal in specimens of thickness greater than ∼1.5 µm. We also show that the limitations of each approach can be ameliorated by combining 3D and 2D measurements using a stereological approach.

SynEM, automated synapse detection for connectomics

Nerve tissue contains a high density of chemical synapses, about 1 per µm3 in the mammalian cerebral cortex. Thus, even for small blocks of nerve tissue, dense connectomic mapping requires the identification of millions to billions of synapses. While the focus of connectomic data analysis has been on neurite reconstruction, synapse detection becomes limiting when datasets grow in size and dense mapping is required. Here, we report SynEM, a method for automated detection of synapses from conventionally en-bloc stained 3D electron microscopy image stacks. The approach is based on a segmentation of the image data and focuses on classifying borders between neuronal processes as synaptic or non-synaptic. SynEM yields 97% precision and recall in binary cortical connectomes with no user interaction. It scales to large volumes of cortical neuropil, plausibly even whole-brain datasets. SynEM removes the burden of manual synapse annotation for large densely mapped connectomes.

webKnossos: efficient online 3D data annotation for connectomics

We report webKnossos, an in-browser annotation tool for 3D electron microscopic data. webKnossos provides flight mode, a single-view egocentric reconstruction method enabling trained annotator crowds to reconstruct at a speed of 1.5 ± 0.6 mm/h for axons and 2.1 ± 0.9 mm/h for dendrites in 3D electron microscopic data from mammalian cortex. webKnossos accelerates neurite reconstruction for connectomics by 4- to 13-fold compared with current state-of-the-art tools, thus extending the range of connectomes that can realistically be mapped in the future.

Automated gold particle quantification of immunogold labeled micrographs

BACKGROUND

Immunogold cytochemistry is the method of choice for precise localization of antigens on a subcellular scale. The process of immunogold quantification in electron micrographs is laborious, especially for proteins with a dense distribution pattern.

NEW METHODS

Here I present a MATLAB based toolbox that is optimized for a typical immunogold analysis workflow. It combines automatic detection of gold particles through a multi-threshold algorithm with manual segmentation of cell membranes and regions of interests.

RESULTS

The automated particle detection algorithm was applied to a typical immunogold dataset of neural tissue, and was able to detect particles with a high degree of precision. Without manual correction, the algorithm detected 97% of all gold particles, with merely a 0.1% false-positive rate.

COMPARISONS WITH EXISTING METHOD(S)

To my knowledge, this is the first free and publicly available software custom made for immunogold analyses. The proposed particle detection method compares favorably to previously published algorithms.

CONCLUSIONS

The software presented here will be valuable tool for researchers in neuroscience working with immunogold cytochemistry.

Quantifying Golgi structure using EM: combining volume-SEM and stereology for higher throughput

Investigating organelles such as the Golgi complex depends increasingly on high-throughput quantitative morphological analyses from multiple experimental or genetic conditions. Light microscopy (LM) has been an effective tool for screening but fails to reveal fine details of Golgi structures such as vesicles, tubules and cisternae. Electron microscopy (EM) has sufficient resolution but traditional transmission EM (TEM) methods are slow and inefficient. Newer volume scanning EM (volume-SEM) methods now have the potential to speed up 3D analysis by automated sectioning and imaging. However, they produce large arrays of sections and/or images, which require labour-intensive 3D reconstruction for quantitation on limited cell numbers. Here, we show that the information storage, digital waste and workload involved in using volume-SEM can be reduced substantially using sampling-based stereology. Using the Golgi as an example, we describe how Golgi populations can be sensed quantitatively using single random slices and how accurate quantitative structural data on Golgi organelles of individual cells can be obtained using only 5–10 sections/images taken from a volume-SEM series (thereby sensing population parameters and cell–cell variability). The approach will be useful in techniques such as correlative LM and EM (CLEM) where small samples of cells are treated and where there may be variable responses. For Golgi study, we outline a series of stereological estimators that are suited to these analyses and suggest workflows, which have the potential to enhance the speed and relevance of data acquisition in volume-SEM.

Automated synaptic connectivity inference for volume electron microscopy

Teravoxel volume electron microscopy data sets from neural tissue can now be acquired in weeks, but data analysis requires years of manual labor. We developed the SyConn framework, which uses deep convolutional neural networks and random forest classifiers to infer a richly annotated synaptic connectivity matrix from manual neurite skeleton reconstructions by automatically identifying mitochondria, synapses and their types, axons, dendrites, spines, myelin, somata and cell types. We tested our approach on serial block-face electron microscopy data sets from zebrafish, mouse and zebra finch, and computed the synaptic wiring of songbird basal ganglia. We found that, for example, basal-ganglia cell types with high firing rates in vivo had higher densities of mitochondria and vesicles and that synapse sizes and quantities scaled systematically, depending on the innervated postsynaptic cell types.

Quantitative neuroanatomy for connectomics in Drosophila

Neuronal circuit mapping using electron microscopy demands laborious proofreading or reconciliation of multiple independent reconstructions. Here, we describe new methods to apply quantitative arbor and network context to iteratively proofread and reconstruct circuits and create anatomically enriched wiring diagrams. We measured the morphological underpinnings of connectivity in new and existing reconstructions of Drosophila sensorimotor (larva) and visual (adult) systems. Synaptic inputs were preferentially located on numerous small, microtubule-free 'twigs' which branch off a single microtubule-containing 'backbone'. Omission of individual twigs accounted for 96% of errors. However, the synapses of highly connected neurons were distributed across multiple twigs. Thus, the robustness of a strong connection to detailed twig anatomy was associated with robustness to reconstruction error. By comparing iterative reconstruction to the consensus of multiple reconstructions, we show that our method overcomes the need for redundant effort through the discovery and application of relationships between cellular neuroanatomy and synaptic connectivity.

DOI:http://dx.doi.org/10.7554/eLife.12059.001

The nervous system contains cells called neurons, which connect to each other to form circuits that send and process information. Each neuron receives and transmits signals to other neurons via very small junctions called synapses. Neurons are shaped a bit like trees, and most input synapses are located in the tiniest branches. Understanding the architecture of a neuron’s branches is important to understand the role that a particular neuron plays in processing information. Therefore, neuroscientists strive to reconstruct the architecture of these branches and how they connect to one another using imaging techniques.

One imaging technique known as serial electron microscopy generates highly detailed images of neural circuits. However, reconstructing neural circuits from such images is notoriously time consuming and error prone. These errors could result in the reconstructed circuit being very different than the real-life circuit. For example, an error that leads to missing out a large branch could result in researchers failing to notice many important connections in the circuit. On the other hand, some errors may not matter much because the neurons share other synapses that are included in the reconstruction.

To understand what effect errors have on the reconstructed circuits, neuroscientists need to have a more detailed understanding of the relationship between the shape of a neuron, its synaptic connections to other neurons, and where errors commonly occur. Here, Schneider-Mizell, Gerhard et al. study this relationship in detail and then devise a faster reconstruction method that uses the shape and other properties of neurons without sacrificing accuracy. The method includes a way to include data from the shape of neurons in the circuit wiring diagrams, revealing circuit patterns that would otherwise go unnoticed.

The experiments use serial electron microscopy images of neurons from fruit flies and show that, from the tiniest larva to the adult fly, neurons form synapses with each other in a similar way. Most errors in the reconstruction only affect the tips of the smallest branches, which generally only host a single synapse. Such omissions do not have a big effect on the reconstructed circuit because strongly connected neurons make multiple synapses onto each other.

Schneider-Mizell, Gerhard et al.'s approach will help researchers to reconstruct neural circuits and analyze them more effectively than was possible before. The algorithms and tools developed in this study are available in an open source software package so that they can be used by other researchers in the future.

DOI:http://dx.doi.org/10.7554/eLife.12059.002

Recent advancement in the challenges to connectomics

Advancement of microscopic technologies established significant progress in our understanding of the brain. In the recent effort to elucidate the complete wiring map of the brain circuitry termed 'connectome', the different modalities of imaging technology, including those of light and electron microscopy, have started providing essential contribution in multiple organisms. The contribution would be impossible without the recent innovation in both acquisition and analyses of the big connectomic data. The current data demonstrated complicated networks with unidirectional and reciprocal connections of the cerebral circuits at the macroscopic and light microscopic ('mesoscopic') levels, and the unimaginable complexity of synaptic connections between axons and dendrites at the electron microscopic ('microscopic') level. At the same time, the data highlighted the necessity to make substantial advancement in methodology of the connectomic studies, including efficient handling and automated analyses of the acquired dataset. Further understanding about structural and functional connectome seems to be facilitated by combinations of the different imaging modalities. Such multidisciplinary approaches will give us the clues to address whether the complete connectome can elucidate fundamental mechanisms processing the basic and higher functions of human brains.

An automated images-to-graphs framework for high resolution connectomics

Reconstructing a map of neuronal connectivity is a critical challenge in contemporary neuroscience. Recent advances in high-throughput serial section electron microscopy (EM) have produced massive 3D image volumes of nanoscale brain tissue for the first time. The resolution of EM allows for individual neurons and their synaptic connections to be directly observed. Recovering neuronal networks by manually tracing each neuronal process at this scale is unmanageable, and therefore researchers are developing automated image processing modules. Thus far, state-of-the-art algorithms focus only on the solution to a particular task (e.g., neuron segmentation or synapse identification). In this manuscript we present the first fully-automated images-to-graphs pipeline (i.e., a pipeline that begins with an imaged volume of neural tissue and produces a brain graph without any human interaction). To evaluate overall performance and select the best parameters and methods, we also develop a metric to assess the quality of the output graphs. We evaluate a set of algorithms and parameters, searching possible operating points to identify the best available brain graph for our assessment metric. Finally, we deploy a reference end-to-end version of the pipeline on a large, publicly available data set. This provides a baseline result and framework for community analysis and future algorithm development and testing. All code and data derivatives have been made publicly available in support of eventually unlocking new biofidelic computational primitives and understanding of neuropathologies.

Combining quantitative 2D and 3D image analysis in the serial block face SEM: application to secretory organelles of pancreatic islet cells

A combination of two-dimensional (2D) and three-dimensional (3D) analyses of tissue volume ultrastructure acquired by serial block face scanning electron microscopy can greatly shorten the time required to obtain quantitative information from big data sets that contain many billions of voxels. Thus, to analyse the number of organelles of a specific type, or the total volume enclosed by a population of organelles within a cell, it is possible to estimate the number density or volume fraction of that organelle using a stereological approach to analyse randomly selected 2D block face views through the cells, and to combine such estimates with precise measurement of 3D cell volumes by delineating the plasma membrane in successive block face images. The validity of such an approach can be easily tested since the entire 3D tissue volume is available in the serial block face scanning electron microscopy data set. We have applied this hybrid 3D/2D technique to determine the number of secretory granules in the endocrine α and β cells of mouse pancreatic islets of Langerhans, and have been able to estimate the total insulin content of a β cell.