A high-throughput framework to detect synapses in electron microscopy images

Few-shot hypercolumn-based mitochondria segmentation in cardiac and outer hair cells in focused ion beam-scanning electron microscopy (FIB-SEM) data

We present a novel AI-based approach to the few-shot automated segmentation of mitochondria in large-scale electron microscopy images. Our framework leverages convolutional features from a pre-trained deep multilayer convolutional neural network, such as VGG-16. We then train a binary gradient boosting classifier on the resulting high-dimensional feature hypercolumns. We extract VGG-16 features from the first four convolutional blocks and apply bilinear upsampling to resize the obtained maps to the input image size. This procedure yields a 2688-dimensional feature hypercolumn for each pixel in a 224 × 224 input image. We then apply L 1-regularized logistic regression for supervised active feature selection to reduce dependencies among the features, to reduce overfitting, as well as to speed-up gradient boosting-based training. During inference we block process 1728 × 2022 large microscopy images. Our experiments show that in such a formulation of transfer learning our processing pipeline is able to achieve high-accuracy results on very challenging datasets containing a large number of irregularly shaped mitochondria in cardiac and outer hair cells. Our proposed few-shot training approach gives competitive performance with the state-of-the-art using far less training data.

Dendritic and Synaptic Degeneration in Pyramidal Neurons of the Sensorimotor Cortex in Neonatal Mice With Kaolin-Induced Hydrocephalus

Obstructive hydrocephalus is a brain disorder in which the circulation of cerebrospinal fluid (CSF) is altered in a manner that causes expansion of fluid-filled intracranial compartments particularly the ventricles. The pyramidal neurons of the sensorimotor cortex are excitatory in nature and their dendritic spines are targets of excitatory synapses. This study evaluated the effect of hydrocephalus on dendritic arborization and synaptic structure of the pyramidal neurons of the sensorimotor cortex of neonatal hydrocephalic mice. Sterile kaolin suspension (0.01 ml of 250 mg/mL) was injected intracisternally into day old mice. Control animals mice received sham injections. Pups were weighed and sacrificed on postnatal days (PND) 7, 14 and 21. Fixed brain tissue blocks were silver impregnated using a modified Golgi staining technique and immunolabeled with synaptophysin to determine dendritic morphology and synaptic integrity respectively. Data were analyzed using ANOVA at α 0.05. Golgi staining revealed diminished arborization of the basal dendrites and loss of dendritic spines in the pyramidal neurons of hydrocephalic mice. Compared to age-matched controls, there was a significant reduction in the percentage immunoreactivity of anti-synaptophysin in hydrocephalic mice on PND 7 (14.26 ± 1.91% vs. 62.57 ± 9.40%), PND 14 (4.19 ± 1.57% vs. 93.01 ± 1.66%) and PND 21 (17.55 ± 2.76% vs. 99.11 ± 0.63%) respectively. These alterations suggest impaired neuronal connections that are essential for the development of cortical circuits and may be the structural basis of the neurobehavioral deficits observed in neonatal hydrocephalus.

Fully-Automatic Synapse Prediction and Validation on a Large Data Set

Extracting a connectome from an electron microscopy (EM) data set requires identification of neurons and determination of connections (synapses) between neurons. As manual extraction of this information is very time-consuming, there has been extensive research efforts to automatically segment the neurons to help guide and eventually replace manual tracing. Until recently, there has been comparatively little research on automatic detection of the actual synapses between neurons. This discrepancy can, in part, be attributed to several factors: obtaining neuronal shapes is a prerequisite for the first step in extracting a connectome, manual tracing is much more time-consuming than annotating synapses, and neuronal contact area can be used as a proxy for synapses in determining connections. However, recent research has demonstrated that contact area alone is not a sufficient predictor of a synaptic connection. Moreover, as segmentation improved, we observed that synapse annotation consumes a more significant fraction of overall reconstruction time (upwards of 50% of total effort). This ratio will only get worse as segmentation improves, gating the overall possible speed-up. Therefore, we address this problem by developing algorithms that automatically detect presynaptic neurons and their postsynaptic partners. In particular, presynaptic structures are detected using a U-Net convolutional neural network (CNN), and postsynaptic partners are detected using a multilayer perceptron (MLP) with features conditioned on the local segmentation. This work is novel because it requires minimal amount of training, leverages advances in image segmentation directly, and provides a complete solution for polyadic synapse detection. We further introduce novel metrics to evaluate our algorithm on connectomes of meaningful size. When applied to the output of our method on EM data from Drosphila, these metrics demonstrate that a completely automatic prediction can be used to effectively characterize most of the connectivity correctly.

Effective automated pipeline for 3D reconstruction of synapses based on deep learning

Background

The locations and shapes of synapses are important in reconstructing connectomes and analyzing synaptic plasticity. However, current synapse detection and segmentation methods are still not adequate for accurately acquiring the synaptic connectivity, and they cannot effectively alleviate the burden of synapse validation.

Results

We propose a fully automated method that relies on deep learning to realize the 3D reconstruction of synapses in electron microscopy (EM) images. The proposed method consists of three main parts: (1) training and employing the faster region convolutional neural networks (R-CNN) algorithm to detect synapses, (2) using the z-continuity of synapses to reduce false positives, and (3) combining the Dijkstra algorithm with the GrabCut algorithm to obtain the segmentation of synaptic clefts. Experimental results were validated by manual tracking, and the effectiveness of our proposed method was demonstrated. The experimental results in anisotropic and isotropic EM volumes demonstrate the effectiveness of our algorithm, and the average precision of our detection (92.8% in anisotropy, 93.5% in isotropy) and segmentation (88.6% in anisotropy, 93.0% in isotropy) suggests that our method achieves state-of-the-art results.

Conclusions

Our fully automated approach contributes to the development of neuroscience, providing neurologists with a rapid approach for obtaining rich synaptic statistics.

Electronic supplementary material

The online version of this article (10.1186/s12859-018-2232-0) contains supplementary material, which is available to authorized users.

Network Design and the Brain

Neural circuits have evolved to accommodate similar information processing challenges as those faced by engineered systems. Here, we compare neural versus engineering strategies for constructing networks. During circuit development, synapses are overproduced and then pruned back over time, whereas in engineered networks, connections are initially sparse and are then added over time. We provide a computational perspective on these two different approaches, including discussion of how and why they are used, insights that one can provide the other, and areas for future joint investigation. By thinking algorithmically about the goals, constraints, and optimization principles used by neural circuits, we can develop brain-derived strategies for enhancing network design, while also stimulating experimental hypotheses about circuit development and function.

Correlative Light Electron Microscopy: Connecting Synaptic Structure and Function

Many core paradigms of contemporary neuroscience are based on information obtained by electron or light microscopy. Intriguingly, these two imaging techniques are often viewed as complementary, yet separate entities. Recent technological advancements in microscopy techniques, labeling tools, and fixation or preparation procedures have fueled the development of a series of hybrid approaches that allow correlating functional fluorescence microscopy data and ultrastructural information from electron micrographs from a singular biological event. As correlative light electron microscopy (CLEM) approaches become increasingly accessible, long-standing neurobiological questions regarding structure-function relation are being revisited. In this review, we will survey what developments in electron and light microscopy have spurred the advent of correlative approaches, highlight the most relevant CLEM techniques that are currently available, and discuss its potential and limitations with respect to neuronal and synapse-specific applications.

Unbiased, High-Throughput Electron Microscopy Analysis of Experience-Dependent Synaptic Changes in the Neocortex

Neocortical circuits can be altered by sensory and motor experience, with experimental evidence supporting both anatomical and electrophysiological changes in synaptic properties. Previous studies have focused on changes in specific neurons or pathways-for example, the thalamocortical circuitry, layer 4-3 (L4-L3) synapses, or in the apical dendrites of L5 neurons- but a broad-scale analysis of experience-induced changes across the cortical column has been lacking. Without this comprehensive approach, a full understanding of how cortical circuits adapt during learning or altered sensory input will be impossible. Here we adapt an electron microscopy technique that selectively labels synapses, in combination with a machine-learning algorithm for semiautomated synapse detection, to perform an unbiased analysis of developmental and experience-dependent changes in synaptic properties across an entire cortical column in mice. Synapse density and length were compared across development and during whisker-evoked plasticity. Between postnatal days 14 and 18, synapse density significantly increases most in superficial layers, and synapse length increases in L3 and L5B. Removal of all but a single whisker row for 24 h led to an apparent increase in synapse density in L2 and a decrease in L6, and a significant increase in length in L3. Targeted electrophysiological analysis of changes in miniature EPSC and IPSC properties in L2 pyramidal neurons showed that mEPSC frequency nearly doubled in the whisker-spared column, a difference that was highly significant. Together, this analysis shows that data-intensive analysis of column-wide changes in synapse properties can generate specific and testable hypotheses about experience-dependent changes in cortical organization.

SIGNIFICANCE STATEMENT

Development and sensory experience can change synapse properties in the neocortex. Here we use a semiautomated analysis of electron microscopy images for an unbiased, column-wide analysis of synapse changes. This analysis reveals new loci for synaptic change that can be verified by targeted electrophysiological investigation. This method can be used as a platform for generating new hypotheses about synaptic changes across different brain areas and experimental conditions.

A Fast Method for the Segmentation of Synaptic Junctions and Mitochondria in Serial Electron Microscopic Images of the Brain

Recent electron microscopy (EM) imaging techniques permit the automatic acquisition of a large number of serial sections from brain samples. Manual segmentation of these images is tedious, time-consuming and requires a high degree of user expertise. Therefore, there is considerable interest in developing automatic segmentation methods. However, currently available methods are computationally demanding in terms of computer time and memory usage, and to work properly many of them require image stacks to be isotropic, that is, voxels must have the same size in the X, Y and Z axes. We present a method that works with anisotropic voxels and that is computationally efficient allowing the segmentation of large image stacks. Our approach involves anisotropy-aware regularization via conditional random field inference and surface smoothing techniques to improve the segmentation and visualization. We have focused on the segmentation of mitochondria and synaptic junctions in EM stacks from the cerebral cortex, and have compared the results to those obtained by other methods. Our method is faster than other methods with similar segmentation results. Our image regularization procedure introduces high-level knowledge about the structure of labels. We have also reduced memory requirements with the introduction of energy optimization in overlapping partitions, which permits the regularization of very large image stacks. Finally, the surface smoothing step improves the appearance of three-dimensional renderings of the segmented volumes.

Decreasing-Rate Pruning Optimizes the Construction of Efficient and Robust Distributed Networks

Robust, efficient, and low-cost networks are advantageous in both biological and engineered systems. During neural network development in the brain, synapses are massively over-produced and then pruned-back over time. This strategy is not commonly used when designing engineered networks, since adding connections that will soon be removed is considered wasteful. Here, we show that for large distributed routing networks, network function is markedly enhanced by hyper-connectivity followed by aggressive pruning and that the global rate of pruning, a developmental parameter not previously studied by experimentalists, plays a critical role in optimizing network structure. We first used high-throughput image analysis techniques to quantify the rate of pruning in the mammalian neocortex across a broad developmental time window and found that the rate is decreasing over time. Based on these results, we analyzed a model of computational routing networks and show using both theoretical analysis and simulations that decreasing rates lead to more robust and efficient networks compared to other rates. We also present an application of this strategy to improve the distributed design of airline networks. Thus, inspiration from neural network formation suggests effective ways to design distributed networks across several domains.

Semi-automatic organelle detection on transmission electron microscopic images

Recent advances in the acquisition of large-scale datasets of transmission electron microscope images have allowed researchers to determine the number and the distribution of subcellular ultrastructures at both the cellular level and the tissue level. For this purpose, it would be very useful to have a computer-assisted system to detect the structures of interest, such as organelles. Using our original image recognition framework CARTA (Clustering-Aided Rapid Training Agent), combined with procedures to highlight and enlarge regions of interest on the image, we have developed a successful method for the semi-automatic detection of plant organelles including mitochondria, amyloplasts, chloroplasts, etioplasts, and Golgi stacks in transmission electron microscope images. Our proposed semi-automatic detection system will be helpful for labelling organelles in the interpretation and/or quantitative analysis of large-scale electron microscope imaging data.

Clift MJ, Blank F, Petri-Fink A, Rothen-Rutishauser B. Quantification of nanoparticles at the single-cell level: an overview about state-of-the-art techniques and their limitations

With the increasing production and use of engineered nanoparticles it is crucial that their interaction with biological systems is understood. Due to the small size of nanoparticles, their identification and localization within single cells is extremely challenging. Therefore, various cutting-edge techniques are required to detect and to quantify metals, metal oxides, magnetic, fluorescent, as well as electron-dense nanoparticles. Several techniques will be discussed in detail, such as inductively coupled plasma atomic emission spectroscopy, flow cytometry, laser scanning microscopy combined with digital image restoration, as well as quantitative analysis by means of stereology on transmission electron microscopy images. An overview will be given regarding the advantages of those visualization/quantification systems, including a thorough discussion about limitations and pitfalls.

Automated detection of synapses in serial section transmission electron microscopy image stacks

We describe a method for fully automated detection of chemical synapses in serial electron microscopy images with highly anisotropic axial and lateral resolution, such as images taken on transmission electron microscopes. Our pipeline starts from classification of the pixels based on 3D pixel features, which is followed by segmentation with an Ising model MRF and another classification step, based on object-level features. Classifiers are learned on sparse user labels; a fully annotated data subvolume is not required for training. The algorithm was validated on a set of 238 synapses in 20 serial 7197×7351 pixel images (4.5×4.5×45 nm resolution) of mouse visual cortex, manually labeled by three independent human annotators and additionally re-verified by an expert neuroscientist. The error rate of the algorithm (12% false negative, 7% false positive detections) is better than state-of-the-art, even though, unlike the state-of-the-art method, our algorithm does not require a prior segmentation of the image volume into cells. The software is based on the ilastik learning and segmentation toolkit and the vigra image processing library and is freely available on our website, along with the test data and gold standard annotations (http://www.ilastik.org/synapse-detection/sstem).