A Fast Method for the Segmentation of Synaptic Junctions and Mitochondria in Serial Electron Microscopic Images of the Brain

Mitochondrial heterogeneity in diseases

As key organelles involved in cellular metabolism, mitochondria frequently undergo adaptive changes in morphology, components and functions in response to various environmental stresses and cellular demands. Previous studies of mitochondria research have gradually evolved, from focusing on morphological change analysis to systematic multiomics, thereby revealing the mitochondrial variation between cells or within the mitochondrial population within a single cell. The phenomenon of mitochondrial variation features is defined as mitochondrial heterogeneity. Moreover, mitochondrial heterogeneity has been reported to influence a variety of physiological processes, including tissue homeostasis, tissue repair, immunoregulation, and tumor progression. Here, we comprehensively review the mitochondrial heterogeneity in different tissues under pathological states, involving variant features of mitochondrial DNA, RNA, protein and lipid components. Then, the mechanisms that contribute to mitochondrial heterogeneity are also summarized, such as the mutation of the mitochondrial genome and the import of mitochondrial proteins that result in the heterogeneity of mitochondrial DNA and protein components. Additionally, multiple perspectives are investigated to better comprehend the mysteries of mitochondrial heterogeneity between cells. Finally, we summarize the prospective mitochondrial heterogeneity-targeting therapies in terms of alleviating mitochondrial oxidative damage, reducing mitochondrial carbon stress and enhancing mitochondrial biogenesis to relieve various pathological conditions. The possibility of recent technological advances in targeted mitochondrial gene editing is also discussed.

Reconstruction of neocortex: Organelles, compartments, cells, circuits, and activity

We assembled a semi-automated reconstruction of L2/3 mouse primary visual cortex from ∼250 × 140 × 90 μm3 of electron microscopic images, including pyramidal and non-pyramidal neurons, astrocytes, microglia, oligodendrocytes and precursors, pericytes, vasculature, nuclei, mitochondria, and synapses. Visual responses of a subset of pyramidal cells are included. The data are publicly available, along with tools for programmatic and three-dimensional interactive access. Brief vignettes illustrate the breadth of potential applications relating structure to function in cortical circuits and neuronal cell biology. Mitochondria and synapse organization are characterized as a function of path length from the soma. Pyramidal connectivity motif frequencies are predicted accurately using a configuration model of random graphs. Pyramidal cells receiving more connections from nearby cells exhibit stronger and more reliable visual responses. Sample code shows data access and analysis.

Whole-cell organelle segmentation in volume electron microscopy

Cells contain hundreds of organelles and macromolecular assemblies. Obtaining a complete understanding of their intricate organization requires the nanometre-level, three-dimensional reconstruction of whole cells, which is only feasible with robust and scalable automatic methods. Here, to support the development of such methods, we annotated up to 35 different cellular organelle classes-ranging from endoplasmic reticulum to microtubules to ribosomes-in diverse sample volumes from multiple cell types imaged at a near-isotropic resolution of 4 nm per voxel with focused ion beam scanning electron microscopy (FIB-SEM)¹. We trained deep learning architectures to segment these structures in 4 nm and 8 nm per voxel FIB-SEM volumes, validated their performance and showed that automatic reconstructions can be used to directly quantify previously inaccessible metrics including spatial interactions between cellular components. We also show that such reconstructions can be used to automatically register light and electron microscopy images for correlative studies. We have created an open data and open-source web repository, 'OpenOrganelle', to share the data, computer code and trained models, which will enable scientists everywhere to query and further improve automatic reconstruction of these datasets.

Few-shot hypercolumn-based mitochondria segmentation in cardiac and outer hair cells in focused ion beam-scanning electron microscopy (FIB-SEM) data

We present a novel AI-based approach to the few-shot automated segmentation of mitochondria in large-scale electron microscopy images. Our framework leverages convolutional features from a pre-trained deep multilayer convolutional neural network, such as VGG-16. We then train a binary gradient boosting classifier on the resulting high-dimensional feature hypercolumns. We extract VGG-16 features from the first four convolutional blocks and apply bilinear upsampling to resize the obtained maps to the input image size. This procedure yields a 2688-dimensional feature hypercolumn for each pixel in a 224 × 224 input image. We then apply L 1-regularized logistic regression for supervised active feature selection to reduce dependencies among the features, to reduce overfitting, as well as to speed-up gradient boosting-based training. During inference we block process 1728 × 2022 large microscopy images. Our experiments show that in such a formulation of transfer learning our processing pipeline is able to achieve high-accuracy results on very challenging datasets containing a large number of irregularly shaped mitochondria in cardiac and outer hair cells. Our proposed few-shot training approach gives competitive performance with the state-of-the-art using far less training data.

Convolutional nets for reconstructing neural circuits from brain images acquired by serial section electron microscopy

Neural circuits can be reconstructed from brain images acquired by serial section electron microscopy. Image analysis has been performed by manual labor for half a century, and efforts at automation date back almost as far. Convolutional nets were first applied to neuronal boundary detection a dozen years ago, and have now achieved impressive accuracy on clean images. Robust handling of image defects is a major outstanding challenge. Convolutional nets are also being employed for other tasks in neural circuit reconstruction: finding synapses and identifying synaptic partners, extending or pruning neuronal reconstructions, and aligning serial section images to create a 3D image stack. Computational systems are being engineered to handle petavoxel images of cubic millimeter brain volumes.

Subcellular connectomic analyses of energy networks in striated muscle

Mapping biological circuit connectivity has revolutionized our understanding of structure-function relationships. Although connectomic analyses have primarily focused on neural systems, electrical connectivity within muscle mitochondrial networks was recently demonstrated to provide a rapid mechanism for cellular energy distribution. However, tools to evaluate organelle connectivity with high spatial fidelity within single cells are currently lacking. Here, we developed a framework to quantitatively assess mitochondrial network connectivity and interactions with cellular sites of energy storage, utilization, and calcium cycling in cardiac, oxidative, and glycolytic muscle. We demonstrate that mitochondrial network configuration, individual mitochondrial size and shape, and the junctions connecting mitochondria within each network are consistent with the differing contraction demands of each muscle type. Moreover, mitochondria-lipid droplet interaction analyses suggest that individual mitochondria within networks may play specialized roles regarding energy distribution and calcium cycling within the cell and reveal the power of connectomic analyses of organelle interactions within single cells.

A fast forward 3D connection algorithm for mitochondria and synapse segmentations from serial EM images

Background

It is becoming increasingly clear that the quantification of mitochondria and synapses is of great significance to understand the function of biological nervous systems. Electron microscopy (EM), with the necessary resolution in three directions, is the only available imaging method to look closely into these issues. Therefore, estimating the number of mitochondria and synapses from the serial EM images is coming into prominence. Since previous studies have achieved preferable 2D segmentation performance, it holds great promise to obtain the 3D connection relationship from the 2D segmentation results.

Results

In this paper, we improve upon Matlab’s function bwconncomp and propose a fast forward 3D connection algorithm for mitochondria and synapse segmentations from serial EM images. To benchmark the performance of the proposed method, two EM datasets with the annotated ground truth are produced for mitochondria and synapses, respectively. Experimental results show that the proposed method can achieve the preferable connection performance that closely matches the ground truth. Moreover, it greatly reduces the computational burden and alleviates the memory requirements compared with the function bwconncomp.

Conclusions

The proposed method can be deemed as an effective strategy to obtain the 3D connection relationship from serial mitochondria and synapse segmentations. It is helpful to accurately and quickly quantify the statistics of the numbers, volumes, surface areas, and lengths, which will greatly facilitate the data analysis of neurobiology research.

Automatic Mitochondria Segmentation for EM Data Using a 3D Supervised Convolutional Network

Recent studies have supported the relation between mitochondrial functions and degenerative disorders related to ageing, such as Alzheimer's and Parkinson's diseases. Since these studies have exposed the need for detailed and high-resolution analysis of physical alterations in mitochondria, it is necessary to be able to perform segmentation and 3D reconstruction of mitochondria. However, due to the variety of mitochondrial structures, automated mitochondria segmentation and reconstruction in electron microscopy (EM) images have proven to be a difficult and challenging task. This paper puts forward an effective and automated pipeline based on deep learning to realize mitochondria segmentation in different EM images. The proposed pipeline consists of three parts: (1) utilizing image registration and histogram equalization as image pre-processing steps to maintain the consistency of the dataset; (2) proposing an effective approach for 3D mitochondria segmentation based on a volumetric, residual convolutional and deeply supervised network; and (3) employing a 3D connection method to obtain the relationship of mitochondria and displaying the 3D reconstruction results. To our knowledge, we are the first researchers to utilize a 3D fully residual convolutional network with a deeply supervised strategy to improve the accuracy of mitochondria segmentation. The experimental results on anisotropic and isotropic EM volumes demonstrate the effectiveness of our method, and the Jaccard index of our segmentation (91.8% in anisotropy, 90.0% in isotropy) and F1 score of detection (92.2% in anisotropy, 90.9% in isotropy) suggest that our approach achieved state-of-the-art results. Our fully automated pipeline contributes to the development of neuroscience by providing neurologists with a rapid approach for obtaining rich mitochondria statistics and helping them elucidate the mechanism and function of mitochondria.

Multi-class segmentation of neuronal structures in electron microscopy images

BACKGROUND

Serial block face scanning electron microscopy (SBFEM) is becoming a popular technology in neuroscience. We have seen in the last years an increasing number of works addressing the problem of segmenting cellular structures in SBFEM images of brain tissue. The vast majority of them is designed to segment one specific structure, typically membranes, synapses and mitochondria. Our hypothesis is that the performance of these algorithms can be improved by concurrently segmenting more than one structure using image descriptions obtained at different scales.

RESULTS

We consider the simultaneous segmentation of two structures, namely, synapses with mitochondria, and mitochondra with membranes. To this end we select three image stacks encompassing different SBFEM acquisition technologies and image resolutions. We introduce both a new Boosting algorithm to perform feature scale selection and the Jaccard Curve as a tool compare several segmentation results. We then experimentally study the gains in performance obtained when simultaneously segmenting two structures with properly selected image descriptor scales. The results show that by doing so we achieve significant gains in segmentation accuracy when compared to the best results in the literature.

CONCLUSIONS

Simultaneously segmenting several neuronal structures described at different scales provides voxel classification algorithms with highly discriminating features that significantly improve segmentation accuracy.

SynEM, automated synapse detection for connectomics

Nerve tissue contains a high density of chemical synapses, about 1 per µm3 in the mammalian cerebral cortex. Thus, even for small blocks of nerve tissue, dense connectomic mapping requires the identification of millions to billions of synapses. While the focus of connectomic data analysis has been on neurite reconstruction, synapse detection becomes limiting when datasets grow in size and dense mapping is required. Here, we report SynEM, a method for automated detection of synapses from conventionally en-bloc stained 3D electron microscopy image stacks. The approach is based on a segmentation of the image data and focuses on classifying borders between neuronal processes as synaptic or non-synaptic. SynEM yields 97% precision and recall in binary cortical connectomes with no user interaction. It scales to large volumes of cortical neuropil, plausibly even whole-brain datasets. SynEM removes the burden of manual synapse annotation for large densely mapped connectomes.

Probabilistic fluorescence-based synapse detection

Deeper exploration of the brain’s vast synaptic networks will require new tools for high-throughput structural and molecular profiling of the diverse populations of synapses that compose those networks. Fluorescence microscopy (FM) and electron microscopy (EM) offer complementary advantages and disadvantages for single-synapse analysis. FM combines exquisite molecular discrimination capacities with high speed and low cost, but rigorous discrimination between synaptic and non-synaptic fluorescence signals is challenging. In contrast, EM remains the gold standard for reliable identification of a synapse, but offers only limited molecular discrimination and is slow and costly. To develop and test single-synapse image analysis methods, we have used datasets from conjugate array tomography (cAT), which provides voxel-conjugate FM and EM (annotated) images of the same individual synapses. We report a novel unsupervised probabilistic method for detection of synapses from multiplex FM (muxFM) image data, and evaluate this method both by comparison to EM gold standard annotated data and by examining its capacity to reproduce known important features of cortical synapse distributions. The proposed probabilistic model-based synapse detector accepts molecular-morphological synapse models as user queries, and delivers a volumetric map of the probability that each voxel represents part of a synapse. Taking human annotation of cAT EM data as ground truth, we show that our algorithm detects synapses from muxFM data alone as successfully as human annotators seeing only the muxFM data, and accurately reproduces known architectural features of cortical synapse distributions. This approach opens the door to data-driven discovery of new synapse types and their density. We suggest that our probabilistic synapse detector will also be useful for analysis of standard confocal and super-resolution FM images, where EM cross-validation is not practical.

Brain function results from communication between neurons connected by complex synaptic networks. Synapses are themselves highly complex and diverse signaling machines, containing protein products of hundreds of different genes, some in hundreds of copies, precisely arranged at each individual synapse. Synapses are fundamental not only to synaptic network function but also to network development, adaptation, and memory. In addition, abnormalities of synapse numbers or their molecular components have been implicated in a variety of mental and neurological disorders. Despite their obvious importance, mammalian synapse populations have so far resisted detailed quantitative study. In human brains and most animal nervous systems, synapses are very small and very densely packed: there are approximately 1 billion synapses per cubic millimeter of human cortex. This volumetric density poses very substantial challenges to proteometric analysis at the critical level of the individual synapse. The present work describes new probabilistic image analysis methods suitable for single-synapse analysis of synapse populations in both animal and human brains, in health and disorder.

Automated synaptic connectivity inference for volume electron microscopy

Teravoxel volume electron microscopy data sets from neural tissue can now be acquired in weeks, but data analysis requires years of manual labor. We developed the SyConn framework, which uses deep convolutional neural networks and random forest classifiers to infer a richly annotated synaptic connectivity matrix from manual neurite skeleton reconstructions by automatically identifying mitochondria, synapses and their types, axons, dendrites, spines, myelin, somata and cell types. We tested our approach on serial block-face electron microscopy data sets from zebrafish, mouse and zebra finch, and computed the synaptic wiring of songbird basal ganglia. We found that, for example, basal-ganglia cell types with high firing rates in vivo had higher densities of mitochondria and vesicles and that synapse sizes and quantities scaled systematically, depending on the innervated postsynaptic cell types.

Progress Towards Mammalian Whole-Brain Cellular Connectomics

Neurons are the fundamental structural units of the nervous system-i.e., the Neuron Doctrine-as the pioneering work of Santiago Ramón y Cajal in the 1880's clearly demonstrated through careful observation of Golgi-stained neuronal morphologies. However, at that time sample preparation, imaging methods and computational tools were either nonexistent or insufficiently developed to permit the precise mapping of an entire brain with all of its neurons and their connections. Some measure of the "mesoscopic" connectional organization of the mammalian brain has been obtained over the past decade by alignment of sparse subsets of labeled neurons onto a reference atlas or via MRI-based diffusion tensor imaging. Neither method, however, provides data on the complete connectivity of all neurons comprising an individual brain. Fortunately, whole-brain cellular connectomics now appears within reach due to recent advances in whole-brain sample preparation and high-throughput electron microscopy (EM), though substantial obstacles remain with respect to large volume electron microscopic acquisitions and automated neurite reconstructions. This perspective examines the current status and problems associated with generating a mammalian whole-brain cellular connectome and argues that the time is right to launch a concerted connectomic attack on a small mammalian whole-brain.