Automatic Reconstruction of Mitochondria and Endoplasmic Reticulum in Electron Microscopy Volumes by Deep Learning

Machine learning-based 3D segmentation of mitochondria in polarized epithelial cells

Mitochondria are dynamic organelles that alter their morphological characteristics in response to functional needs. Therefore, mitochondrial morphology is an important indicator of mitochondrial function and cellular health. Reliable segmentation of mitochondrial networks in microscopy images is a crucial initial step for further quantitative evaluation of their morphology. However, 3D mitochondrial segmentation, especially in cells with complex network morphology, such as in highly polarized cells, remains challenging. To improve the quality of 3D segmentation of mitochondria in super-resolution microscopy images, we took a machine learning approach, using 3D Trainable Weka, an ImageJ plugin. We demonstrated that, compared with other commonly used methods, our approach segmented mitochondrial networks effectively, with improved accuracy in different polarized epithelial cell models, including differentiated human retinal pigment epithelial (RPE) cells. Furthermore, using several tools for quantitative analysis following segmentation, we revealed mitochondrial fragmentation in bafilomycin-treated RPE cells.

Dysregulation of cellular membrane homeostasis as a crucial modulator of cancer risk

Cellular membranes serve as an epicentre combining extracellular and cytosolic components with membranous effectors, which together support numerous fundamental cellular signalling pathways that mediate biological responses. To execute their functions, membrane proteins, lipids and carbohydrates arrange, in a highly coordinated manner, into well-defined assemblies displaying diverse biological and biophysical characteristics that modulate several signalling events. The loss of membrane homeostasis can trigger oncogenic signalling. More recently, it has been documented that select membrane active dietaries (MADs) can reshape biological membranes and subsequently decrease cancer risk. In this review, we emphasize the significance of membrane domain structure, organization and their signalling functionalities as well as how loss of membrane homeostasis can steer aberrant signalling. Moreover, we describe in detail the complexities associated with the examination of these membrane domains and their association with cancer. Finally, we summarize the current literature on MADs and their effects on cellular membranes, including various mechanisms of dietary chemoprevention/interception and the functional links between nutritional bioactives, membrane homeostasis and cancer biology.

A semi-automatic method for extracting mitochondrial cristae characteristics from 3D focused ion beam scanning electron microscopy data

Mitochondria are the main suppliers of energy for cells and their bioenergetic function is regulated by mitochondrial dynamics: the constant changes in mitochondria size, shape, and cristae morphology to secure cell homeostasis. Although changes in mitochondrial function are implicated in a wide range of diseases, our understanding is challenged by a lack of reliable ways to extract spatial features from the cristae, the detailed visualization of which requires electron microscopy (EM). Here, we present a semi-automatic method for the segmentation, 3D reconstruction, and shape analysis of mitochondria, cristae, and intracristal spaces based on 2D EM images of the murine hippocampus. We show that our method provides a more accurate characterization of mitochondrial ultrastructure in 3D than common 2D approaches and propose an operational index of mitochondria's internal organization. With an improved consistency of 3D shape analysis and a decrease in the workload needed for large-scale analysis, we speculate that this tool will help increase our understanding of mitochondrial dynamics in health and disease.

Segmentation in large-scale cellular electron microscopy with deep learning: A literature survey

Electron microscopy (EM) enables high-resolution imaging of tissues and cells based on 2D and 3D imaging techniques. Due to the laborious and time-consuming nature of manual segmentation of large-scale EM datasets, automated segmentation approaches are crucial. This review focuses on the progress of deep learning-based segmentation techniques in large-scale cellular EM throughout the last six years, during which significant progress has been made in both semantic and instance segmentation. A detailed account is given for the key datasets that contributed to the proliferation of deep learning in 2D and 3D EM segmentation. The review covers supervised, unsupervised, and self-supervised learning methods and examines how these algorithms were adapted to the task of segmenting cellular and sub-cellular structures in EM images. The special challenges posed by such images, like heterogeneity and spatial complexity, and the network architectures that overcame some of them are described. Moreover, an overview of the evaluation measures used to benchmark EM datasets in various segmentation tasks is provided. Finally, an outlook of current trends and future prospects of EM segmentation is given, especially with large-scale models and unlabeled images to learn generic features across EM datasets.

ERnet: a tool for the semantic segmentation and quantitative analysis of endoplasmic reticulum topology

The ability to quantify structural changes of the endoplasmic reticulum (ER) is crucial for understanding the structure and function of this organelle. However, the rapid movement and complex topology of ER networks make this challenging. Here, we construct a state-of-the-art semantic segmentation method that we call ERnet for the automatic classification of sheet and tubular ER domains inside individual cells. Data are skeletonized and represented by connectivity graphs, enabling precise and efficient quantification of network connectivity. ERnet generates metrics on topology and integrity of ER structures and quantifies structural change in response to genetic or metabolic manipulation. We validate ERnet using data obtained by various ER-imaging methods from different cell types as well as ground truth images of synthetic ER structures. ERnet can be deployed in an automatic high-throughput and unbiased fashion and identifies subtle changes in ER phenotypes that may inform on disease progression and response to therapy.

Deep neural network automated segmentation of cellular structures in volume electron microscopy

Volume electron microscopy is an important imaging modality in contemporary cell biology. Identification of intracellular structures is a laborious process limiting the effective use of this potentially powerful tool. We resolved this bottleneck with automated segmentation of intracellular substructures in electron microscopy (ASEM), a new pipeline to train a convolutional neural network to detect structures of a wide range in size and complexity. We obtained dedicated models for each structure based on a small number of sparsely annotated ground truth images from only one or two cells. Model generalization was improved with a rapid, computationally effective strategy to refine a trained model by including a few additional annotations. We identified mitochondria, Golgi apparatus, endoplasmic reticulum, nuclear pore complexes, caveolae, clathrin-coated pits, and vesicles imaged by focused ion beam scanning electron microscopy. We uncovered a wide range of membrane-nuclear pore diameters within a single cell and derived morphological metrics from clathrin-coated pits and vesicles, consistent with the classical constant-growth assembly model.

Instance segmentation of mitochondria in electron microscopy images with a generalist deep learning model trained on a diverse dataset

Mitochondria are extremely pleomorphic organelles. Automatically annotating each one accurately and precisely in any 2D or volume electron microscopy (EM) image is an unsolved computational challenge. Current deep learning-based approaches train models on images that provide limited cellular contexts, precluding generality. To address this, we amassed a highly heterogeneous ∼1.5 × 10⁶ image 2D unlabeled cellular EM dataset and segmented ∼135,000 mitochondrial instances therein. MitoNet, a model trained on these resources, performs well on challenging benchmarks and on previously unseen volume EM datasets containing tens of thousands of mitochondria. We release a Python package and napari plugin, empanada, to rapidly run inference, visualize, and proofread instance segmentations. A record of this paper's transparent peer review process is included in the supplemental information.

A 3D analysis revealed complexe mitochondria morphologies in porcine cumulus cells

In the ovarian follicle, a bilateral cell-to-cell communication exists between the female germ cell and the cumulus cells which surround the oocyte. This communication allows the transit of small size molecules known to impact oocyte developmental competence. Pyruvate derivatives produced by mitochondria, are one of these transferred molecules. Interestingly, mitochondria may adopt a variety of morphologies to regulate their functions. In this study, we described mitochondrial morphologies in porcine cumulus cells. Active mitochondria were stained with TMRM (Tetramethylrhodamine, Methyl Ester, Perchlorate) and observed with 2D confocal microscopy showing mitochondria of different morphologies such as short, intermediate, long, and very long. The number of mitochondria of each phenotype was quantified in cells and the results showed that most cells contained elongated mitochondria. Scanning electron microscopy (SEM) analysis confirmed at nanoscale resolution the different mitochondrial morphologies including round, short, intermediate, and long. Interestingly, 3D visualisation by focused ion-beam scanning electron microscopy (FIB-SEM) revealed different complex mitochondrial morphologies including connected clusters of different sizes, branched mitochondria, as well as individual mitochondria. Since mitochondrial dynamics is a key regulator of function, the description of the mitochondrial network organisation will allow to further study mitochondrial dynamics in cumulus cells in response to various conditions such as in vitro maturation.

DeepContact: High-throughput quantification of membrane contact sites based on electron microscopy imaging

Membrane contact site (MCS)-mediated organelle interactions play essential roles in the cell. Quantitative analysis of MCSs reveals vital clues for cellular responses under various physiological and pathological conditions. However, an efficient tool is lacking. Here, we developed DeepContact, a deep-learning protocol for optimizing organelle segmentation and contact analysis based on label-free EM. DeepContact presents high efficiency and flexibility in interactive visualizations, accommodating new morphologies of organelles and recognizing contacts in versatile width ranges, which enables statistical analysis of various types of MCSs in multiple systems. DeepContact profiled previously unidentified coordinative rearrangements of MCS types in cultured cells with combined nutritional conditions. DeepContact also unveiled a subtle wave of ER-mitochondrial entanglement in Sertoli cells during the seminiferous epithelial cycle, indicating its potential in bridging MCS dynamics to physiological and pathological processes.

Automatic segmentation and reconstruction of intracellular compartments in volumetric electron microscopy data

BACKGROUND AND OBJECTIVES

In recent years, electron microscopy is enabling the acquisition of volumetric data with resolving power to directly observe the ultrastructure of intracellular compartments. New insights and knowledge about cell processes that are offered by such data require a comprehensive analysis which is limited by the time-consuming manual segmentation and reconstruction methods.

METHOD

We present methods for automatic segmentation, reconstruction, and analysis of intracellular compartments from volumetric data obtained by the dual-beam electron microscopy. We specifically address segmentation of fusiform vesicles and the Golgi apparatus, reconstruction of mitochondria and fusiform vesicles, and morphological analysis of the reconstructed mitochondria.

RESULTS AND CONCLUSION

Evaluation on the public UroCell dataset demonstrated high accuracy of the proposed methods for segmentation of fusiform vesicles and the Golgi apparatus, as well as for reconstruction of mitochondria and analysis of their shapes, while reconstruction of fusiform vesicles proved to be more challenging. We published an extension of the UroCell dataset with all of the data used in this work, to further contribute to research on automatic analysis of the ultrastructure of intracellular compartments.

VGG-UNet/VGG-SegNet Supported Automatic Segmentation of Endoplasmic Reticulum Network in Fluorescence Microscopy Images

This research work aims to implement an automated segmentation process to extract the endoplasmic reticulum (ER) network in fluorescence microscopy images (FMI) using pretrained convolutional neural network (CNN). The threshold level of the raw FMT is complex, and extraction of the ER network is a challenging task. Hence, an image conversion procedure is initially employed to reduce its complexity. This work employed the pretrained CNN schemes, such as VGG-UNet and VGG-SegNet, to mine the ER network from the chosen FMI test images. The proposed ER segmentation pipeline consists of the following phases; (i) clinical image collection, 16-bit to 8-bit conversion and resizing; (ii) implementation of pretrained VGG-UNet and VGG-SegNet; (iii) extraction of the binary form of ER network; (iv) comparing the mined ER with ground-truth; and (v) computation of image measures and validation. The considered FMI dataset consists of 223 test images, and image augmentation is then implemented to increase these images. The result of this scheme is then confirmed against other CNN methods, such as U-Net, SegNet, and Res-UNet. The experimental outcome confirms a segmentation accuracy of >98% with VGG-UNet and VGG-SegNet. The results of this research authenticate that the proposed pipeline can be considered to examine the clinical-grade FMI.

Integrated Array Tomography for 3D Correlative Light and Electron Microscopy

Volume electron microscopy (EM) of biological systems has grown exponentially in recent years due to innovative large-scale imaging approaches. As a standalone imaging method, however, large-scale EM typically has two major limitations: slow rates of acquisition and the difficulty to provide targeted biological information. We developed a 3D image acquisition and reconstruction pipeline that overcomes both of these limitations by using a widefield fluorescence microscope integrated inside of a scanning electron microscope. The workflow consists of acquiring large field of view fluorescence microscopy (FM) images, which guide to regions of interest for successive EM (integrated correlative light and electron microscopy). High precision EM-FM overlay is achieved using cathodoluminescent markers. We conduct a proof-of-concept of our integrated workflow on immunolabelled serial sections of tissues. Acquisitions are limited to regions containing biological targets, expediting total acquisition times and reducing the burden of excess data by tens or hundreds of GBs.

Quantifying Regulated Mitochondrial Fission in Macrophages

Mitochondria have co-evolved with eukaryotic cells for more than a billion years, becoming an important cog in their machinery. They are best known for being tasked with energy generation through the production of adenosine triphosphate, but they also have roles in several other cellular processes, for example, immune and inflammatory responses. Mitochondria have important functions in macrophages, key innate immune cells that detect pathogens and drive inflammation. Mitochondrial activity is influenced by the highly dynamic nature of the mitochondrial network, which alternates between interconnected tubular and fragmented forms. The dynamic balance between this interconnected fused network and fission-mediated mitochondrial fragmentation modulates inflammatory responses such as production of cytokines and mitochondrial reactive oxygen species. Here we describe methods to differentiate mouse bone marrow cells into macrophages and the use of light microscopy, electron microscopy, flow cytometry, and Western blotting to quantify regulated mitochondrial dynamics in these differentiated macrophages.

Research Techniques Made Simple: Volume Scanning Electron Microscopy

Volume scanning electron microscopy (VSEM) involves the serial sectioning and imaging of a sample using scanning electron microscopy (SEM), followed by segmentation and three-dimensional (3D) reconstruction using computer software packages to allow visualization of 3D structures. VSEM can reveal qualitative and quantitative properties of organelles and cells within tissues at nanoscale. The ability to visualize spatial relationships of structures of interest within and across cells in 3D space in particular sets VSEM apart from conventional SEM and transmission electron microscopy. Here, we provide an overview of VSEM platforms and image processing, highlighting characteristics that will aid selection of a method to address specific research questions in dermatological research.

Contrastive Learning for Mitochondria Segmentation

Mitochondria segmentation in electron microscopy images is essential in neuroscience. However, due to the image degradation during the imaging process, the large variety of mitochondrial structures, as well as the presence of noise, artifacts and other sub-cellular structures, mitochondria segmentation is very challenging. In this paper, we propose a novel and effective contrastive learning framework to learn a better feature representation from hard examples to improve segmentation. Specifically, we adopt a point sampling strategy to pick out representative pixels from hard examples in the training phase. Based on these sampled pixels, we introduce a pixel-wise label-based contrastive loss which consists of a similarity loss term and a consistency loss term. The similarity term can increase the similarity of pixels from the same class and the separability of pixels from different classes in feature space, while the consistency term is able to enhance the sensitivity of the 3D model to changes in image content from frame to frame. We demonstrate the effectiveness of our method on MitoEM dataset as well as FIB-SEM dataset and show better or on par with state-of-the-art results.

Whole-cell organelle segmentation in volume electron microscopy

Cells contain hundreds of organelles and macromolecular assemblies. Obtaining a complete understanding of their intricate organization requires the nanometre-level, three-dimensional reconstruction of whole cells, which is only feasible with robust and scalable automatic methods. Here, to support the development of such methods, we annotated up to 35 different cellular organelle classes-ranging from endoplasmic reticulum to microtubules to ribosomes-in diverse sample volumes from multiple cell types imaged at a near-isotropic resolution of 4 nm per voxel with focused ion beam scanning electron microscopy (FIB-SEM)¹. We trained deep learning architectures to segment these structures in 4 nm and 8 nm per voxel FIB-SEM volumes, validated their performance and showed that automatic reconstructions can be used to directly quantify previously inaccessible metrics including spatial interactions between cellular components. We also show that such reconstructions can be used to automatically register light and electron microscopy images for correlative studies. We have created an open data and open-source web repository, 'OpenOrganelle', to share the data, computer code and trained models, which will enable scientists everywhere to query and further improve automatic reconstruction of these datasets.

Three-dimensional ATUM-SEM reconstruction and analysis of hepatic endoplasmic reticulum‒organelle interactions

The endoplasmic reticulum (ER) is a contiguous and complicated membrane network in eukaryotic cells, and membrane contact sites (MCSs) between the ER and other organelles perform vital cellular functions, including lipid homeostasis, metabolite exchange, calcium level regulation, and organelle division. Here, we establish a whole pipeline to reconstruct all ER, mitochondria, lipid droplets, lysosomes, peroxisomes, and nuclei by automated tape-collecting ultramicrotome scanning electron microscopy and deep learning techniques, which generates an unprecedented 3D model for mapping liver samples. Furthermore, the morphology of various organelles and the MCSs between the ER and other organelles are systematically analyzed. We found that the ER presents with predominantly flat cisternae and is knitted tightly all throughout the intracellular space and around other organelles. In addition, the ER has a smaller volume-to-membrane surface area ratio than other organelles, which suggests that the ER could be more suited for functions that require a large membrane surface area. Our data also indicate that ER‒mitochondria contacts are particularly abundant, especially for branched mitochondria. Our study provides 3D reconstructions of various organelles in liver samples together with important fundamental information for biochemical and functional studies in the liver.

Interorganelle communication, aging, and neurodegeneration

Our cells are comprised of billions of proteins, lipids, and other small molecules packed into their respective subcellular organelles, with the daunting task of maintaining cellular homeostasis over a lifetime. However, it is becoming increasingly evident that organelles do not act as autonomous discrete units but rather as interconnected hubs that engage in extensive communication through membrane contacts. In the last few years, our understanding of how these contacts coordinate organelle function has redefined our view of the cell. This review aims to present novel findings on the cellular interorganelle communication network and how its dysfunction may contribute to aging and neurodegeneration. The consequences of disturbed interorganellar communication are intimately linked with age-related pathologies. Given that both aging and neurodegenerative diseases are characterized by the concomitant failure of multiple cellular pathways, coordination of organelle communication and function could represent an emerging regulatory mechanism critical for long-term cellular homeostasis. We anticipate that defining the relationships between interorganelle communication, aging, and neurodegeneration will open new avenues for therapeutics.

Deep Learning-Based Classification of GAD67-Positive Neurons Without the Immunosignal

Excitatory neurons and GABAergic interneurons constitute neural circuits and play important roles in information processing. In certain brain regions, such as the neocortex and the hippocampus, there are fewer interneurons than excitatory neurons. Interneurons have been quantified via immunohistochemistry, for example, for GAD67, an isoform of glutamic acid decarboxylase. Additionally, the expression level of other proteins varies among cell types. For example, NeuN, a commonly used marker protein for postmitotic neurons, is expressed differently across brain regions and cell classes. Thus, we asked whether GAD67-immunopositive neurons can be detected using the immunofluorescence signals of NeuN and the fluorescence signals of Nissl substances. To address this question, we stained neurons in layers 2/3 of the primary somatosensory cortex (S1) and the primary motor cortex (M1) of mice and manually labeled the neurons as either cell type using GAD67 immunosignals. We then sought to detect GAD67-positive neurons without GAD67 immunosignals using a custom-made deep learning-based algorithm. Using this deep learning-based model, we succeeded in the binary classification of the neurons using Nissl and NeuN signals without referring to the GAD67 signals. Furthermore, we confirmed that our deep learning-based method surpassed classic machine-learning methods in terms of binary classification performance. Combined with the visualization of the hidden layer of our deep learning algorithm, our model provides a new platform for identifying unbiased criteria for cell-type classification.