A workflow for the automatic segmentation of organelles in electron microscopy image stacks

Modular segmentation, spatial analysis and visualization of volume electron microscopy datasets

Volume electron microscopy is the method of choice for the in situ interrogation of cellular ultrastructure at the nanometer scale, and with the increase in large raw image datasets generated, improving computational strategies for image segmentation and spatial analysis is necessary. Here we describe a practical and annotation-efficient pipeline for organelle-specific segmentation, spatial analysis and visualization of large volume electron microscopy datasets using freely available, user-friendly software tools that can be run on a single standard workstation. The procedures are aimed at researchers in the life sciences with modest computational expertise, who use volume electron microscopy and need to generate three-dimensional (3D) segmentation labels for different types of cell organelles while minimizing manual annotation efforts, to analyze the spatial interactions between organelle instances and to visualize the 3D segmentation results. We provide detailed guidelines for choosing well-suited segmentation tools for specific cell organelles, and to bridge compatibility issues between freely available open-source tools, we distribute the critical steps as easily installable Album solutions for deep learning segmentation, spatial analysis and 3D rendering. Our detailed description can serve as a reference for similar projects requiring particular strategies for single- or multiple-organelle analysis, which can be achieved with computational resources commonly available to single-user setups.

Segmentation of cellular ultrastructures on sparsely labeled 3D electron microscopy images using deep learning

Focused ion beam-scanning electron microscopy (FIB-SEM) images can provide a detailed view of the cellular ultrastructure of tumor cells. A deeper understanding of their organization and interactions can shed light on cancer mechanisms and progression. However, the bottleneck in the analysis is the delineation of the cellular structures to enable quantitative measurements and analysis. We mitigated this limitation using deep learning to segment cells and subcellular ultrastructure in 3D FIB-SEM images of tumor biopsies obtained from patients with metastatic breast and pancreatic cancers. The ultrastructures, such as nuclei, nucleoli, mitochondria, endosomes, and lysosomes, are relatively better defined than their surroundings and can be segmented with high accuracy using a neural network trained with sparse manual labels. Cell segmentation, on the other hand, is much more challenging due to the lack of clear boundaries separating cells in the tissue. We adopted a multi-pronged approach combining detection, boundary propagation, and tracking for cell segmentation. Specifically, a neural network was employed to detect the intracellular space; optical flow was used to propagate cell boundaries across the z-stack from the nearest ground truth image in order to facilitate the separation of individual cells; finally, the filopodium-like protrusions were tracked to the main cells by calculating the intersection over union measure for all regions detected in consecutive images along z-stack and connecting regions with maximum overlap. The proposed cell segmentation methodology resulted in an average Dice score of 0.93. For nuclei, nucleoli, and mitochondria, the segmentation achieved Dice scores of 0.99, 0.98, and 0.86, respectively. The segmentation of FIB-SEM images will enable interpretative rendering and provide quantitative image features to be associated with relevant clinical variables.

Efficient semi-supervised semantic segmentation of electron microscopy cancer images with sparse annotations

Electron microscopy (EM) enables imaging at a resolution of nanometers and can shed light on how cancer evolves to develop resistance to therapy. Acquiring these images has become a routine task.However, analyzing them is now a bottleneck, as manual structure identification is very time-consuming and can take up to several months for a single sample. Deep learning approaches offer a suitable solution to speed up the analysis. In this work, we present a study of several state-of-the-art deep learning models for the task of segmenting nuclei and nucleoli in volumes from tumor biopsies. We compared previous results obtained with the ResUNet architecture to the more recent UNet++, FracTALResNet, SenFormer, and CEECNet models. In addition, we explored the utilization of unlabeled images through semi-supervised learning with Cross Pseudo Supervision. We have trained and evaluated all of the models on sparse manual labels from three fully annotated in-house datasets that we have made available on demand, demonstrating improvements in terms of 3D Dice score. From the analysis of these results, we drew conclusions on the relative gains of using more complex models, and semi-supervised learning as well as the next steps for the mitigation of the manual segmentation bottleneck.

Segmentation in large-scale cellular electron microscopy with deep learning: A literature survey

Electron microscopy (EM) enables high-resolution imaging of tissues and cells based on 2D and 3D imaging techniques. Due to the laborious and time-consuming nature of manual segmentation of large-scale EM datasets, automated segmentation approaches are crucial. This review focuses on the progress of deep learning-based segmentation techniques in large-scale cellular EM throughout the last six years, during which significant progress has been made in both semantic and instance segmentation. A detailed account is given for the key datasets that contributed to the proliferation of deep learning in 2D and 3D EM segmentation. The review covers supervised, unsupervised, and self-supervised learning methods and examines how these algorithms were adapted to the task of segmenting cellular and sub-cellular structures in EM images. The special challenges posed by such images, like heterogeneity and spatial complexity, and the network architectures that overcame some of them are described. Moreover, an overview of the evaluation measures used to benchmark EM datasets in various segmentation tasks is provided. Finally, an outlook of current trends and future prospects of EM segmentation is given, especially with large-scale models and unlabeled images to learn generic features across EM datasets.

Lossy Image Compression in a Preclinical Multimodal Imaging Study

The growing use of multimodal high-resolution volumetric data in pre-clinical studies leads to challenges related to the management and handling of the large amount of these datasets. Contrarily to the clinical context, currently there are no standard guidelines to regulate the use of image compression in pre-clinical contexts as a potential alleviation of this problem. In this work, the authors study the application of lossy image coding to compress high-resolution volumetric biomedical data. The impact of compression on the metrics and interpretation of volumetric data was quantified for a correlated multimodal imaging study to characterize murine tumor vasculature, using volumetric high-resolution episcopic microscopy (HREM), micro-computed tomography (µCT), and micro-magnetic resonance imaging (µMRI). The effects of compression were assessed by measuring task-specific performances of several biomedical experts who interpreted and labeled multiple data volumes compressed at different degrees. We defined trade-offs between data volume reduction and preservation of visual information, which ensured the preservation of relevant vasculature morphology at maximum compression efficiency across scales. Using the Jaccard Index (JI) and the average Hausdorff Distance (HD) after vasculature segmentation, we could demonstrate that, in this study, compression that yields to a 256-fold reduction of the data size allowed to keep the error induced by compression below the inter-observer variability, with minimal impact on the assessment of the tumor vasculature across scales.

Do perineuronal nets stabilize the engram of a synaptic circuit?

Perineuronal nets (PNN), a specialized form of ECM (?), surround numerous neurons in the CNS and allow synaptic connectivity through holes in its structure. We hypothesis that PNNs serve as gatekeepers that guard and protect synaptic territory, and thus may stabilize an engram circuit. We present high-resolution, and 3D EM images of PNN- engulfed neurons showing that synapses occupy the PNN holes, and that invasion of other cellular components are rare. PNN constituents are long-lived and can be eroded faster in an enriched environment, while synaptic proteins have high turnover rate. Preventing PNN erosion by using pharmacological inhibition of PNN-modifying proteases or MMP9 knockout mice allowed normal fear memory acquisition but diminished remote-memory stabilization, supporting the above hypothesis.

Significance

In this multidisciplinary work, we challenge the hypothesis that the pattern of holes in the perineuronal nets (PNN) hold the code for very-long-term memories. The scope of this work might lead us closer to the understanding of how we can vividly remember events from childhood to death bed. We postulate that the PNN holes hold the code for the engram. To test this hypothesis, we used three independent experimental strategies; high-resolution 3D electron microscopy, Stable Isotop Labeling in Mammals (SILAM) for proteins longevity, and pharmacologically and genetically interruption of memory consolidation in fear conditioning experiments. All of these experimental results did not dispute the PNN hypothesis.

Cryo-EM tomography and automatic segmentation delineate modular structures in the postsynaptic density

Postsynaptic densities (PSDs) are large protein complexes associated with the postsynaptic membrane of excitatory synapses important for synaptic function including plasticity. Conventional electron microscopy (EM) typically depicts PSDs as compact disk-like structures of hundreds of nanometers in size. Biochemically isolated PSDs were also similar in dimension revealing a predominance of proteins with the ability to polymerize into an extensive scaffold; several EM studies noted their irregular contours with often small granular structures (<30 nm) and holes. Super-resolution light microscopy studies observed clusters of PSD elements and their activity-induced lateral movement. Furthermore, our recent EM study on PSD fractions after sonication observed PSD fragments (40–90 nm in size) separate from intact PSDs; however, such structures within PSDs remained unidentified. Here we examined isolated PSDs by cryo-EM tomography with our new approach of automatic segmentation that enables delineation of substructures and their quantitative analysis. The delineated substructures broadly varied in size, falling behind 30 nm or exceeding 100 nm and showed that a considerable portion of the substructures (>38%) in isolated PSDs was in the same size range as those fragments. Furthermore, substructures spanning the entire thickness of the PSD were found, large enough to contain both membrane-associated and cytoplasmic proteins of the PSD; interestingly, they were similar to nanodomains in frequency. The structures detected here appear to constitute the isolated PSD as modules of various compositions, and this modular nature may facilitate remodeling of the PSD for proper synaptic function and plasticity.

3D reconstruction of the cerebellar germinal layer reveals tunneling connections between developing granule cells

The difficulty of retrieving high-resolution, in vivo evidence of the proliferative and migratory processes occurring in neural germinal zones has limited our understanding of neurodevelopmental mechanisms. Here, we used a connectomic approach using a high-resolution, serial-sectioning scanning electron microscopy volume to investigate the laminar cytoarchitecture of the transient external granular layer (EGL) of the developing cerebellum, where granule cells coordinate a series of mitotic and migratory events. By integrating image segmentation, three-dimensional reconstruction, and deep-learning approaches, we found and characterized anatomically complex intercellular connections bridging pairs of cerebellar granule cells throughout the EGL. Connected cells were either mitotic, migratory, or transitioning between these two cell stages, displaying a chronological continuum of proliferative and migratory events never previously observed in vivo at this resolution. This unprecedented ultrastructural characterization poses intriguing hypotheses about intercellular connectivity between developing progenitors and its possible role in the development of the central nervous system.

Spatial mapping of mitochondrial networks and bioenergetics in lung cancer

Mitochondria are critical to the governance of metabolism and bioenergetics in cancer cells1. The mitochondria form highly organized networks, in which their outer and inner membrane structures define their bioenergetic capacity2,3. However, in vivo studies delineating the relationship between the structural organization of mitochondrial networks and their bioenergetic activity have been limited. Here we present an in vivo structural and functional analysis of mitochondrial networks and bioenergetic phenotypes in non-small cell lung cancer (NSCLC) using an integrated platform consisting of positron emission tomography imaging, respirometry and three-dimensional scanning block-face electron microscopy. The diverse bioenergetic phenotypes and metabolic dependencies we identified in NSCLC tumours align with distinct structural organization of mitochondrial networks present. Further, we discovered that mitochondrial networks are organized into distinct compartments within tumour cells. In tumours with high rates of oxidative phosphorylation (OXPHOSHI) and fatty acid oxidation, we identified peri-droplet mitochondrial networks wherein mitochondria contact and surround lipid droplets. By contrast, we discovered that in tumours with low rates of OXPHOS (OXPHOSLO), high glucose flux regulated perinuclear localization of mitochondria, structural remodelling of cristae and mitochondrial respiratory capacity. Our findings suggest that in NSCLC, mitochondrial networks are compartmentalized into distinct subpopulations that govern the bioenergetic capacity of tumours.

Impact of Training Data, Ground Truth and Shape Variability in the Deep Learning-Based Semantic Segmentation of HeLa Cells Observed with Electron Microscopy

This paper investigates the impact of the amount of training data and the shape variability on the segmentation provided by the deep learning architecture U-Net. Further, the correctness of ground truth (GT) was also evaluated. The input data consisted of a three-dimensional set of images of HeLa cells observed with an electron microscope with dimensions 8192×8192×517. From there, a smaller region of interest (ROI) of 2000×2000×300 was cropped and manually delineated to obtain the ground truth necessary for a quantitative evaluation. A qualitative evaluation was performed on the 8192×8192 slices due to the lack of ground truth. Pairs of patches of data and labels for the classes nucleus, nuclear envelope, cell and background were generated to train U-Net architectures from scratch. Several training strategies were followed, and the results were compared against a traditional image processing algorithm. The correctness of GT, that is, the inclusion of one or more nuclei within the region of interest was also evaluated. The impact of the extent of training data was evaluated by comparing results from 36,000 pairs of data and label patches extracted from the odd slices in the central region, to 135,000 patches obtained from every other slice in the set. Then, 135,000 patches from several cells from the 8192×8192 slices were generated automatically using the image processing algorithm. Finally, the two sets of 135,000 pairs were combined to train once more with 270,000 pairs. As would be expected, the accuracy and Jaccard similarity index improved as the number of pairs increased for the ROI. This was also observed qualitatively for the 8192×8192 slices. When the 8192×8192 slices were segmented with U-Nets trained with 135,000 pairs, the architecture trained with automatically generated pairs provided better results than the architecture trained with the pairs from the manually segmented ground truths. This suggests that the pairs that were extracted automatically from many cells provided a better representation of the four classes of the various cells in the 8192×8192 slice than those pairs that were manually segmented from a single cell. Finally, the two sets of 135,000 pairs were combined, and the U-Net trained with these provided the best results.

Instance segmentation of mitochondria in electron microscopy images with a generalist deep learning model trained on a diverse dataset

Mitochondria are extremely pleomorphic organelles. Automatically annotating each one accurately and precisely in any 2D or volume electron microscopy (EM) image is an unsolved computational challenge. Current deep learning-based approaches train models on images that provide limited cellular contexts, precluding generality. To address this, we amassed a highly heterogeneous ∼1.5 × 10⁶ image 2D unlabeled cellular EM dataset and segmented ∼135,000 mitochondrial instances therein. MitoNet, a model trained on these resources, performs well on challenging benchmarks and on previously unseen volume EM datasets containing tens of thousands of mitochondria. We release a Python package and napari plugin, empanada, to rapidly run inference, visualize, and proofread instance segmentations. A record of this paper's transparent peer review process is included in the supplemental information.

Electron microscopy of cardiac 3D nanodynamics: form, function, future

The 3D nanostructure of the heart, its dynamic deformation during cycles of contraction and relaxation, and the effects of this deformation on cell function remain largely uncharted territory. Over the past decade, the first inroads have been made towards 3D reconstruction of heart cells, with a native resolution of around 1 nm³, and of individual molecules relevant to heart function at a near-atomic scale. These advances have provided access to a new generation of data and have driven the development of increasingly smart, artificial intelligence-based, deep-learning image-analysis algorithms. By high-pressure freezing of cardiomyocytes with millisecond accuracy after initiation of an action potential, pseudodynamic snapshots of contraction-induced deformation of intracellular organelles can now be captured. In combination with functional studies, such as fluorescence imaging, exciting insights into cardiac autoregulatory processes at nano-to-micro scales are starting to emerge. In this Review, we discuss the progress in this fascinating new field to highlight the fundamental scientific insight that has emerged, based on technological breakthroughs in biological sample preparation, 3D imaging and data analysis; to illustrate the potential clinical relevance of understanding 3D cardiac nanodynamics; and to predict further progress that we can reasonably expect to see over the next 10 years.

Reconstruction of neocortex: Organelles, compartments, cells, circuits, and activity

We assembled a semi-automated reconstruction of L2/3 mouse primary visual cortex from ∼250 × 140 × 90 μm3 of electron microscopic images, including pyramidal and non-pyramidal neurons, astrocytes, microglia, oligodendrocytes and precursors, pericytes, vasculature, nuclei, mitochondria, and synapses. Visual responses of a subset of pyramidal cells are included. The data are publicly available, along with tools for programmatic and three-dimensional interactive access. Brief vignettes illustrate the breadth of potential applications relating structure to function in cortical circuits and neuronal cell biology. Mitochondria and synapse organization are characterized as a function of path length from the soma. Pyramidal connectivity motif frequencies are predicted accurately using a configuration model of random graphs. Pyramidal cells receiving more connections from nearby cells exhibit stronger and more reliable visual responses. Sample code shows data access and analysis.

Whole-cell organelle segmentation in volume electron microscopy

Cells contain hundreds of organelles and macromolecular assemblies. Obtaining a complete understanding of their intricate organization requires the nanometre-level, three-dimensional reconstruction of whole cells, which is only feasible with robust and scalable automatic methods. Here, to support the development of such methods, we annotated up to 35 different cellular organelle classes-ranging from endoplasmic reticulum to microtubules to ribosomes-in diverse sample volumes from multiple cell types imaged at a near-isotropic resolution of 4 nm per voxel with focused ion beam scanning electron microscopy (FIB-SEM)¹. We trained deep learning architectures to segment these structures in 4 nm and 8 nm per voxel FIB-SEM volumes, validated their performance and showed that automatic reconstructions can be used to directly quantify previously inaccessible metrics including spatial interactions between cellular components. We also show that such reconstructions can be used to automatically register light and electron microscopy images for correlative studies. We have created an open data and open-source web repository, 'OpenOrganelle', to share the data, computer code and trained models, which will enable scientists everywhere to query and further improve automatic reconstruction of these datasets.

A Close Look at Molecular Self-Assembly with the Transmission Electron Microscope

Molecular self-assembly is pervasive in the formation of living and synthetic materials. Knowledge gained from research into the principles of molecular self-assembly drives innovation in the biological, chemical, and materials sciences. Self-assembly processes span a wide range of temporal and spatial domains and are often unintuitive and complex. Studying such complex processes requires an arsenal of analytical and computational tools. Within this arsenal, the transmission electron microscope stands out for its unique ability to visualize and quantify self-assembly structures and processes. This review describes the contribution that the transmission electron microscope has made to the field of molecular self-assembly. An emphasis is placed on which TEM methods are applicable to different structures and processes and how TEM can be used in combination with other experimental or computational methods. Finally, we provide an outlook on the current challenges to, and opportunities for, increasing the impact that the transmission electron microscope can have on molecular self-assembly.

Deep learning for automatic segmentation of the nuclear envelope in electron microscopy data, trained with volunteer segmentations

Advancements in volume electron microscopy mean it is now possible to generate thousands of serial images at nanometre resolution overnight, yet the gold standard approach for data analysis remains manual segmentation by an expert microscopist, resulting in a critical research bottleneck. Although some machine learning approaches exist in this domain, we remain far from realizing the aspiration of a highly accurate, yet generic, automated analysis approach, with a major obstacle being lack of sufficient high-quality ground-truth data. To address this, we developed a novel citizen science project, Etch a Cell, to enable volunteers to manually segment the nuclear envelope (NE) of HeLa cells imaged with serial blockface scanning electron microscopy. We present our approach for aggregating multiple volunteer annotations to generate a high-quality consensus segmentation and demonstrate that data produced exclusively by volunteers can be used to train a highly accurate machine learning algorithm for automatic segmentation of the NE, which we share here, in addition to our archived benchmark data.

Volumetric Semantic Instance Segmentation of the Plasma Membrane of HeLa Cells

In this work, an unsupervised volumetric semantic instance segmentation of the plasma membrane of HeLa cells as observed with serial block face scanning electron microscopy is described. The resin background of the images was segmented at different slices of a 3D stack of 518 slices with 8192 × 8192 pixels each. The background was used to create a distance map, which helped identify and rank the cells by their size at each slice. The centroids of the cells detected at different slices were linked to identify them as a single cell that spanned a number of slices. A subset of these cells, i.e., the largest ones and those not close to the edges were selected for further processing. The selected cells were then automatically cropped to smaller regions of interest of 2000 × 2000 × 300 voxels that were treated as cell instances. Then, for each of these volumes, the nucleus was segmented, and the cell was separated from any neighbouring cells through a series of traditional image processing steps that followed the plasma membrane. The segmentation process was repeated for all the regions of interest previously selected. For one cell for which the ground truth was available, the algorithm provided excellent results in Accuracy (AC) and the Jaccard similarity Index (JI): nucleus: JI =0.9665, AC =0.9975, cell including nucleus JI =0.8711, AC =0.9655, cell excluding nucleus JI =0.8094, AC =0.9629. A limitation of the algorithm for the plasma membrane segmentation was the presence of background. In samples with tightly packed cells, this may not be available. When tested for these conditions, the segmentation of the nuclear envelope was still possible. All the code and data were released openly through GitHub, Zenodo and EMPIAR.

CEM500K, a large-scale heterogeneous unlabeled cellular electron microscopy image dataset for deep learning

Automated segmentation of cellular electron microscopy (EM) datasets remains a challenge. Supervised deep learning (DL) methods that rely on region-of-interest (ROI) annotations yield models that fail to generalize to unrelated datasets. Newer unsupervised DL algorithms require relevant pre-training images, however, pre-training on currently available EM datasets is computationally expensive and shows little value for unseen biological contexts, as these datasets are large and homogeneous. To address this issue, we present CEM500K, a nimble 25 GB dataset of 0.5 × 106 unique 2D cellular EM images curated from nearly 600 three-dimensional (3D) and 10,000 two-dimensional (2D) images from >100 unrelated imaging projects. We show that models pre-trained on CEM500K learn features that are biologically relevant and resilient to meaningful image augmentations. Critically, we evaluate transfer learning from these pre-trained models on six publicly available and one newly derived benchmark segmentation task and report state-of-the-art results on each. We release the CEM500K dataset, pre-trained models and curation pipeline for model building and further expansion by the EM community. Data and code are available at https://www.ebi.ac.uk/pdbe/emdb/empiar/entry/10592/ and https://git.io/JLLTz.

HIVE-Net: Centerline-aware hierarchical view-ensemble convolutional network for mitochondria segmentation in EM images

BACKGROUND AND OBJECTIVE

With the advancement of electron microscopy (EM) imaging technology, neuroscientists can investigate the function of various intracellular organelles, e.g, mitochondria, at nano-scale. Semantic segmentation of electron microscopy (EM) is an essential step to efficiently obtain reliable morphological statistics. Despite the great success achieved using deep convolutional neural networks (CNNs), they still produce coarse segmentations with lots of discontinuities and false positives for mitochondria segmentation.

METHODS

In this study, we introduce a centerline-aware multitask network by utilizing centerline as an intrinsic shape cue of mitochondria to regularize the segmentation. Since the application of 3D CNNs on large medical volumes is usually hindered by their substantial computational cost and storage overhead, we introduce a novel hierarchical view-ensemble convolution (HVEC), a simple alternative of 3D convolution to learn 3D spatial contexts using more efficient 2D convolutions. The HVEC enables both decomposing and sharing multi-view information, leading to increased learning capacity.

RESULTS

Extensive validation results on two challenging benchmarks show that, the proposed method performs favorably against the state-of-the-art methods in accuracy and visual quality but with a greatly reduced model size. Moreover, the proposed model also shows significantly improved generalization ability, especially when training with quite limited amount of training data. Detailed sensitivity analysis and ablation study have also been conducted, which show the robustness of the proposed model and effectiveness of the proposed modules.

CONCLUSIONS

The experiments highlighted that the proposed architecture enables both simplicity and efficiency leading to increased capacity of learning spatial contexts. Moreover, incorporating shape cues such as centerline information is a promising approach to improve the performance of mitochondria segmentation.

Brain Ultrastructure: Putting the Pieces Together

Unraveling the fine structure of the brain is important to provide a better understanding of its normal and abnormal functioning. Application of high-resolution electron microscopic techniques gives us an unprecedented opportunity to discern details of the brain parenchyma at nanoscale resolution, although identifying different cell types and their unique features in two-dimensional, or three-dimensional images, remains a challenge even to experts in the field. This article provides insights into how to identify the different cell types in the central nervous system, based on nuclear and cytoplasmic features, amongst other unique characteristics. From the basic distinction between neurons and their supporting cells, the glia, to differences in their subcellular compartments, organelles and their interactions, ultrastructural analyses can provide unique insights into the changes in brain function during aging and disease conditions, such as stroke, neurodegeneration, infection and trauma. Brain parenchyma is composed of a dense mixture of neuronal and glial cell bodies, together with their intertwined processes. Intracellular components that vary between cells, and can become altered with aging or disease, relate to the cytoplasmic and nucleoplasmic density, nuclear heterochromatin pattern, mitochondria, endoplasmic reticulum and Golgi complex, lysosomes, neurosecretory vesicles, and cytoskeletal elements (actin, intermediate filaments, and microtubules). Applying immunolabeling techniques to visualize membrane-bound or intracellular proteins in neurons and glial cells gives an even better appreciation of the subtle differences unique to these cells across contexts of health and disease. Together, our observations reveal how simple ultrastructural features can be used to identify specific changes in cell types, their health status, and functional relationships in the brain.

Brain Ultrastructure: Putting the Pieces Together

Unraveling the fine structure of the brain is important to provide a better understanding of its normal and abnormal functioning. Application of high-resolution electron microscopic techniques gives us an unprecedented opportunity to discern details of the brain parenchyma at nanoscale resolution, although identifying different cell types and their unique features in two-dimensional, or three-dimensional images, remains a challenge even to experts in the field. This article provides insights into how to identify the different cell types in the central nervous system, based on nuclear and cytoplasmic features, amongst other unique characteristics. From the basic distinction between neurons and their supporting cells, the glia, to differences in their subcellular compartments, organelles and their interactions, ultrastructural analyses can provide unique insights into the changes in brain function during aging and disease conditions, such as stroke, neurodegeneration, infection and trauma. Brain parenchyma is composed of a dense mixture of neuronal and glial cell bodies, together with their intertwined processes. Intracellular components that vary between cells, and can become altered with aging or disease, relate to the cytoplasmic and nucleoplasmic density, nuclear heterochromatin pattern, mitochondria, endoplasmic reticulum and Golgi complex, lysosomes, neurosecretory vesicles, and cytoskeletal elements (actin, intermediate filaments, and microtubules). Applying immunolabeling techniques to visualize membrane-bound or intracellular proteins in neurons and glial cells gives an even better appreciation of the subtle differences unique to these cells across contexts of health and disease. Together, our observations reveal how simple ultrastructural features can be used to identify specific changes in cell types, their health status, and functional relationships in the brain.

Digital postprocessing and image segmentation for objective analysis of colorimetric reactions

Recently, there has been an explosion of scientific literature describing the use of colorimetry for monitoring the progression or the endpoint result of colorimetric reactions. The availability of inexpensive imaging technology (e.g., scanners, Raspberry Pi, smartphones and other sub-$50 digital cameras) has lowered the barrier to accessing cost-efficient, objective detection methodologies. However, to exploit these imaging devices as low-cost colorimetric detectors, it is paramount that they interface with flexible software that is capable of image segmentation and probing a variety of color spaces (RGB, HSB, Y'UV, L*a*b*, etc.). Development of tailor-made software (e.g., smartphone applications) for advanced image analysis requires complex, custom-written processing algorithms, advanced computer programming knowledge and/or expertise in physics, mathematics, pattern recognition and computer vision and learning. Freeware programs, such as ImageJ, offer an alternative, affordable path to robust image analysis. Here we describe a protocol that uses the ImageJ program to process images of colorimetric experiments. In practice, this protocol consists of three distinct workflow options. This protocol is accessible to uninitiated users with little experience in image processing or color science and does not require fluorescence signals, expensive imaging equipment or custom-written algorithms. We anticipate that total analysis time per region of interest is ~6 min for new users and <3 min for experienced users, although initial color threshold determination might take longer.

Synaptic Specializations of Melanopsin-Retinal Ganglion Cells in Multiple Brain Regions Revealed by Genetic Label for Light and Electron Microscopy

The form and synaptic fine structure of melanopsin-expressing retinal ganglion cells, also called intrinsically photosensitive retinal ganglion cells (ipRGCs), were determined using a new membrane-targeted version of a genetic probe for correlated light and electron microscopy (CLEM). ipRGCs project to multiple brain regions, and because the method labels the entire neuron, it was possible to analyze nerve terminals in multiple retinorecipient brain regions, including the suprachiasmatic nucleus (SCN), olivary pretectal nucleus (OPN), and subregions of the lateral geniculate. Although ipRGCs provide the only direct retinal input to the OPN and SCN, ipRGC terminal arbors and boutons were found to be remarkably different in each target region. A network of dendro-dendritic chemical synapses (DDCSs) was also revealed in the SCN, with ipRGC axon terminals preferentially synapsing on the DDCS-linked cells. The methods developed to enable this analysis should propel other CLEM studies of long-distance brain circuits at high resolution.

Two Stream Active Query Suggestion for Active Learning in Connectomics

For large-scale vision tasks in biomedical images, the labeled data is often limited to train effective deep models. Active learning is a common solution, where a query suggestion method selects representative unlabeled samples for annotation, and the new labels are used to improve the base model. However, most query suggestion models optimize their learnable parameters only on the limited labeled data and consequently become less effective for the more challenging unlabeled data. To tackle this, we propose a two-stream active query suggestion approach. In addition to the supervised feature extractor, we introduce an unsupervised one optimized on all raw images to capture diverse image features, which can later be improved by fine-tuning on new labels. As a use case, we build an end-to-end active learning framework with our query suggestion method for 3D synapse detection and mitochondria segmentation in connectomics. With the framework, we curate, to our best knowledge, the largest connectomics dataset with dense synapses and mitochondria annotation. On this new dataset, our method outperforms previous state-of-the-art methods by 3.1% for synapse and 3.8% for mitochondria in terms of region-of-interest proposal accuracy. We also apply our method to image classification, where it outperforms previous approaches on CIFAR-10 under the same limited annotation budget. The project page is https://zudi-lin.github.io/projects/#two_stream_active.

Development of protocols for the first serial block-face scanning electron microscopy (SBF SEM) studies of bone tissue

There is an unmet need for a high-resolution three-dimensional (3D) technique to simultaneously image osteocytes and the matrix in which these cells reside. In serial block-face scanning electron microscopy (SBF SEM), an ultramicrotome mounted within the vacuum chamber of a microscope repeatedly sections a resin-embedded block of tissue. Backscattered electron scans of the block face provide a stack of high-resolution two-dimensional images, which can be used to visualise and quantify cells and organelles in 3D. High-resolution 3D images of biological tissues from SBF SEM have been exploited considerably to date in the neuroscience field. However, non-brain samples, in particular hard biological tissues, have appeared more challenging to image by SBF SEM due to the difficulties of sectioning and rendering the samples conductive. We have developed and propose protocols for bone tissue preparation using SBF SEM, for imaging simultaneously soft and hard bone tissue components in 3D. We review the state of the art in high-resolution imaging of osteocytes, provide a historical perspective of SBF SEM, and we present first SBF SEM proof-of-concept studies for murine and human tissue. The application of SBF SEM to hard tissues will facilitate qualitative and quantitative 3D studies of tissue microstructure and ultrastructure in bone development, ageing and pathologies such as osteoporosis and osteoarthritis.

Segmentation and Modelling of the Nuclear Envelope of HeLa Cells Imaged with Serial Block Face Scanning Electron Microscopy

This paper describes an unsupervised algorithm, which segments the nuclear envelope of HeLa cells imaged by Serial Block Face Scanning Electron Microscopy. The algorithm exploits the variations of pixel intensity in different cellular regions by calculating edges, which are then used to generate superpixels. The superpixels are morphologically processed and those that correspond to the nuclear region are selected through the analysis of size, position, and correspondence with regions detected in neighbouring slices. The nuclear envelope is segmented from the nuclear region. The three-dimensional segmented nuclear envelope is then modelled against a spheroid to create a two-dimensional (2D) surface. The 2D surface summarises the complex 3D shape of the nuclear envelope and allows the extraction of metrics that may be relevant to characterise the nature of cells. The algorithm was developed and validated on a single cell and tested in six separate cells, each with 300 slices of 2000 × 2000 pixels. Ground truth was available for two of these cells, i.e., 600 hand-segmented slices. The accuracy of the algorithm was evaluated with two similarity metrics: Jaccard Similarity Index and Mean Hausdorff distance. Jaccard values of the first/second segmentation were 93%/90% for the whole cell, and 98%/94% between slices 75 and 225, as the central slices of the nucleus are more regular than those on the extremes. Mean Hausdorff distances were 9/17 pixels for the whole cells and 4/13 pixels for central slices. One slice was processed in approximately 8 s and a whole cell in 40 min. The algorithm outperformed active contours in both accuracy and time.

Three-dimensional ultrastructure and hyperspectral imaging of metabolite accumulation and dynamics in Haematococcus and Chlorella

Phycology has developed alongside light and electron microscopy techniques. Since the 1950s, progress in the field has accelerated dramatically with the advent of electron microscopy. Transmission electron microscopes can only acquire imaging data on a 2D plane. Currently, many of the life sciences are seeking to obtain 3D images with electron microscopy for the accurate interpretation of subcellular dynamics. Three-dimensional reconstruction using serial sections is a method that can cover relatively large cells or tissues without requiring special equipment. Another challenge is monitoring secondary metabolites (such as lipids or carotenoids) in intact cells. This became feasible with hyperspectral cameras, which enable the acquisition of wide-range spectral information in living cells. Here, we review bioimaging studies on the intracellular dynamics of substances such as lipids, carotenoids and phosphorus using conventional to state-of-the-art microscopy techniques in the field of algal biorefining.

Generative models and abstractions for large-scale neuroanatomy datasets

Neural datasets are increasing rapidly in both resolution and volume. In neuroanatomy, this trend has been accelerated by innovations in imaging technology. As full datasets are impractical and unnecessary for many applications, it is important to identify abstractions that distill useful features of neural structure, organization, and anatomy. In this review article, we discuss several such abstractions and highlight recent algorithmic advances in working with these models. In particular, we discuss the use of generative models in neuroanatomy; such models may be considered 'meta-abstractions' that capture distributions over other abstractions.

Subcellular connectomic analyses of energy networks in striated muscle

Mapping biological circuit connectivity has revolutionized our understanding of structure-function relationships. Although connectomic analyses have primarily focused on neural systems, electrical connectivity within muscle mitochondrial networks was recently demonstrated to provide a rapid mechanism for cellular energy distribution. However, tools to evaluate organelle connectivity with high spatial fidelity within single cells are currently lacking. Here, we developed a framework to quantitatively assess mitochondrial network connectivity and interactions with cellular sites of energy storage, utilization, and calcium cycling in cardiac, oxidative, and glycolytic muscle. We demonstrate that mitochondrial network configuration, individual mitochondrial size and shape, and the junctions connecting mitochondria within each network are consistent with the differing contraction demands of each muscle type. Moreover, mitochondria-lipid droplet interaction analyses suggest that individual mitochondria within networks may play specialized roles regarding energy distribution and calcium cycling within the cell and reveal the power of connectomic analyses of organelle interactions within single cells.

Automatic Mitochondria Segmentation for EM Data Using a 3D Supervised Convolutional Network

Recent studies have supported the relation between mitochondrial functions and degenerative disorders related to ageing, such as Alzheimer's and Parkinson's diseases. Since these studies have exposed the need for detailed and high-resolution analysis of physical alterations in mitochondria, it is necessary to be able to perform segmentation and 3D reconstruction of mitochondria. However, due to the variety of mitochondrial structures, automated mitochondria segmentation and reconstruction in electron microscopy (EM) images have proven to be a difficult and challenging task. This paper puts forward an effective and automated pipeline based on deep learning to realize mitochondria segmentation in different EM images. The proposed pipeline consists of three parts: (1) utilizing image registration and histogram equalization as image pre-processing steps to maintain the consistency of the dataset; (2) proposing an effective approach for 3D mitochondria segmentation based on a volumetric, residual convolutional and deeply supervised network; and (3) employing a 3D connection method to obtain the relationship of mitochondria and displaying the 3D reconstruction results. To our knowledge, we are the first researchers to utilize a 3D fully residual convolutional network with a deeply supervised strategy to improve the accuracy of mitochondria segmentation. The experimental results on anisotropic and isotropic EM volumes demonstrate the effectiveness of our method, and the Jaccard index of our segmentation (91.8% in anisotropy, 90.0% in isotropy) and F1 score of detection (92.2% in anisotropy, 90.9% in isotropy) suggest that our approach achieved state-of-the-art results. Our fully automated pipeline contributes to the development of neuroscience by providing neurologists with a rapid approach for obtaining rich mitochondria statistics and helping them elucidate the mechanism and function of mitochondria.

CDeep3M-Plug-and-Play cloud-based deep learning for image segmentation

As biomedical imaging datasets expand, deep neural networks are considered vital for image processing, yet community access is still limited by setting up complex computational environments and availability of high-performance computing resources. We address these bottlenecks with CDeep3M, a ready-to-use image segmentation solution employing a cloud-based deep convolutional neural network. We benchmark CDeep3M on large and complex two-dimensional and three-dimensional imaging datasets from light, X-ray, and electron microscopy.

Characterization of MLKL-mediated Plasma Membrane Rupture in Necroptosis

Necroptosis is a programmed cell death pathway triggered by activation of receptor interacting protein kinase 3 (RIPK3), which phosphorylates and activates the mixed lineage kinase-like domain pseudokinase, MLKL, to rupture or permeabilize the plasma membrane. Necroptosis is an inflammatory pathway associated with multiple pathologies including autoimmunity, infectious and cardiovascular diseases, stroke, neurodegeneration, and cancer. Here, we describe protocols that can be used to characterize MLKL as the executioner of plasma membrane rupture in necroptosis. We visualize the process of necroptosis in cells using live-cell imaging with conventional and confocal fluorescence microscopy, and in fixed cells using electron microscopy, which together revealed the redistribution of MLKL from the cytosol to the plasma membrane prior to induction of large holes in the plasma membrane. We present in vitro nuclear magnetic resonance (NMR) analysis using lipids to identify putative modulators of MLKL-mediated necroptosis. Based on this method, we identified quantitative lipid-binding preferences and phosphatidyl-inositol phosphates (PIPs) as critical binders of MLKL that are required for plasma membrane targeting and permeabilization in necroptosis.

Probability Map Viewer: near real-time probability map generator of serial block electron microscopy collections

Summary

To expedite the review of semi-automated probability maps of organelles and other features from 3D electron microscopy data we have developed Probability Map Viewer, a Java-based web application that enables the computation and visualization of probability map generation results in near real-time as the data are being collected from the microscope. Probability Map Viewer allows the user to select one or more voxel classifiers, apply them on a sub-region of an active collection, and visualize the results as overlays on the raw data via any web browser using a personal computer or mobile device. Thus, Probability Map Viewer accelerates and informs the image analysis workflow by providing a tool for experimenting with and optimizing dataset-specific segmentation strategies during imaging.

Availability and implementation

https://github.com/crbs/probabilitymapviewer.

Contact

mellisman@ucsd.edu.

Supplementary information

Supplementary data are available at Bioinformatics online.

Comparison of 3D cellular imaging techniques based on scanned electron probes: Serial block face SEM vs. Axial bright-field STEM tomography

Microscopies based on focused electron probes allow the cell biologist to image the 3D ultrastructure of eukaryotic cells and tissues extending over large volumes, thus providing new insight into the relationship between cellular architecture and function of organelles. Here we compare two such techniques: electron tomography in conjunction with axial bright-field scanning transmission electron microscopy (BF-STEM), and serial block face scanning electron microscopy (SBF-SEM). The advantages and limitations of each technique are illustrated by their application to determining the 3D ultrastructure of human blood platelets, by considering specimen geometry, specimen preparation, beam damage and image processing methods. Many features of the complex membranes composing the platelet organelles can be determined from both approaches, although STEM tomography offers a higher ∼3 nm isotropic pixel size, compared with ∼5 nm for SBF-SEM in the plane of the block face and ∼30 nm in the perpendicular direction. In this regard, we demonstrate that STEM tomography is advantageous for visualizing the platelet canalicular system, which consists of an interconnected network of narrow (∼50-100 nm) membranous cisternae. In contrast, SBF-SEM enables visualization of complete platelets, each of which extends ∼2 µm in minimum dimension, whereas BF-STEM tomography can typically only visualize approximately half of the platelet volume due to a rapid non-linear loss of signal in specimens of thickness greater than ∼1.5 µm. We also show that the limitations of each approach can be ameliorated by combining 3D and 2D measurements using a stereological approach.

SynEM, automated synapse detection for connectomics

Nerve tissue contains a high density of chemical synapses, about 1 per µm3 in the mammalian cerebral cortex. Thus, even for small blocks of nerve tissue, dense connectomic mapping requires the identification of millions to billions of synapses. While the focus of connectomic data analysis has been on neurite reconstruction, synapse detection becomes limiting when datasets grow in size and dense mapping is required. Here, we report SynEM, a method for automated detection of synapses from conventionally en-bloc stained 3D electron microscopy image stacks. The approach is based on a segmentation of the image data and focuses on classifying borders between neuronal processes as synaptic or non-synaptic. SynEM yields 97% precision and recall in binary cortical connectomes with no user interaction. It scales to large volumes of cortical neuropil, plausibly even whole-brain datasets. SynEM removes the burden of manual synapse annotation for large densely mapped connectomes.

Machine learning applications in cell image analysis

Machine learning (ML) refers to a set of automatic pattern recognition methods that have been successfully applied across various problem domains, including biomedical image analysis. This review focuses on ML applications for image analysis in light microscopy experiments with typical tasks of segmenting and tracking individual cells, and modelling of reconstructed lineage trees. After describing a typical image analysis pipeline and highlighting challenges of automatic analysis (for example, variability in cell morphology, tracking in presence of clutters) this review gives a brief historical outlook of ML, followed by basic concepts and definitions required for understanding examples. This article then presents several example applications at various image processing stages, including the use of supervised learning methods for improving cell segmentation, and the application of active learning for tracking. The review concludes with remarks on parameter setting and future directions.

Automated synaptic connectivity inference for volume electron microscopy

Teravoxel volume electron microscopy data sets from neural tissue can now be acquired in weeks, but data analysis requires years of manual labor. We developed the SyConn framework, which uses deep convolutional neural networks and random forest classifiers to infer a richly annotated synaptic connectivity matrix from manual neurite skeleton reconstructions by automatically identifying mitochondria, synapses and their types, axons, dendrites, spines, myelin, somata and cell types. We tested our approach on serial block-face electron microscopy data sets from zebrafish, mouse and zebra finch, and computed the synaptic wiring of songbird basal ganglia. We found that, for example, basal-ganglia cell types with high firing rates in vivo had higher densities of mitochondria and vesicles and that synapse sizes and quantities scaled systematically, depending on the innervated postsynaptic cell types.

Progress Towards Mammalian Whole-Brain Cellular Connectomics

Neurons are the fundamental structural units of the nervous system-i.e., the Neuron Doctrine-as the pioneering work of Santiago Ramón y Cajal in the 1880's clearly demonstrated through careful observation of Golgi-stained neuronal morphologies. However, at that time sample preparation, imaging methods and computational tools were either nonexistent or insufficiently developed to permit the precise mapping of an entire brain with all of its neurons and their connections. Some measure of the "mesoscopic" connectional organization of the mammalian brain has been obtained over the past decade by alignment of sparse subsets of labeled neurons onto a reference atlas or via MRI-based diffusion tensor imaging. Neither method, however, provides data on the complete connectivity of all neurons comprising an individual brain. Fortunately, whole-brain cellular connectomics now appears within reach due to recent advances in whole-brain sample preparation and high-throughput electron microscopy (EM), though substantial obstacles remain with respect to large volume electron microscopic acquisitions and automated neurite reconstructions. This perspective examines the current status and problems associated with generating a mammalian whole-brain cellular connectome and argues that the time is right to launch a concerted connectomic attack on a small mammalian whole-brain.

Making microscopy count: quantitative light microscopy of dynamic processes in living plants

Cell theory has officially reached 350 years of age as the first use of the word 'cell' in a biological context can be traced to a description of plant material by Robert Hooke in his historic publication 'Micrographia: or some physiological definitions of minute bodies'. The 2015 Royal Microscopical Society Botanical Microscopy meeting was a celebration of the streams of investigation initiated by Hooke to understand at the subcellular scale how plant cell function and form arises. Much of the work presented, and Honorary Fellowships awarded, reflected the advanced application of bioimaging informatics to extract quantitative data from micrographs that reveal dynamic molecular processes driving cell growth and physiology. The field has progressed from collecting many pixels in multiple modes to associating these measurements with objects or features that are meaningful biologically. The additional complexity involves object identification that draws on a different type of expertise from computer science and statistics that is often impenetrable to biologists. There are many useful tools and approaches being developed, but we now need more interdisciplinary exchange to use them effectively. In this review we show how this quiet revolution has provided tools available to any personal computer user. We also discuss the oft-neglected issue of quantifying algorithm robustness and the exciting possibilities offered through the integration of physiological information generated by biosensors with object detection and tracking.

Optimizing the 3D-reconstruction technique for serial block-face scanning electron microscopy

BACKGROUND

Elucidating the anatomy of neuronal circuits and localizing the synaptic connections between neurons, can give us important insights in how the neuronal circuits work. We are using serial block-face scanning electron microscopy (SBEM) to investigate the anatomy of a collision detection circuit including the Lobula Giant Movement Detector (LGMD) neuron in the locust, Locusta migratoria. For this, thousands of serial electron micrographs are produced that allow us to trace the neuronal branching pattern.

NEW METHOD

The reconstruction of neurons was previously done manually by drawing cell outlines of each cell in each image separately. This approach was very time consuming and troublesome. To make the process more efficient a new interactive software was developed. It uses the contrast between the neuron under investigation and its surrounding for semi-automatic segmentation.

RESULTS

For segmentation the user sets starting regions manually and the algorithm automatically selects a volume within the neuron until the edges corresponding to the neuronal outline are reached. Internally the algorithm optimizes a 3D active contour segmentation model formulated as a cost function taking the SEM image edges into account. This reduced the reconstruction time, while staying close to the manual reference segmentation result.

COMPARISON WITH EXISTING METHODS

Our algorithm is easy to use for a fast segmentation process, unlike previous methods it does not require image training nor an extended computing capacity.

CONCLUSION

Our semi-automatic segmentation algorithm led to a dramatic reduction in processing time for the 3D-reconstruction of identified neurons.

Sequential Engagement of Distinct MLKL Phosphatidylinositol-Binding Sites Executes Necroptosis

Necroptosis is a cell death pathway regulated by the receptor interacting protein kinase 3 (RIPK3) and the mixed lineage kinase domain-like (MLKL) pseudokinase. How MLKL executes plasma membrane rupture upon phosphorylation by RIPK3 remains controversial. Here, we characterize the hierarchical transduction of structural changes in MLKL that culminate in necroptosis. The MLKL brace, proximal to the N-terminal helix bundle (NB), is involved in oligomerization to facilitate plasma membrane targeting through the low-affinity binding of NB to phosphorylated inositol polar head groups of phosphatidylinositol phosphate (PIP) phospholipids. At the membrane, the NB undergoes a "rolling over" mechanism to expose additional higher-affinity PIP-binding sites responsible for robust association to the membrane and displacement of the brace from the NB. PI(4,5)P2 is the preferred PIP-binding partner. We investigate the specific association of MLKL with PIPs and subsequent structural changes during necroptosis.

Microscopy Image Browser: A Platform for Segmentation and Analysis of Multidimensional Datasets

Understanding the structure-function relationship of cells and organelles in their natural context requires multidimensional imaging. As techniques for multimodal 3-D imaging have become more accessible, effective processing, visualization, and analysis of large datasets are posing a bottleneck for the workflow. Here, we present a new software package for high-performance segmentation and image processing of multidimensional datasets that improves and facilitates the full utilization and quantitative analysis of acquired data, which is freely available from a dedicated website. The open-source environment enables modification and insertion of new plug-ins to customize the program for specific needs. We provide practical examples of program features used for processing, segmentation and analysis of light and electron microscopy datasets, and detailed tutorials to enable users to rapidly and thoroughly learn how to use the program.

Visualizing viral protein structures in cells using genetic probes for correlated light and electron microscopy

Structural studies of viral proteins most often use high-resolution techniques such as X-ray crystallography, nuclear magnetic resonance, single particle negative stain, or cryo-electron microscopy (EM) to reveal atomic interactions of soluble, homogeneous viral proteins or viral protein complexes. Once viral proteins or complexes are separated from their host's cellular environment, their natural in situ structure and details of how they interact with other cellular components may be lost. EM has been an invaluable tool in virology since its introduction in the late 1940's and subsequent application to cells in the 1950's. EM studies have expanded our knowledge of viral entry, viral replication, alteration of cellular components, and viral lysis. Most of these early studies were focused on conspicuous morphological cellular changes, because classic EM metal stains were designed to highlight classes of cellular structures rather than specific molecular structures. Much later, to identify viral proteins inducing specific structural configurations at the cellular level, immunostaining with a primary antibody followed by colloidal gold secondary antibody was employed to mark the location of specific viral proteins. This technique can suffer from artifacts in cellular ultrastructure due to compromises required to provide access to the immuno-reagents. Immunolocalization methods also require the generation of highly specific antibodies, which may not be available for every viral protein. Here we discuss new methods to visualize viral proteins and structures at high resolutions in situ using correlated light and electron microscopy (CLEM). We discuss the use of genetically encoded protein fusions that oxidize diaminobenzidine (DAB) into an osmiophilic polymer that can be visualized by EM. Detailed protocols for applying the genetically encoded photo-oxidizing protein MiniSOG to a viral protein, photo-oxidation of the fusion protein to yield DAB polymer staining, and preparation of photo-oxidized samples for TEM and serial block-face scanning EM (SBEM) for large-scale volume EM data acquisition are also presented. As an example, we discuss the recent multi-scale analysis of Adenoviral protein E4-ORF3 that reveals a new type of multi-functional polymer that disrupts multiple cellular proteins. This new capability to visualize unambiguously specific viral protein structures at high resolutions in the native cellular environment is revealing new insights into how they usurp host proteins and functions to drive pathological viral replication.