A unified framework for automated 3-d segmentation of surface-stained living cells and a comprehensive segmentation evaluation

Collagen XVIII regulates extracellular matrix integrity in the developing nephrons and impacts nephron progenitor cell behavior

Renal development is a complex process in which two major processes, tubular branching and nephron development, regulate each other reciprocally. Our previous findings have indicated that collagen XVIII (ColXVIII), an extracellular matrix protein, affects the renal branching morphogenesis. We investigate here the role of ColXVIII in nephron formation and the behavior of nephron progenitor cells (NPCs) using isoform-specific ColXVIII knockout mice. The results show that the short ColXVIII isoform predominates in the early epithelialized nephron structures whereas the two longer isoforms are expressed only in the later phases of glomerular formation. Meanwhile, electron microscopy showed that the ColXVIII mutant embryonic kidneys have ultrastructural defects at least from embryonic day 16.5 onwards. Similar structural defects had previously been observed in adult ColXVIII-deficient mice, indicating a congenital origin. The lack of ColXVIII led to a reduced NPC population in which changes in NPC proliferation and maintenance and in macrophage influx were perceived to play a role. The changes in NPC behavior in turn led to notably reduced overall nephron formation. In conclusion, the results show that ColXVIII has multiple roles in renal development, both in ureteric branching and in NPC behavior.

DFTNet: Dual-Path Feature Transfer Network for Weakly Supervised Medical Image Segmentation

Medical image segmentation has long suffered from the problem of expensive labels. Acquiring pixel-level annotations is time-consuming, labor-intensive, and relies on extensive expert knowledge. Bounding box annotations, in contrast, are relatively easy to acquire. Thus, in this paper, we explore to segment images through a novel Dual-path Feature Transfer design with only bounding box annotations. Specifically, a Target-aware Reconstructor is proposed to extract target-related features by reconstructing the pixels within the bounding box through the channel and spatial attention module. Then, a sliding Feature Fusion and Transfer Module (FFTM) fuses the extracted features from Reconstructor and transfers them to guide the Segmentor for segmentation. Finally, we present the Confidence Ranking Loss (CRLoss) which dynamically assigns weights to the loss of each pixel based on the network's confidence. CRLoss mitigates the impact of inaccurate pseudo-labels on performance. Extensive experiments demonstrate that our proposed model achieves state-of-the-art performance on the Medical Segmentation Decathlon (MSD) Brain Tumour and PROMISE12 datasets.

Identification of Retinal Ganglion Cells from β-III Stained Fluorescent Microscopic Images

Optic nerve crush in mouse model is widely used for investigating the course following retinal ganglion cell (RGCs) injury. Manual cell counting from β-III tubulin stained microscopic images has been routinely performed to monitor RGCs after an optic nerve crush injury, but is time-consuming and prone to observer variability. This paper describes an automatic technique for RGC identification. We developed and validated (i) a sensitive cell candidate segmentation scheme and (ii) a classifier that removed false positives while retaining true positives. Two major contributions were made in cell candidate segmentation. First, a homomorphic filter was designed to adjust for the inhomogeneous illumination caused by uneven penetration of β-III tubulin antibody. Second, the optimal segmentation parameters for cell detection are highly image-specific. To address this issue, we introduced an offline-online parameter tuning approach. Offline tuning optimized model parameters based on training images and online tuning further optimized the parameters at the testing stage without needing access to the ground truth. In the cell identification stage, 31 geometric, statistical and textural features were extracted from each segmented cell candidate, which was subsequently classified as true or false positives by support vector machine. The homomorphic filter and the online parameter tuning approach together increased cell recall by 28%. The entire pipeline attained a recall, precision and coefficient of determination (r²) of 85.3%, 97.1% and 0.994. The availability of the proposed pipeline will allow efficient, accurate and reproducible RGC quantification required for assessing the death/survival of RGCs in disease models.

Automated 3D Axonal Morphometry of White Matter

Axonal structure underlies white matter functionality and plays a major role in brain connectivity. The current literature on the axonal structure is based on the analysis of two-dimensional (2D) cross-sections, which, as we demonstrate, is precarious. To be able to quantify three-dimensional (3D) axonal morphology, we developed a novel pipeline, called ACSON (AutomatiC 3D Segmentation and morphometry Of axoNs), for automated 3D segmentation and morphometric analysis of the white matter ultrastructure. The automated pipeline eliminates the need for time-consuming manual segmentation of 3D datasets. ACSON segments myelin, myelinated and unmyelinated axons, mitochondria, cells and vacuoles, and analyzes the morphology of myelinated axons. We applied the pipeline to serial block-face scanning electron microscopy images of the corpus callosum of sham-operated (n = 2) and brain injured (n = 3) rats 5 months after the injury. The 3D morphometry showed that cross-sections of myelinated axons were elliptic rather than circular, and their diameter varied substantially along their longitudinal axis. It also showed a significant reduction in the myelinated axon diameter of the ipsilateral corpus callosum of rats 5 months after brain injury, indicating ongoing axonal alterations even at this chronic time-point.

Biologically constrained optimization based cell membrane segmentation in C. elegans embryos

BACKGROUND

Recent advances in bioimaging and automated analysis methods have enabled the large-scale systematic analysis of cellular dynamics during the embryonic development of Caenorhabditis elegans. Most of these analyses have focused on cell lineage tracing rather than cell shape dynamics. Cell shape analysis requires cell membrane segmentation, which is challenging because of insufficient resolution and image quality. This problem is currently solved by complicated segmentation methods requiring laborious and time consuming parameter adjustments.

RESULTS

Our new framework BCOMS (Biologically Constrained Optimization based cell Membrane Segmentation) automates the extraction of the cell shape of C. elegans embryos. Both the segmentation and evaluation processes are automated. To automate the evaluation, we solve an optimization problem under biological constraints. The performance of BCOMS was validated against a manually created ground truth of the 24-cell stage embryo. The average deviation of 25 cell shape features was 5.6%. The deviation was mainly caused by membranes parallel to the focal planes, which either contact the surfaces of adjacent cells or make no contact with other cells. Because segmentation of these membranes was difficult even by manual inspection, the automated segmentation was sufficiently accurate for cell shape analysis. As the number of manually created ground truths is necessarily limited, we compared the segmentation results between two adjacent time points. Across all cells and all cell cycles, the average deviation of the 25 cell shape features was 4.3%, smaller than that between the automated segmentation result and ground truth.

CONCLUSIONS

BCOMS automated the accurate extraction of cell shapes in developing C. elegans embryos. By replacing image processing parameters with easily adjustable biological constraints, BCOMS provides a user-friendly framework. The framework is also applicable to other model organisms. Creating the biological constraints is a critical step requiring collaboration between an experimentalist and a software developer.

Novel fixed z-direction (FiZD) kidney primordia and an organoid culture system for time-lapse confocal imaging

Tissue, organ and organoid cultures provide suitable models for developmental studies, but our understanding of how the organs are assembled at the single-cell level still remains unclear. We describe here a novel fixed z-direction (FiZD) culture setup that permits high-resolution confocal imaging of organoids and embryonic tissues. In a FiZD culture a permeable membrane compresses the tissues onto a glass coverslip and the spacers adjust the thickness, enabling the tissue to grow for up to 12 days. Thus, the kidney rudiment and the organoids can adjust to the limited z-directional space and yet advance the process of kidney morphogenesis, enabling long-term time-lapse and high-resolution confocal imaging. As the data quality achieved was sufficient for computer-assisted cell segmentation and analysis, the method can be used for studying morphogenesis ex vivo at the level of the single constituent cells of a complex mammalian organogenesis model system.

Summary: Time-lapse confocal imaging of organoids and embryonic tissues through fixed z-direction culture allows long-term single-cell resolution live imaging of tissue growth and morphogenesis.

PDE Based Algorithms for Smooth Watersheds

Watershed segmentation is useful for a number of image segmentation problems with a wide range of practical applications. Traditionally, the tracking of the immersion front is done by applying a fast sorting algorithm. In this work, we explore a continuous approach based on a geometric description of the immersion front which gives rise to a partial differential equation. The main advantage of using a partial differential equation to track the immersion front is that the method becomes versatile and may easily be stabilized by introducing regularization terms. Coupling the geometric approach with a proper "merging strategy" creates a robust algorithm which minimizes over- and under-segmentation even without predefined markers. Since reliable markers defined prior to segmentation can be difficult to construct automatically for various reasons, being able to treat marker-free situations is a major advantage of the proposed method over earlier watershed formulations. The motivation for the methods developed in this paper is taken from high-throughput screening of cells. A fully automated segmentation of single cells enables the extraction of cell properties from large data sets, which can provide substantial insight into a biological model system. Applying smoothing to the boundaries can improve the accuracy in many image analysis tasks requiring a precise delineation of the plasma membrane of the cell. The proposed segmentation method is applied to real images containing fluorescently labeled cells, and the experimental results show that our implementation is robust and reliable for a variety of challenging segmentation tasks.

Robust Nucleus/Cell Detection and Segmentation in Digital Pathology and Microscopy Images: A Comprehensive Review

Digital pathology and microscopy image analysis is widely used for comprehensive studies of cell morphology or tissue structure. Manual assessment is labor intensive and prone to interobserver variations. Computer-aided methods, which can significantly improve the objectivity and reproducibility, have attracted a great deal of interest in recent literature. Among the pipeline of building a computer-aided diagnosis system, nucleus or cell detection and segmentation play a very important role to describe the molecular morphological information. In the past few decades, many efforts have been devoted to automated nucleus/cell detection and segmentation. In this review, we provide a comprehensive summary of the recent state-of-the-art nucleus/cell segmentation approaches on different types of microscopy images including bright-field, phase-contrast, differential interference contrast, fluorescence, and electron microscopies. In addition, we discuss the challenges for the current methods and the potential future work of nucleus/cell detection and segmentation.

Survey statistics of automated segmentations applied to optical imaging of mammalian cells

BACKGROUND

The goal of this survey paper is to overview cellular measurements using optical microscopy imaging followed by automated image segmentation. The cellular measurements of primary interest are taken from mammalian cells and their components. They are denoted as two- or three-dimensional (2D or 3D) image objects of biological interest. In our applications, such cellular measurements are important for understanding cell phenomena, such as cell counts, cell-scaffold interactions, cell colony growth rates, or cell pluripotency stability, as well as for establishing quality metrics for stem cell therapies. In this context, this survey paper is focused on automated segmentation as a software-based measurement leading to quantitative cellular measurements.

METHODS

We define the scope of this survey and a classification schema first. Next, all found and manually filteredpublications are classified according to the main categories: (1) objects of interests (or objects to be segmented), (2) imaging modalities, (3) digital data axes, (4) segmentation algorithms, (5) segmentation evaluations, (6) computational hardware platforms used for segmentation acceleration, and (7) object (cellular) measurements. Finally, all classified papers are converted programmatically into a set of hyperlinked web pages with occurrence and co-occurrence statistics of assigned categories.

RESULTS

The survey paper presents to a reader: (a) the state-of-the-art overview of published papers about automated segmentation applied to optical microscopy imaging of mammalian cells, (b) a classification of segmentation aspects in the context of cell optical imaging, (c) histogram and co-occurrence summary statistics about cellular measurements, segmentations, segmented objects, segmentation evaluations, and the use of computational platforms for accelerating segmentation execution, and (d) open research problems to pursue.

CONCLUSIONS

The novel contributions of this survey paper are: (1) a new type of classification of cellular measurements and automated segmentation, (2) statistics about the published literature, and (3) a web hyperlinked interface to classification statistics of the surveyed papers at https://isg.nist.gov/deepzoomweb/resources/survey/index.html.

Novel microscopy-based screening method reveals regulators of contact-dependent intercellular transfer

Contact-dependent intercellular transfer (codeIT) of cellular constituents can have functional consequences for recipient cells, such as enhanced survival and drug resistance. Pathogenic viruses, prions and bacteria can also utilize this mechanism to spread to adjacent cells and potentially evade immune detection. However, little is known about the molecular mechanism underlying this intercellular transfer process. Here, we present a novel microscopy-based screening method to identify regulators and cargo of codeIT. Single donor cells, carrying fluorescently labelled endocytic organelles or proteins, are co-cultured with excess acceptor cells. CodeIT is quantified by confocal microscopy and image analysis in 3D, preserving spatial information. An siRNA-based screening using this method revealed the involvement of several myosins and small GTPases as codeIT regulators. Our data indicates that cellular protrusions and tubular recycling endosomes are important for codeIT. We automated image acquisition and analysis to facilitate large-scale chemical and genetic screening efforts to identify key regulators of codeIT.

Segmentation-driven image registration- application to 4D DCE-MRI recordings of the moving kidneys

Dynamic contrast enhanced magnetic resonance imaging (DCE-MRI) of the kidneys requires proper motion correction and segmentation to enable an estimation of glomerular filtration rate through pharmacokinetic modeling. Traditionally, co-registration, segmentation, and pharmacokinetic modeling have been applied sequentially as separate processing steps. In this paper, a combined 4D model for simultaneous registration and segmentation of the whole kidney is presented. To demonstrate the model in numerical experiments, we used normalized gradients as data term in the registration and a Mahalanobis distance from the time courses of the segmented regions to a training set for supervised segmentation. By applying this framework to an input consisting of 4D image time series, we conduct simultaneous motion correction and two-region segmentation into kidney and background. The potential of the new approach is demonstrated on real DCE-MRI data from ten healthy volunteers.

CellSegm - a MATLAB toolbox for high-throughput 3D cell segmentation

: The application of fluorescence microscopy in cell biology often generates a huge amount of imaging data. Automated whole cell segmentation of such data enables the detection and analysis of individual cells, where a manual delineation is often time consuming, or practically not feasible. Furthermore, compared to manual analysis, automation normally has a higher degree of reproducibility. CellSegm, the software presented in this work, is a Matlab based command line software toolbox providing an automated whole cell segmentation of images showing surface stained cells, acquired by fluorescence microscopy. It has options for both fully automated and semi-automated cell segmentation. Major algorithmic steps are: (i) smoothing, (ii) Hessian-based ridge enhancement, (iii) marker-controlled watershed segmentation, and (iv) feature-based classfication of cell candidates. Using a wide selection of image recordings and code snippets, we demonstrate that CellSegm has the ability to detect various types of surface stained cells in 3D. After detection and outlining of individual cells, the cell candidates can be subject to software based analysis, specified and programmed by the end-user, or they can be analyzed by other software tools. A segmentation of tissue samples with appropriate characteristics is also shown to be resolvable in CellSegm. The command-line interface of CellSegm facilitates scripting of the separate tools, all implemented in Matlab, offering a high degree of flexibility and tailored workflows for the end-user. The modularity and scripting capabilities of CellSegm enable automated workflows and quantitative analysis of microscopic data, suited for high-throughput image based screening.

A flexible and robust approach for segmenting cell nuclei from 2D microscopy images using supervised learning and template matching

We describe a new supervised learning-based template matching approach for segmenting cell nuclei from microscopy images. The method uses examples selected by a user for building a statistical model that captures the texture and shape variations of the nuclear structures from a given dataset to be segmented. Segmentation of subsequent, unlabeled, images is then performed by finding the model instance that best matches (in the normalized cross correlation sense) local neighborhood in the input image. We demonstrate the application of our method to segmenting nuclei from a variety of imaging modalities, and quantitatively compare our results to several other methods. Quantitative results using both simulated and real image data show that, while certain methods may work well for certain imaging modalities, our software is able to obtain high accuracy across several imaging modalities studied. Results also demonstrate that, relative to several existing methods, the template-based method we propose presents increased robustness in the sense of better handling variations in illumination, variations in texture from different imaging modalities, providing more smooth and accurate segmentation borders, as well as handling better cluttered nuclei.

A visual targeting system for the microinjection of unstained adherent cells

Automatic localization and targeting are critical steps in automating the process of microinjecting adherent cells. This process is currently performed manually by highly trained operators and is characterized as a laborious task with low success rate. Therefore, automation is desired to increase the efficiency and consistency of the operations. This research offers a contribution to this procedure through the development of a vision system for a robotic microinjection setup. Its goals are to automatically locate adherent cells in a culture dish and target them for a microinjection. Here the major concern was the achievement of an error-free targeting system to guarantee high consistency in microinjection experiments. To accomplish this, a novel visual targeting algorithm integrating different image processing techniques was proposed. This framework employed defocusing microscopy to highlight cell features and improve cell segmentation and targeting reliability. Three main image processing techniques, operating at three different focus levels in a bright field (BF) microscope, were used: an anisotropic contour completion (ACC) method, a local intensity variation background-foreground classifier, and a grayscale threshold-based segmentation. The proposed framework combined information gathered by each of these methods using a validation map and this was shown to provide reliable cell targeting results. Experiments conducted with sets of real images from two different cell lines (CHO-K1 and HEK), which contained a total of more than 650 cells, yielded flawless targeting results along with a cell detection ratio greater than 50%.

Partitioning and Exocytosis of Secretory Granules during Division of PC12 Cells

The biogenesis, maturation, and exocytosis of secretory granules in interphase cells have been well documented, whereas the distribution and exocytosis of these hormone-storing organelles during cell division have received little attention. By combining ultrastructural analyses and time-lapse microscopy, we here show that, in dividing PC12 cells, the prominent peripheral localization of secretory granules is retained during prophase but clearly reduced during prometaphase, ending up with only few peripherally localized secretory granules in metaphase cells. During anaphase and telophase, secretory granules exhibited a pronounced movement towards the cell midzone and, evidently, their tracks colocalized with spindle microtubules. During cytokinesis, secretory granules were excluded from the midbody and accumulated at the bases of the intercellular bridge. Furthermore, by measuring exocytosis at the single granule level, we showed, that during all stages of cell division, secretory granules were competent for regulated exocytosis. In conclusion, our data shed new light on the complex molecular machinery of secretory granule redistribution during cell division, which facilitates their release from the F-actin-rich cortex and active transport along spindle microtubules.

Automatic segmentation of cell nuclei in Feulgen-stained histological sections of prostate cancer and quantitative evaluation of segmentation results

Digital image analysis of cell nuclei is useful to obtain quantitative information for the diagnosis and prognosis of cancer. However, the lack of a reliable automatic nuclear segmentation is a limiting factor for high-throughput nuclear image analysis. We have developed a method for automatic segmentation of nuclei in Feulgen-stained histological sections of prostate cancer. A local adaptive thresholding with an object perimeter gradient verification step detected the nuclei and was combined with an active contour model that featured an optimized initialization and worked within a restricted region to improve convergence of the segmentation of each nucleus. The method was tested on 30 randomly selected image frames from three cases, comparing the results from the automatic algorithm to a manual delineation of 924 nuclei. The automatic method segmented a few more nuclei compared to the manual method, and about 73% of the manually segmented nuclei were also segmented by the automatic method. For each nucleus segmented both manually and automatically, the accuracy (i.e., agreement with manual delineation) was estimated. The mean segmentation sensitivity/specificity were 95%/96%. The results from the automatic method were not significantly different from the ground truth provided by manual segmentation. This opens the possibility for large-scale nuclear analysis based on automatic segmentation of nuclei in Feulgen-stained histological sections.

Evaluation of segmentation algorithms on cell populations using CDF curves

Cell segmentation is a critical step in the analysis pipeline for most imaging cytometry experiments and evaluating the performance of segmentation algorithms is important for aiding the selection of segmentation algorithms. Four popular algorithms are evaluated based on their cell segmentation performance. Because segmentation involves the classification of pixels belonging to regions within the cell or belonging to background, these algorithms are evaluated based on their total misclassification error. Misclassification error is particularly relevant in the analysis of quantitative descriptors of cell morphology involving pixel counts, such as projected area, aspect ratio and diameter. Since the cumulative distribution function captures completely the stochastic properties of a population of misclassification errors it is used to compare segmentation performance.

Gebiss: an ImageJ plugin for the specification of ground truth and the performance evaluation of 3D segmentation algorithms

Background

Image segmentation is a crucial step in quantitative microscopy that helps to define regions of tissues, cells or subcellular compartments. Depending on the degree of user interactions, segmentation methods can be divided into manual, automated or semi-automated approaches. 3D image stacks usually require automated methods due to their large number of optical sections. However, certain applications benefit from manual or semi-automated approaches. Scenarios include the quantification of 3D images with poor signal-to-noise ratios or the generation of so-called ground truth segmentations that are used to evaluate the accuracy of automated segmentation methods.

Results

We have developed Gebiss; an ImageJ plugin for the interactive segmentation, visualisation and quantification of 3D microscopic image stacks. We integrated a variety of existing plugins for threshold-based segmentation and volume visualisation.

Conclusions

We demonstrate the application of Gebiss to the segmentation of nuclei in live Drosophila embryos and the quantification of neurodegeneration in Drosophila larval brains. Gebiss was developed as a cross-platform ImageJ plugin and is freely available on the web at http://imaging.bii.a-star.edu.sg/projects/gebiss/.

Interactive segmentation of clustered cells via geodesic commute distance and constrained density weighted Nyström method

An interactive method is proposed for complex cell segmentation, in particular of clustered cells. This article has two main contributions: First, we explore a hybrid combination of the random walk and the geodesic graph based methods for image segmentation and propose the novel concept of geodesic commute distance to classify pixels. The computation of geodesic commute distance requires an eigenvector decomposition of the weighted Laplacian matrix of a graph constructed from the image to be segmented. Second, by incorporating pairwise constraints from seeds into the algorithm, we present a novel method for eigenvector decomposition, namely a constrained density weighted Nyström method. Both visual and quantitative comparison with other semiautomatic algorithms including Voronoi-based segmentation, grow cut, graph cuts, random walk, and geodesic method are given to evaluate the performance of the proposed method, which is a powerful tool for quantitative analysis of clustered cell images in live cell imaging.