Vertebrate neural stem cell segmentation, tracking and lineaging with validation and editing

BNP-Track: A framework for multi-particle superresolved tracking

When tracking fluorescently labeled molecules (termed "emitters") under widefield microscopes, point spread function overlap of neighboring molecules is inevitable in both dilute and especially crowded environments. In such cases, superresolution methods leveraging rare photophysical events to distinguish static targets nearby in space introduce temporal delays that compromise tracking. As we have shown in a companion manuscript, for dynamic targets, information on neighboring fluorescent molecules is encoded as spatial intensity correlations across pixels and temporal correlations in intensity patterns across time frames. We then demonstrated how we used all spatiotemporal correlations encoded in the data to achieve superresolved tracking. That is, we showed the results of full posterior inference over both the number of emitters and their associated tracks simultaneously and self-consistently through Bayesian nonparametrics. In this companion manuscript we focus on testing the robustness of our tracking tool, BNP-Track, across sets of parameter regimes and compare BNP-Track to competing tracking methods in the spirit of a prior Nature Methods tracking competition. We explore additional features of BNP-Track including how a stochastic treatment of background yields greater accuracy in emitter number determination and how BNP-Track corrects for point spread function blur (or "aliasing") introduced by intraframe motion in addition to propagating error originating from myriad sources (such as criss-crossing tracks, out-of-focus particles, pixelation, shot and camera artefact, stochastic background) in posterior inference over emitter numbers and their associated tracks. While head-to-head comparison with other tracking methods is not possible (as competitors cannot simultaneously learn molecule numbers and associated tracks), we can give competing methods some advantages in order to perform approximate head-to-head comparison. We show that even under such optimistic scenarios, BNP-Track is capable of tracking multiple diffraction-limited point emitters conventional tracking methods cannot resolve thereby extending the superresolution paradigm to dynamical targets.

Spatiotemporal control of ERK pulse frequency coordinates fate decisions during mammary acinar morphogenesis

The signaling events controlling proliferation, survival, and apoptosis during mammary epithelial acinar morphogenesis remain poorly characterized. By imaging single-cell ERK activity dynamics in MCF10A acini, we find that these fates depend on the average frequency of non-periodic ERK pulses. High pulse frequency is observed during initial acinus growth, correlating with rapid cell motility and proliferation. Subsequent decrease in motility correlates with lower ERK pulse frequency and quiescence. Later, during lumen formation, coordinated multicellular ERK waves emerge, correlating with high and low ERK pulse frequencies in outer surviving and inner dying cells, respectively. Optogenetic entrainment of ERK pulses causally connects high ERK pulse frequency with inner cell survival. Acini harboring the PIK3CA H1047R mutation display increased ERK pulse frequency and inner cell survival. Thus, fate decisions during acinar morphogenesis are coordinated by different spatiotemporal modalities of ERK pulse frequency.

CellTrackVis: analyzing the performance of cell tracking algorithms

Live-cell imaging is a common data acquisition technique used by biologists to analyze cell behavior. Since manually tracking cells in a video sequence is extremely time-consuming, many automatic algorithms have been developed in the last twenty years to accomplish the task. However, none of these algorithms can yet claim robust tracking performance at the varying of acquisition conditions (e.g., cell type, acquisition device, cell treatments). While many visualization tools exist to help with cell behavior analysis, there are no tools to help with the algorithm's validation. This paper proposes CellTrackVis, a new visualization tool for evaluating cell tracking algorithms. CellTrackVis allows comparing automatically generated cell tracks with ground truth data to help biologists select the best-suited algorithm for their experimented pipeline. Moreover, CellTackVis can be used as a debugging tool while developing a new cell tracking algorithm to investigate where, when, and why each tracking error occurred.

A graph-based cell tracking algorithm with few manually tunable parameters and automated segmentation error correction

Automatic cell segmentation and tracking enables to gain quantitative insights into the processes driving cell migration. To investigate new data with minimal manual effort, cell tracking algorithms should be easy to apply and reduce manual curation time by providing automatic correction of segmentation errors. Current cell tracking algorithms, however, are either easy to apply to new data sets but lack automatic segmentation error correction, or have a vast set of parameters that needs either manual tuning or annotated data for parameter tuning. In this work, we propose a tracking algorithm with only few manually tunable parameters and automatic segmentation error correction. Moreover, no training data is needed. We compare the performance of our approach to three well-performing tracking algorithms from the Cell Tracking Challenge on data sets with simulated, degraded segmentation—including false negatives, over- and under-segmentation errors. Our tracking algorithm can correct false negatives, over- and under-segmentation errors as well as a mixture of the aforementioned segmentation errors. On data sets with under-segmentation errors or a mixture of segmentation errors our approach performs best. Moreover, without requiring additional manual tuning, our approach ranks several times in the top 3 on the 6^th edition of the Cell Tracking Challenge.

OrganoidTracker: Efficient cell tracking using machine learning and manual error correction

Time-lapse microscopy is routinely used to follow cells within organoids, allowing direct study of division and differentiation patterns. There is an increasing interest in cell tracking in organoids, which makes it possible to study their growth and homeostasis at the single-cell level. As tracking these cells by hand is prohibitively time consuming, automation using a computer program is required. Unfortunately, organoids have a high cell density and fast cell movement, which makes automated cell tracking difficult. In this work, a semi-automated cell tracker has been developed. To detect the nuclei, we use a machine learning approach based on a convolutional neural network. To form cell trajectories, we link detections at different time points together using a min-cost flow solver. The tracker raises warnings for situations with likely errors. Rapid changes in nucleus volume and position are reported for manual review, as well as cases where nuclei divide, appear and disappear. When the warning system is adjusted such that virtually error-free lineage trees can be obtained, still less than 2% of all detected nuclei positions are marked for manual analysis. This provides an enormous speed boost over manual cell tracking, while still providing tracking data of the same quality as manual tracking.

A novel tracking and analysis system for time-lapse cell imaging of Saccharomyces cerevisiae

Recent studies have revealed that tracking single cells using time-lapse fluorescence microscopy is an optimal tool for spatiotemporal evaluation of proteins of interest. Using this approach with Saccharomyces cerevisiae as a model organism, we previously found that heterochromatin regions involved in epigenetic regulation differ between individual cells. Determining the regularity of this phenomenon requires measurement of spatiotemporal epigenetic-dependent changes in protein levels across more than one generation. In past studies, we conducted these analyses manually to obtain a dendrogram, but this required more than 15 h, even for a single set of microscopic cell images. Thus, in this study, we developed a software-based analysis system to analyze time-lapse cellular images of S. cerevisiae, which allowed automatic generation of a dendrogram from a given set of time-lapse cell images. This approach is divided into two phases: a cell extraction and tracking phase, and an analysis phase. The cell extraction and tracking phase generates a set of necessary information for each cell, such as geometrical properties and the daughter-mother relationships, using image processing-based analysis techniques. Then, based on this information, the analysis phase allows generation of the final dendrogram by analyzing the fluorescent characteristics of each cell. The system is equipped with manual error correction to correct for the inevitable errors that occur in these analyses. The time required to obtain the final dendrograms was drastically reduced from 15 h in manual analysis to 0.8 h using this novel system.

Practical Considerations in Particle and Object Tracking and Analysis

The rapid advancement of live-cell imaging technologies has enabled biologists to generate high-dimensional data to follow biological movement at the microscopic level. Yet, the "perceived" ease of use of modern microscopes has led to challenges whereby sub-optimal data are commonly generated that cannot support quantitative tracking and analysis as a result of various ill-advised decisions made during image acquisition. Even optimally acquired images often require further optimization through digital processing before they can be analyzed. In writing this article, we presume our target audience to be biologists with a foundational understanding of digital image acquisition and processing, who are seeking to understand the essential steps for particle/object tracking experiments. It is with this targeted readership in mind that we review the basic principles of image-processing techniques as well as analysis strategies commonly used for tracking experiments. We conclude this technical survey with a discussion of how movement behavior can be mathematically modeled and described. © 2019 by John Wiley & Sons, Inc.

Separating Touching Cells Using Pixel Replicated Elliptical Shape Models

One of the most important and error-prone tasks in biological image analysis is the segmentation of touching or overlapping cells. Particularly for optical microscopy, including transmitted light and confocal fluorescence microscopy, there is often no consistent discriminative information to separate cells that touch or overlap. It is desired to partition touching foreground pixels into cells using the binary threshold image information only, and optionally incorporating gradient information. The most common approaches for segmenting touching and overlapping cells in these scenarios are based on the watershed transform. We describe a new approach called pixel replication for the task of segmenting elliptical objects that touch or overlap. Pixel replication uses the image Euclidean distance transform in combination with Gaussian mixture models to better exploit practically effective optimization for delineating objects with elliptical decision boundaries. Pixel replication improves significantly on commonly used methods based on watershed transforms, or based on fitting Gaussian mixtures directly to the thresholded image data. Pixel replication works equivalently on both 2-D and 3-D image data, and naturally combines information from multi-channel images. The accuracy of the proposed technique is measured using both the segmentation accuracy on simulated ellipse data and the tracking accuracy on validated stem cell tracking results extracted from hundreds of live-cell microscopy image sequences. Pixel replication is shown to be significantly more accurate compared with other approaches. Variance relationships are derived, allowing a more practically effective Gaussian mixture model to extract cell boundaries for data generated from the threshold image using the uniform elliptical distribution and from the distance transform image using the triangular elliptical distribution.

Contact guidance is cell cycle-dependent

Cancer cell migration is essential for metastasis, during which cancer cells move through the tumor and reach the blood vessels. In vivo, cancer cells are exposed to contact guidance and chemotactic cues. Depending on the strength of such cues, cells will migrate in a random or directed manner. While similar cues may also stimulate cell proliferation, it is not clear whether cell cycle progression affects migration of cancer cells and whether this effect is different in random versus directed migration. In this study, we tested the effect of cell cycle progression on contact guided migration in 2D and 3D environments, in the breast carcinoma cell line, FUCCI-MDA-MB-231. The results were quantified from live cell microscopy images using the open source lineage editing and validation image analysis tools (LEVER). In 2D, cells were placed inside 10 μm-wide microchannels to stimulate contact guidance, with or without an additional chemotactic gradient of the soluble epidermal growth factor. In 3D, contact guidance was modeled by aligned collagen fibers. In both 2D and 3D, contact guidance was cell cycle-dependent, while the addition of the chemo-attractant gradient in 2D increased cell velocity and persistence in directionally migrating cells, regardless of their cell cycle phases. In both 2D and 3D contact guidance, cells in the G1 phase of the cell cycle outperformed cells in the S/G2 phase in terms of migration persistence and instantaneous velocity. These data suggest that in the presence of contact guidance cues in vivo, breast carcinoma cells in the G1 phase of the cell cycle may be more efficient in reaching the neighboring vasculature.

Accelerating Live Single-Cell Signalling Studies

The dynamics of signalling networks that couple environmental conditions with cellular behaviour can now be characterised in exquisite detail using live single-cell imaging experiments. Recent improvements in our abilities to introduce fluorescent sensors into cells, coupled with advances in pipelines for quantifying and extracting single-cell data, mean that high-throughput systematic analyses of signalling dynamics are becoming possible. In this review, we consider current technologies that are driving progress in the scale and range of such studies. Moreover, we discuss novel approaches that are allowing us to explore how pathways respond to changes in inputs and even predict the fate of a cell based upon its signalling history and state.

Characterising live cell behaviour: Traditional label-free and quantitative phase imaging approaches

Label-free imaging uses inherent contrast mechanisms within cells to create image contrast without introducing dyes/labels, which may confound results. Quantitative phase imaging is label-free and offers higher content and contrast compared to traditional techniques. High-contrast images facilitate generation of individual cell metrics via more robust segmentation and tracking, enabling formation of a label-free dynamic phenotype describing cell-to-cell heterogeneity and temporal changes. Compared to population-level averages, individual cell-level dynamic phenotypes have greater power to differentiate between cellular responses to treatments, which has clinical relevance e.g. in the treatment of cancer. Furthermore, as the data is obtained label-free, the same cells can be used for further assays or expansion, of potential benefit for the fields of regenerative and personalised medicine.

Lineage mapper: A versatile cell and particle tracker

The ability to accurately track cells and particles from images is critical to many biomedical problems. To address this, we developed Lineage Mapper, an open-source tracker for time-lapse images of biological cells, colonies, and particles. Lineage Mapper tracks objects independently of the segmentation method, detects mitosis in confluence, separates cell clumps mistakenly segmented as a single cell, provides accuracy and scalability even on terabyte-sized datasets, and creates division and/or fusion lineages. Lineage Mapper has been tested and validated on multiple biological and simulated problems. The software is available in ImageJ and Matlab at isg.nist.gov.

Robust cell tracking in epithelial tissues through identification of maximum common subgraphs

Tracking of cells in live-imaging microscopy videos of epithelial sheets is a powerful tool for investigating fundamental processes in embryonic development. Characterizing cell growth, proliferation, intercalation and apoptosis in epithelia helps us to understand how morphogenetic processes such as tissue invagination and extension are locally regulated and controlled. Accurate cell tracking requires correctly resolving cells entering or leaving the field of view between frames, cell neighbour exchanges, cell removals and cell divisions. However, current tracking methods for epithelial sheets are not robust to large morphogenetic deformations and require significant manual interventions. Here, we present a novel algorithm for epithelial cell tracking, exploiting the graph-theoretic concept of a 'maximum common subgraph' to track cells between frames of a video. Our algorithm does not require the adjustment of tissue-specific parameters, and scales in sub-quadratic time with tissue size. It does not rely on precise positional information, permitting large cell movements between frames and enabling tracking in datasets acquired at low temporal resolution due to experimental constraints such as phototoxicity. To demonstrate the method, we perform tracking on the Drosophila embryonic epidermis and compare cell-cell rearrangements to previous studies in other tissues. Our implementation is open source and generally applicable to epithelial tissues.

Measuring Process Dynamics and Nuclear Migration for Clones of Neural Progenitor Cells

Neural stem and progenitor cells (NPCs) generate processes that extend from the cell body in a dynamic manner. The NPC nucleus migrates along these processes with patterns believed to be tightly coupled to mechanisms of cell cycle regulation and cell fate determination. Here, we describe a new segmentation and tracking approach that allows NPC processes and nuclei to be reliably tracked across multiple rounds of cell division in phase-contrast microscopy images. Results are presented for mouse adult and embryonic NPCs from hundreds of clones, or lineage trees, containing tens of thousands of cells and millions of segmentations. New visualization approaches allow the NPC nuclear and process features to be effectively visualized for an entire clone. Significant differences in process and nuclear dynamics were found among type A and type C adult NPCs, and also between embryonic NPCs cultured from the anterior and posterior cerebral cortex.

LEVER: software tools for segmentation, tracking and lineaging of proliferating cells

The analysis of time-lapse images showing cells dividing to produce clones of related cells is an important application in biological microscopy. Imaging at the temporal resolution required to establish accurate tracking for vertebrate stem or cancer cells often requires the use of transmitted light or phase-contrast microscopy. Processing these images requires automated segmentation, tracking and lineaging algorithms. There is also a need for any errors in the automated processing to be easily identified and quickly corrected. We have developed LEVER, an open source software tool that combines the automated image analysis for phase-contrast microscopy movies with an easy-to-use interface for validating the results and correcting any errors.

AVAILABILITY AND IMPLEMENTATION

LEVER is available free and open source, licensed under the GNU GPLv3. Details on obtaining and using LEVER are available at http://n2t.net/ark:/87918/d9rp4t CONTACT: acohen@coe.drexel.edu.

Cell morphology-based classification of red blood cells using holographic imaging informatics

We present methods that automatically select a linear or nonlinear classifier for red blood cell (RBC) classification by analyzing the equality of the covariance matrices in Gabor-filtered holographic images. First, the phase images of the RBCs are numerically reconstructed from their holograms, which are recorded using off-axis digital holographic microscopy (DHM). Second, each RBC is segmented using a marker-controlled watershed transform algorithm and the inner part of the RBC is identified and analyzed. Third, the Gabor wavelet transform is applied to the segmented cells to extract a series of features, which then undergo a multivariate statistical test to evaluate the equality of the covariance matrices of the different shapes of the RBCs using selected features. When these covariance matrices are not equal, a nonlinear classification scheme based on quadratic functions is applied; otherwise, a linear classification is applied. We used the stomatocyte, discocyte, and echinocyte RBC for classifier training and testing. Simulation results demonstrated that 10 of the 14 RBC features are useful in RBC classification. Experimental results also revealed that the covariance matrices of the three main RBC groups are not equal and that a nonlinear classification method has a much lower misclassification rate. The proposed automated RBC classification method has the potential for use in drug testing and the diagnosis of RBC-related diseases.

Automated single particle detection and tracking for large microscopy datasets

Recent advances in optical microscopy have enabled the acquisition of very large datasets from living cells with unprecedented spatial and temporal resolutions. Our ability to process these datasets now plays an essential role in order to understand many biological processes. In this paper, we present an automated particle detection algorithm capable of operating in low signal-to-noise fluorescence microscopy environments and handling large datasets. When combined with our particle linking framework, it can provide hitherto intractable quantitative measurements describing the dynamics of large cohorts of cellular components from organelles to single molecules. We begin with validating the performance of our method on synthetic image data, and then extend the validation to include experiment images with ground truth. Finally, we apply the algorithm to two single-particle-tracking photo-activated localization microscopy biological datasets, acquired from living primary cells with very high temporal rates. Our analysis of the dynamics of very large cohorts of 10 000 s of membrane-associated protein molecules show that they behave as if caged in nanodomains. We show that the robustness and efficiency of our method provides a tool for the examination of single-molecule behaviour with unprecedented spatial detail and high acquisition rates.

Live Imaging of Adult Neural Stem Cells in Rodents

The generation of cells of the neural lineage within the brain is not restricted to early development. New neurons, oligodendrocytes, and astrocytes are produced in the adult brain throughout the entire murine life. However, despite the extensive research performed in the field of adult neurogenesis during the past years, fundamental questions regarding the cell biology of adult neural stem cells (aNSCs) remain to be uncovered. For instance, it is crucial to elucidate whether a single aNSC is capable of differentiating into all three different macroglial cell types in vivo or these distinct progenies constitute entirely separate lineages. Similarly, the cell cycle length, the time and mode of division (symmetric vs. asymmetric) that these cells undergo within their lineage progression are interesting questions under current investigation. In this sense, live imaging constitutes a valuable ally in the search of reliable answers to the previous questions. In spite of the current limitations of technology new approaches are being developed and outstanding amount of knowledge is being piled up providing interesting insights in the behavior of aNSCs. Here, we will review the state of the art of live imaging as well as the alternative models that currently offer new answers to critical questions.

Isolation and Culture of Adult Zebrafish Brain-derived Neurospheres

The zebrafish is a highly relevant model organism for understanding the cellular and molecular mechanisms involved in neurogenesis and brain regeneration in vertebrates. However, an in-depth analysis of the molecular mechanisms underlying zebrafish adult neurogenesis has been limited due to the lack of a reliable protocol for isolating and culturing neural adult stem/progenitor cells. Here we provide a reproducible method to examine adult neurogenesis using a neurosphere assay derived from zebrafish whole brain or from the telencephalon, tectum and cerebellum regions of the adult zebrafish brain. The protocol involves, first the microdissection of zebrafish adult brain, then single cell dissociation and isolation of self-renewing multipotent neural stem/progenitor cells. The entire procedure takes eight days. Additionally, we describe how to manipulate gene expression in zebrafish neurospheres, which will be particularly useful to test the role of specific signaling pathways during adult neural stem/progenitor cell proliferation and differentiation in zebrafish.

Evaluating Cell Processes, Quality, and Biomarkers in Pluripotent Stem Cells Using Video Bioinformatics

There is a foundational need for quality control tools in stem cell laboratories engaged in basic research, regenerative therapies, and toxicological studies. These tools require automated methods for evaluating cell processes and quality during in vitro passaging, expansion, maintenance, and differentiation. In this paper, an unbiased, automated high-content profiling toolkit, StemCellQC, is presented that non-invasively extracts information on cell quality and cellular processes from time-lapse phase-contrast videos. Twenty four (24) morphological and dynamic features were analyzed in healthy, unhealthy, and dying human embryonic stem cell (hESC) colonies to identify those features that were affected in each group. Multiple features differed in the healthy versus unhealthy/dying groups, and these features were linked to growth, motility, and death. Biomarkers were discovered that predicted cell processes before they were detectable by manual observation. StemCellQC distinguished healthy and unhealthy/dying hESC colonies with 96% accuracy by non-invasively measuring and tracking dynamic and morphological features over 48 hours. Changes in cellular processes can be monitored by StemCellQC and predictions can be made about the quality of pluripotent stem cell colonies. This toolkit reduced the time and resources required to track multiple pluripotent stem cell colonies and eliminated handling errors and false classifications due to human bias. StemCellQC provided both user-specified and classifier-determined analysis in cases where the affected features are not intuitive or anticipated. Video analysis algorithms allowed assessment of biological phenomena using automatic detection analysis, which can aid facilities where maintaining stem cell quality and/or monitoring changes in cellular processes are essential. In the future StemCellQC can be expanded to include other features, cell types, treatments, and differentiating cells.

Cell Tracking Accuracy Measurement Based on Comparison of Acyclic Oriented Graphs

Tracking motile cells in time-lapse series is challenging and is required in many biomedical applications. Cell tracks can be mathematically represented as acyclic oriented graphs. Their vertices describe the spatio-temporal locations of individual cells, whereas the edges represent temporal relationships between them. Such a representation maintains the knowledge of all important cellular events within a captured field of view, such as migration, division, death, and transit through the field of view. The increasing number of cell tracking algorithms calls for comparison of their performance. However, the lack of a standardized cell tracking accuracy measure makes the comparison impracticable. This paper defines and evaluates an accuracy measure for objective and systematic benchmarking of cell tracking algorithms. The measure assumes the existence of a ground-truth reference, and assesses how difficult it is to transform a computed graph into the reference one. The difficulty is measured as a weighted sum of the lowest number of graph operations, such as split, delete, and add a vertex and delete, add, and alter the semantics of an edge, needed to make the graphs identical. The measure behavior is extensively analyzed based on the tracking results provided by the participants of the first Cell Tracking Challenge hosted by the 2013 IEEE International Symposium on Biomedical Imaging. We demonstrate the robustness and stability of the measure against small changes in the choice of weights for diverse cell tracking algorithms and fluorescence microscopy datasets. As the measure penalizes all possible errors in the tracking results and is easy to compute, it may especially help developers and analysts to tune their algorithms according to their needs.

A review of progress in single particle tracking: from methods to biophysical insights

Optical microscopy has for centuries been a key tool to study living cells with minimum invasiveness. The advent of single molecule techniques over the past two decades has revolutionized the field of cell biology by providing a more quantitative picture of the complex and highly dynamic organization of living systems. Amongst these techniques, single particle tracking (SPT) has emerged as a powerful approach to study a variety of dynamic processes in life sciences. SPT provides access to single molecule behavior in the natural context of living cells, thereby allowing a complete statistical characterization of the system under study. In this review we describe the foundations of SPT together with novel optical implementations that nowadays allow the investigation of single molecule dynamic events with increasingly high spatiotemporal resolution using molecular densities closer to physiological expression levels. We outline some of the algorithms for the faithful reconstruction of SPT trajectories as well as data analysis, and highlight biological examples where the technique has provided novel insights into the role of diffusion regulating cellular function. The last part of the review concentrates on different theoretical models that describe anomalous transport behavior and ergodicity breaking observed from SPT studies in living cells.

CellProfiler Tracer: exploring and validating high-throughput, time-lapse microscopy image data

Background

Time-lapse analysis of cellular images is an important and growing need in biology. Algorithms for cell tracking are widely available; what researchers have been missing is a single open-source software package to visualize standard tracking output (from software like CellProfiler) in a way that allows convenient assessment of track quality, especially for researchers tuning tracking parameters for high-content time-lapse experiments. This makes quality assessment and algorithm adjustment a substantial challenge, particularly when dealing with hundreds of time-lapse movies collected in a high-throughput manner.

Results

We present CellProfiler Tracer, a free and open-source tool that complements the object tracking functionality of the CellProfiler biological image analysis package. Tracer allows multi-parametric morphological data to be visualized on object tracks, providing visualizations that have already been validated within the scientific community for time-lapse experiments, and combining them with simple graph-based measures for highlighting possible tracking artifacts.

Conclusions

CellProfiler Tracer is a useful, free tool for inspection and quality control of object tracking data, available from http://www.cellprofiler.org/tracer/.

Electronic supplementary material

The online version of this article (doi:10.1186/s12859-015-0759-x) contains supplementary material, which is available to authorized users.

Survey statistics of automated segmentations applied to optical imaging of mammalian cells

BACKGROUND

The goal of this survey paper is to overview cellular measurements using optical microscopy imaging followed by automated image segmentation. The cellular measurements of primary interest are taken from mammalian cells and their components. They are denoted as two- or three-dimensional (2D or 3D) image objects of biological interest. In our applications, such cellular measurements are important for understanding cell phenomena, such as cell counts, cell-scaffold interactions, cell colony growth rates, or cell pluripotency stability, as well as for establishing quality metrics for stem cell therapies. In this context, this survey paper is focused on automated segmentation as a software-based measurement leading to quantitative cellular measurements.

METHODS

We define the scope of this survey and a classification schema first. Next, all found and manually filteredpublications are classified according to the main categories: (1) objects of interests (or objects to be segmented), (2) imaging modalities, (3) digital data axes, (4) segmentation algorithms, (5) segmentation evaluations, (6) computational hardware platforms used for segmentation acceleration, and (7) object (cellular) measurements. Finally, all classified papers are converted programmatically into a set of hyperlinked web pages with occurrence and co-occurrence statistics of assigned categories.

RESULTS

The survey paper presents to a reader: (a) the state-of-the-art overview of published papers about automated segmentation applied to optical microscopy imaging of mammalian cells, (b) a classification of segmentation aspects in the context of cell optical imaging, (c) histogram and co-occurrence summary statistics about cellular measurements, segmentations, segmented objects, segmentation evaluations, and the use of computational platforms for accelerating segmentation execution, and (d) open research problems to pursue.

CONCLUSIONS

The novel contributions of this survey paper are: (1) a new type of classification of cellular measurements and automated segmentation, (2) statistics about the published literature, and (3) a web hyperlinked interface to classification statistics of the surveyed papers at https://isg.nist.gov/deepzoomweb/resources/survey/index.html.

Computational Image Analysis Reveals Intrinsic Multigenerational Differences between Anterior and Posterior Cerebral Cortex Neural Progenitor Cells

Time-lapse microscopy can capture patterns of development through multiple divisions for an entire clone of proliferating cells. Images are taken every few minutes over many days, generating data too vast to process completely by hand. Computational analysis of this data can benefit from occasional human guidance. Here we combine improved automated algorithms with minimized human validation to produce fully corrected segmentation, tracking, and lineaging results with dramatic reduction in effort. A web-based viewer provides access to data and results. The improved approach allows efficient analysis of large numbers of clones. Using this method, we studied populations of progenitor cells derived from the anterior and posterior embryonic mouse cerebral cortex, each growing in a standardized culture environment. Progenitors from the anterior cortex were smaller, less motile, and produced smaller clones compared to those from the posterior cortex, demonstrating cell-intrinsic differences that may contribute to the areal organization of the cerebral cortex.

•

Open-source automated software designed to track stem/progenitor clones in time-lapse movies

•

Software tools for easy data validation and visualization greatly improve efficiency

•

Lineage tree reconstruction from hundreds of embryonic mouse forebrain clones

•

Intrinsic differences in progenitor behavior from anterior/posterior cerebral cortex

Normalized Compression Distance of Multisets with Applications

Pairwise normalized compression distance (NCD) is a parameter-free, feature-free, alignment-free, similarity metric based on compression. We propose an NCD of multisets that is also metric. Previously, attempts to obtain such an NCD failed. For classification purposes it is superior to the pairwise NCD in accuracy and implementation complexity. We cover the entire trajectory from theoretical underpinning to feasible practice. It is applied to biological (stem cell, organelle transport) and OCR classification questions that were earlier treated with the pairwise NCD. With the new method we achieved significantly better results. The theoretic foundation is Kolmogorov complexity.

Extracting meaning from biological imaging data

Biological imaging continues to improve, capturing continually longer-term, richer, and more complex data, penetrating deeper into live tissue. How do we gain insight into the dynamic processes of disease and development from terabytes of multidimensional image data? Here I describe a collaborative approach to extracting meaning from biological imaging data. The collaboration consists of teams of biologists and engineers working together. Custom computational tools are built to best exploit application-specific knowledge in order to visualize and analyze large and complex data sets. The image data are summarized, extracting and modeling the features that capture the objects and relationships in the data. The summarization is validated, the results visualized, and errors corrected as needed. Finally, the customized analysis and visualization tools together with the image data and the summarization results are shared. This Perspective provides a brief guide to the mathematical ideas that rigorously quantify the notion of extracting meaning from biological image, and to the practical approaches that have been used to apply these ideas to a wide range of applications in cell and tissue optical imaging.

Segmentation and tracking of stem cells in time lapse microscopy to quantify dynamic behavioral changes during spheroid formation

Dynamic behavior of stem cells during in vitro development is diverse. Previous cell tracking studies have focused more on cell proliferation than on cell aggregation. However, the enhancement of cell proliferation in association with cell aggregation has been reported. In a previous study, we also demonstrated that the aggregation of adult human mesenchymal stem cells to form three-dimensional (3D) cellular spheroids helped maintain the expression of stemness marker genes in the cells. However, the dynamic behavioral changes triggered by spheroid formation remain to be investigated. A scheme of image processing techniques is proposed to meet this need. A hybrid-thresholding technique was first developed for efficient segmentation of cell clusters, after which a cell tracking method based on pair-matching with topological constraints was designed. Two morphological indices were derived to track the timing of 3D spheroid formation during the cellular aggregation process. Five cell motility indices measured from single cells and 3D spheroids were then compared. After confirmation of more than 90% correspondence between the results obtained by manual tracking and the proposed methods, an analysis of cellular behavior reveals a significant increase in motility in association with spheroid formation, consistent with a previous report that used a gene expression approach. This study proposed a systematic image analysis method to quantify the dynamic behavior of stem cells for stemness evaluation during cell culturing in vitro. Results demonstrated the validity of the developed platform in investigation of the dynamic behavior of cell aggregation in stem cell cultures in vitro.

Visualization and correction of automated segmentation, tracking and lineaging from 5-D stem cell image sequences

BACKGROUND

Neural stem cells are motile and proliferative cells that undergo mitosis, dividing to produce daughter cells and ultimately generating differentiated neurons and glia. Understanding the mechanisms controlling neural stem cell proliferation and differentiation will play a key role in the emerging fields of regenerative medicine and cancer therapeutics. Stem cell studies in vitro from 2-D image data are well established. Visualizing and analyzing large three dimensional images of intact tissue is a challenging task. It becomes more difficult as the dimensionality of the image data increases to include time and additional fluorescence channels. There is a pressing need for 5-D image analysis and visualization tools to study cellular dynamics in the intact niche and to quantify the role that environmental factors play in determining cell fate.

RESULTS

We present an application that integrates visualization and quantitative analysis of 5-D (x,y,z,t,channel) and large montage confocal fluorescence microscopy images. The image sequences show stem cells together with blood vessels, enabling quantification of the dynamic behaviors of stem cells in relation to their vascular niche, with applications in developmental and cancer biology. Our application automatically segments, tracks, and lineages the image sequence data and then allows the user to view and edit the results of automated algorithms in a stereoscopic 3-D window while simultaneously viewing the stem cell lineage tree in a 2-D window. Using the GPU to store and render the image sequence data enables a hybrid computational approach. An inference-based approach utilizing user-provided edits to automatically correct related mistakes executes interactively on the system CPU while the GPU handles 3-D visualization tasks.

CONCLUSIONS

By exploiting commodity computer gaming hardware, we have developed an application that can be run in the laboratory to facilitate rapid iteration through biological experiments. We combine unsupervised image analysis algorithms with an interactive visualization of the results. Our validation interface allows for each data set to be corrected to 100% accuracy, ensuring that downstream data analysis is accurate and verifiable. Our tool is the first to combine all of these aspects, leveraging the synergies obtained by utilizing validation information from stereo visualization to improve the low level image processing tasks.

Objective comparison of particle tracking methods

Particle tracking is of key importance for quantitative analysis of intracellular dynamic processes from time-lapse microscopy image data. Because manually detecting and following large numbers of individual particles is not feasible, automated computational methods have been developed for these tasks by many groups. Aiming to perform an objective comparison of methods, we gathered the community and organized an open competition in which participating teams applied their own methods independently to a commonly defined data set including diverse scenarios. Performance was assessed using commonly defined measures. Although no single method performed best across all scenarios, the results revealed clear differences between the various approaches, leading to notable practical conclusions for users and developers.

Multisensory interface for 5D stem cell image volumes

Biological imaging of live cell and tissue using 3D microscopy is able to capture time-lapse image sequences showing multiple molecular markers labeling different biological structures simultaneously. In order to analyze this complex multi-dimensional image sequence data, there is a need for automated quantitative algorithms, and for methods to visualize and interact with both the data and the analytical results. Traditional computational human input devices such as the keyboard and mouse are no longer adequate for complex tasks such as manipulating and navigating 3+ dimensional volumes. In this paper, we have developed a new interaction system for interfacing with big data sets using the human visual system together with touch, force and audio feedback. This system includes real-time dynamic 3D visualization, haptic interaction via exoskeletal glove, and tonal auditory components that seamlessly create an immersive environment for efficient qualitative analysis.

Segmentation of occluded hematopoietic stem cells from tracking

Image sequences of live proliferating cells often contain visual ambiguities that are difficult even for human domain experts to resolve. Here we present a new approach to analyzing image sequences that capture the development of clones of hematopoietic stem cells (HSCs) from live cell time lapse microscopy. The HSCs cannot survive long term imaging unless they are cultured together with a secondary cell type, OP9 stromal cells. The HSCs frequently disappear under the OP9 cell layer, making segmentation difficult or impossible from a single image frame, even for a human domain expert. We have developed a new approach to the segmentation of HSCs that captures these occluded cells. Starting with an a priori segmentation that uses a Monte Carlo technique to estimate the number of cells in a clump of touching cells, we proceed to track and lineage the image data. Following user validation of the lineage information, an a posteriori resegmentation step utilizing tracking results delineates the HSCs occluded by the OP9 layer. Resegmentation has been applied to 3031 occluded segmentations from 77 tracks, correctly recovering over 84% of the occluded segmentations.

Neural stem cells: generating and regenerating the brain

One of the landmark events of the past 25 years in neuroscience research was the establishment of neural stem cells (NSCs) as a life-long source of neurons and glia, a concept that shattered the dogma that the nervous system lacked regenerative power. Stem cells afford the plasticity to generate, repair, and change nervous system function. Combined with reprogramming technology, human somatic cell-derived NSCs and their progeny can model neurological diseases with improved accuracy. As technology advances, we anticipate further important discoveries and novel therapies based on the knowledge and application of these powerful cells.

An analysis-synthesis approach for neurosphere modelisation under phase-contrast microscopy

The study of stem cells is one of the most important biomedical research. Understanding their development could allow multiple applications in regenerative medicine. For this purpose, automated solutions for the observation of stem cell development process are needed. This study introduces an on-line analysis method for the modelling of neurosphere evolution during the early time of their development under phase contrast microscopy. From the corresponding phase contrast time-lapse sequences, we extract information from the neurosphere using a combination of phase contrast physics deconvolution and curve detection for locate the cells inside the neurosphere. Then, based on prior biological knowledge, we generate possible and optimal 3-dimensional configuration using 2D to 3D registration methods and evolutionary optimisation algorithm.

PhagoSight: an open-source MATLAB® package for the analysis of fluorescent neutrophil and macrophage migration in a zebrafish model

Neutrophil migration in zebrafish larvae is increasingly used as a model to study the response of these leukocytes to different determinants of the cellular inflammatory response. However, it remains challenging to extract comprehensive information describing the behaviour of neutrophils from the multi-dimensional data sets acquired with widefield or confocal microscopes. Here, we describe PhagoSight, an open-source software package for the segmentation, tracking and visualisation of migrating phagocytes in three dimensions. The algorithms in PhagoSight extract a large number of measurements that summarise the behaviour of neutrophils, but that could potentially be applied to any moving fluorescent cells. To derive a useful panel of variables quantifying aspects of neutrophil migratory behaviour, and to demonstrate the utility of PhagoSight, we evaluated changes in the volume of migrating neutrophils. Cell volume increased as neutrophils migrated towards the wound region of injured zebrafish. PhagoSight is openly available as MATLAB® m-files under the GNU General Public License. Synthetic data sets and a comprehensive user manual are available from http://www.phagosight.org.

Towards comprehensive cell lineage reconstructions in complex organisms using light-sheet microscopy

Understanding the development of complex multicellular organisms as a function of the underlying cell behavior is one of the most fundamental goals of developmental biology. The ability to quantitatively follow cell dynamics in entire developing embryos is an indispensable step towards such a system-level understanding. In recent years, light-sheet fluorescence microscopy has emerged as a particularly promising strategy for recording the in vivo data required to realize this goal. Using light-sheet fluorescence microscopy, entire complex organisms can be rapidly imaged in three dimensions at sub-cellular resolution, achieving high temporal sampling and excellent signal-to-noise ratio without damaging the living specimen or bleaching fluorescent markers. The resulting datasets allow following individual cells in vertebrate and higher invertebrate embryos over up to several days of development. However, the complexity and size of these multi-terabyte recordings typically preclude comprehensive manual analyses. Thus, new computational approaches are required to automatically segment cell morphologies, accurately track cell identities and systematically analyze cell behavior throughout embryonic development. We review current efforts in light-sheet microscopy and bioimage informatics towards this goal, and argue that comprehensive cell lineage reconstructions are finally within reach for many key model organisms, including fruit fly, zebrafish and mouse.

Axonal transport analysis using Multitemporal Association Tracking

Multitemporal Association Tracking (MAT) is a new graph-based method for multitarget tracking in biological applications that reduces the error rate and implementation complexity compared to approaches based on bipartite matching. The data association problem is solved over a window of future detection data using a graph-based cost function that approximates the Bayesian a posteriori association probability. MAT has been applied to hundreds of image sequences, tracking organelle and vesicles to quantify the deficiencies in axonal transport that can accompany neurodegenerative disorders such as Huntington's Disease and Multiple Sclerosis and to quantify changes in transport in response to therapeutic interventions.