Cell population tracking and lineage construction with spatiotemporal context

CMTT-JTracker: a fully test-time adaptive framework serving automated cell lineage construction

Cell tracking is an essential function needed in automated cellular activity monitoring. In practice, processing methods striking a balance between computational efficiency and accuracy as well as demonstrating robust generalizability across diverse cell datasets are highly desired. This paper develops a central-metric fully test-time adaptive framework for cell tracking (CMTT-JTracker). Firstly, a CMTT mechanism is designed for the pre-segmentation of cell images, which enables extracting target information at different resolutions without additional training. Next, a multi-task learning network with the spatial attention scheme is developed to simultaneously realize detection and re-identification tasks based on features extracted by CMTT. Experimental results demonstrate that the CMTT-JTracker exhibits remarkable biological and tracking performance compared with benchmarking tracking methods. It achieves a multiple object tracking accuracy (MOTA) of $0.894$ on Fluo-N2DH-SIM+ and a MOTA of $0.850$ on PhC-C2DL-PSC. Experimental results further confirm that the CMTT applied solely as a segmentation unit outperforms the SOTA segmentation benchmarks on various datasets, particularly excelling in scenarios with dense cells. The Dice coefficients of the CMTT range from a high of $0.928$ to a low of $0.758$ across different datasets.

Machine learning enhanced cell tracking

Quantifying cell biology in space and time requires computational methods to detect cells, measure their properties, and assemble these into meaningful trajectories. In this aspect, machine learning (ML) is having a transformational effect on bioimage analysis, now enabling robust cell detection in multidimensional image data. However, the task of cell tracking, or constructing accurate multi-generational lineages from imaging data, remains an open challenge. Most cell tracking algorithms are largely based on our prior knowledge of cell behaviors, and as such, are difficult to generalize to new and unseen cell types or datasets. Here, we propose that ML provides the framework to learn aspects of cell behavior using cell tracking as the task to be learned. We suggest that advances in representation learning, cell tracking datasets, metrics, and methods for constructing and evaluating tracking solutions can all form part of an end-to-end ML-enhanced pipeline. These developments will lead the way to new computational methods that can be used to understand complex, time-evolving biological systems.

Adaptive tracking algorithm for trajectory analysis of cells and layer-by-layer assessment of motility dynamics

Tracking biological objects such as cells or subcellular components imaged with time-lapse microscopy enables us to understand the molecular principles about the dynamics of cell behaviors. However, automatic object detection, segmentation and extracting trajectories remain as a rate-limiting step due to intrinsic challenges of video processing. This paper presents an adaptive tracking algorithm (Adtari) that automatically finds the optimum search radius and cell linkages to determine trajectories in consecutive frames. A critical assumption in most tracking studies is that displacement remains unchanged throughout the movie and cells in a few frames are usually analyzed to determine its magnitude. Tracking errors and inaccurate association of cells may occur if the user does not correctly evaluate the value or prior knowledge is not present on cell movement. The key novelty of our method is that minimum intercellular distance and maximum displacement of cells between frames are dynamically computed and used to determine the threshold distance. Since the space between cells is highly variable in a given frame, our software recursively alters the magnitude to determine all plausible matches in the trajectory analysis. Our method therefore eliminates a major preprocessing step where a constant distance was used to determine the neighbor cells in tracking methods. Cells having multiple overlaps and splitting events were further evaluated by using the shape attributes including perimeter, area, ellipticity and distance. The features were applied to determine the closest matches by minimizing the difference in their magnitudes. Finally, reporting section of our software were used to generate instant maps by overlaying cell features and trajectories. Adtari was validated by using videos with variable signal-to-noise, contrast ratio and cell density. We compared the adaptive tracking with constant distance and other methods to evaluate performance and its efficiency. Our algorithm yields reduced mismatch ratio, increased ratio of whole cell track, higher frame tracking efficiency and allows layer-by-layer assessment of motility to characterize single-cells. Adaptive tracking provides a reliable, accurate, time efficient and user-friendly open source software that is well suited for analysis of 2D fluorescence microscopy video datasets.

Quantitative phase velocimetry measures bulk intracellular transport of cell mass during the cell cycle

Transport of mass within cells helps maintain homeostasis and is disrupted by disease and stress. Here, we develop quantitative phase velocimetry (QPV) as a label-free approach to make the invisible flow of mass within cells visible and quantifiable. We benchmark our approach against alternative image registration methods, a theoretical error model, and synthetic data. Our method tracks not just individual labeled particles or molecules, but the entire flow of bulk material through the cell. This enables us to measure diffusivity within distinct cell compartments using a single approach, which we use here for direct comparison of nuclear and cytoplasmic diffusivity. As a label-free method, QPV can be used for long-term tracking to capture dynamics through the cell cycle.

No Cell Left behind: Automated, Stochastic, Physics-Based Tracking of Every Cell in a Dense, Growing Colony

Motivation: Precise tracking of individual cells—especially tracking the family lineage, for example in a developing embryo—has widespread applications in biology and medicine. Due to significant noise in microscope images, existing methods have difficulty precisely tracking cell activities. These difficulties often require human intervention to resolve. Humans are helpful because our brain naturally and automatically builds a simulation “model” of any scene that we observe. Because we understand simple truths about the world—for example cells can move and divide, but they cannot instantaneously move vast distances—this model “in our heads” helps us to severely constrain the possible interpretations of what we see, allowing us to easily distinguish signal from noise, and track the motion of cells even in the presence of extreme levels of noise that would completely confound existing automated methods. Results: Here, we mimic the ability of the human brain by building an explicit computer simulation model of the scene. Our simulated cells are programmed to allow movement and cell division consistent with reality. At each video frame, we stochastically generate millions of nearby “Universes” and evolve them stochastically to the next frame. We then find and fit the best universes to reality by minimizing the residual between the real image frame and a synthetic image of the simulation. The rule-based simulation puts extremely stringent constraints on possible interpretations of the data, allowing our system to perform far better than existing methods even in the presense of extreme levels of image noise. We demonstrate the viability of this method by accurately tracking every cell in a colony that grows from 4 to over 300 individuals, doing about as well as a human can in the difficult task of tracking cell lineages.

A review for Cell and Particle Tracking on Microscopy Images using Algorithms and Deep Learning Technologies

Time-lapse microscopy images generated by biological experiments have been widely used for observing target activities, such as the motion trajectories and survival states. Based on these observations, biologists can conclude experimental results or present new hypotheses for several biological applications, i.e. virus research or drug design. Many methods or tools have been proposed in the past to observe cell and particle activities, which are defined as single cell tracking and single particle tracking problems, by using algorithms and deep learning technologies. In this article, a review for these works is presented in order to summarize the past methods and research topics at first, then points out the problems raised by these works, and finally proposes future research directions. The contributions of this article will help researchers to understand past development trends and further propose innovative technologies.

Correction of uneven illumination in color microscopic image based on fully convolutional network

The correction of uneven illumination in microscopic image is a basic task in medical imaging. Most of the existing methods are designed for monochrome images. An effective fully convolutional network (FCN) is proposed to directly process color microscopic image in this paper. The proposed method estimates the distribution of illumination information in input image, and then carry out the correction of the corresponding uneven illumination through a feature encoder module, a feature decoder module, and a detail supplement module. In this process, overlapping residual blocks are designed to better transfer the illumination information, and in particular a well-designed weighted loss function ensures that the network can not only correct the illumination but also preserve image details. The proposed method is compared with some related methods on real pathological cell images qualitatively and quantitatively. Experimental results show that our method achieves the excellent performance. The proposed method is also applied to the preprocessing of whole slide imaging (WSI) tiles, which greatly improves the effect of image mosaicking.

A novel tracking and analysis system for time-lapse cell imaging of Saccharomyces cerevisiae

Recent studies have revealed that tracking single cells using time-lapse fluorescence microscopy is an optimal tool for spatiotemporal evaluation of proteins of interest. Using this approach with Saccharomyces cerevisiae as a model organism, we previously found that heterochromatin regions involved in epigenetic regulation differ between individual cells. Determining the regularity of this phenomenon requires measurement of spatiotemporal epigenetic-dependent changes in protein levels across more than one generation. In past studies, we conducted these analyses manually to obtain a dendrogram, but this required more than 15 h, even for a single set of microscopic cell images. Thus, in this study, we developed a software-based analysis system to analyze time-lapse cellular images of S. cerevisiae, which allowed automatic generation of a dendrogram from a given set of time-lapse cell images. This approach is divided into two phases: a cell extraction and tracking phase, and an analysis phase. The cell extraction and tracking phase generates a set of necessary information for each cell, such as geometrical properties and the daughter-mother relationships, using image processing-based analysis techniques. Then, based on this information, the analysis phase allows generation of the final dendrogram by analyzing the fluorescent characteristics of each cell. The system is equipped with manual error correction to correct for the inevitable errors that occur in these analyses. The time required to obtain the final dendrograms was drastically reduced from 15 h in manual analysis to 0.8 h using this novel system.

SeeVis-3D space-time cube rendering for visualization of microfluidics image data

Motivation

Live cell imaging plays a pivotal role in understanding cell growth. Yet, there is a lack of visualization alternatives for quick qualitative characterization of colonies.

Results

SeeVis is a Python workflow for automated and qualitative visualization of time-lapse microscopy data. It automatically pre-processes the movie frames, finds particles, traces their trajectories and visualizes them in a space-time cube offering three different color mappings to highlight different features. It supports the user in developing a mental model for the data. SeeVis completes these steps in 1.15 s/frame and creates a visualization with a selected color mapping.

Availability and implementation

https://github.com/ghattab/seevis/

Supplementary information

Supplementary data are available at Bioinformatics online.

Automatic detection of circulating tumor cells in darkfield microscopic images of unstained blood using boosting techniques

Circulating tumor cells (CTCs) are nowadays one of the most promising tumor biomarkers. It is well correlated with overall survival and progression-free survival in breast cancer, as well as in many other types of human cancer. In addition, enumeration and analysis of CTCs could be important for monitoring the response to different therapeutic agents, thus guiding the treatment of cancer patients and offering the promise of a more personalized approach. In this article, we present a new method that could be used for the automatic detection of CTC in blood, based on the microscopic appearance of unstained cells. The proposed method is based on the evaluation of image characteristics and boosting techniques. A dataset of 263 dark field microscopy images was constructed and used for our tests, containing blood spiked with three different types of tumor cells. An overall sensitivity of 92.87% and a specificity of 99.98% were obtained for the detection of CTC, performances which proved to be comparable to those obtained by human experts.

Phase contrast time-lapse microscopy datasets with automated and manual cell tracking annotations

Phase contrast time-lapse microscopy is a non-destructive technique that generates large volumes of image-based information to quantify the behaviour of individual cells or cell populations. To guide the development of algorithms for computer-aided cell tracking and analysis, 48 time-lapse image sequences, each spanning approximately 3.5 days, were generated with accompanying ground truths for C2C12 myoblast cells cultured under 4 different media conditions, including with fibroblast growth factor 2 (FGF2), bone morphogenetic protein 2 (BMP2), FGF2 + BMP2, and control (no growth factor). The ground truths generated contain information for tracking at least 3 parent cells and their descendants within these datasets and were validated using a two-tier system of manual curation. This comprehensive, validated dataset will be useful in advancing the development of computer-aided cell tracking algorithms and function as a benchmark, providing an invaluable opportunity to deepen our understanding of individual and population-based cell dynamics for biomedical research.

A Dynamic-Shape-Prior Guided Snake Model with Application in Visually Tracking Dense Cell Populations

This work proposes a dynamic-shape-prior guided snake model (DSP G-snake) that is designed to improve the overall stability of the point-based snake model. The dynamic shape prior is first proposed for snakes, that efficiently unifies different types of high-level priors into a new force term. To be specific, a global-topology regularity is first introduced that settles the inherent self-intersection problem with snakes. The problem that a snake's snaxels tend to unevenly distribute along the contour is also handled, leading to good parameterization. Unlike existing methods that employ learning templates or commonly enforce hard priors, the dynamic-template scheme strongly respects the deformation flexibility of the model, while retaining a decent global topology for the snake. It is verified by experiments that the proposed algorithm can effectively prevent snakes from self-crossing, or automatically untie an already selfintersected contour. In addition, the proposed model is combined with existing forces and applied to the very challenging task of tracking dense biological cell populations. The DSP G-snake model has enabled an improvement of up to 30% in tracking accuracy with respect to regular model-based approaches. Through experiments on real cellular datasets, with highly dense populations and relatively large displacements, it is confirmed that the proposed approach has enabled superior performance, in comparison to modern active-contour competitors as well as state-of-the-art cell tracking frameworks.

Automated Bright Field Segmentation of Cells and Vacuoles Using Image Processing Technique

Understanding the mechanisms and other variants of programmed cell death will help provide deeper insight into various disease processes. Although complex procedures are required to distinguish each type of cell death, the formation of vacuoles is one of the important features in some process of cell death under different conditions. Thus, monitoring and counting the number of vacuoles and the ratio of cells with vacuoles is a commonly used method to indicate and quantify the efficacy of the therapy. Several studies have shown that image processing can provide a quick, convenient and precise mean of performing cell detection. Hence, this study uses an image processing technique to detect and quantify vacuolated cells without the need for dyes. The system both counts the number of vacuolated cells and determines the ratio of cells with vacuoles. The performance of the proposed image processing system was evaluated using 38 images. It has been shown that a strong correlation exists between the automated counts and the manual counts. Furthermore, the absolute percentage errors between automated counts and manual counts for cell detection and vacuolated cell detection using data pooled from all images are 3.61 and 3.33%, respectively. A user-friendly graphical user interface (GUI) is also developed and freely available for download, providing researchers in biomedicine with a more convenient instrument for vacuolization analysis.

Cell Population Tracking in a Honeycomb Structure Using an IMM Filter Based 3D Local Graph Matching Model

Developing algorithms for plant cell population tracking is very critical for the modeling of plant cell growth pattern and gene expression dynamics. The tracking of plant cells in microscopic image stacks is very challenging for several reasons: (1) plant cells are densely packed in a specific honeycomb structure; (2) they are frequently dividing; and (3) they are imaged in different layers within 3D image stacks. Based on an existing 2D local graph matching algorithm, this paper focuses on building a 3D plant cell matching model, by exploiting the cells' 3D spatiotemporal context. Furthermore, the Interacting Multi-Model filter (IMM) is combined with the 3D local graph matching model to track the plant cell population simultaneously. Because our tracking algorithm does not require the identification of "tracking seeds", the tracking stability and efficiency are greatly enhanced. Last, the plant cell lineages are achieved by associating the cell tracklets, using a maximum-a-posteriori (MAP) method. Compared with the 2D matching method, the experimental results on multiple datasets show that our proposed approach does not only greatly improve the tracking accuracy by 18 percent, but also successfully tracks the plant cells located at the high curvature primordial region, which is not addressed in previous work.

Automated noninvasive epithelial cell counting in phase contrast microscopy images with automated parameter selection

Cell counting is commonly used to determine proliferation rates in cell cultures and for adherent cells it is often a ‘destructive’ process requiring disruption of the cell monolayer resulting in the inability to follow cell growth longitudinally. This process is time consuming and utilises significant resource. In this study a relatively inexpensive, rapid and widely applicable phase contrast microscopy‐based technique has been developed that emulates the contrast changes taking place when bright field microscope images of epithelial cell cultures are defocused. Processing of the resulting images produces an image that can be segmented using a global threshold; the number of cells is then deduced from the number of segmented regions and these cell counts can be used to generate growth curves. The parameters of this method were tuned using the discrete mereotopological relations between ground truth and processed images. Cell count accuracy was improved using linear discriminant analysis to identify spurious noise regions for removal.

The proposed cell counting technique was validated by comparing the results with a manual count of cells in images, and subsequently applied to generate growth curves for oral keratinocyte cultures supplemented with a range of concentrations of foetal calf serum. The approach developed has broad applicability and utility for researchers with standard laboratory imaging equipment.

A probabilistic approach to joint cell tracking and segmentation in high-throughput microscopy videos

We present a novel computational framework for the analysis of high-throughput microscopy videos of living cells. The proposed framework is generally useful and can be applied to different datasets acquired in a variety of laboratory settings. This is accomplished by tying together two fundamental aspects of cell lineage construction, namely cell segmentation and tracking, via a Bayesian inference of dynamic models. In contrast to most existing approaches, which aim to be general, no assumption of cell shape is made. Spatial, temporal, and cross-sectional variation of the analysed data are accommodated by two key contributions. First, time series analysis is exploited to estimate the temporal cell shape uncertainty in addition to cell trajectory. Second, a fast marching (FM) algorithm is used to integrate the inferred cell properties with the observed image measurements in order to obtain image likelihood for cell segmentation, and association. The proposed approach has been tested on eight different time-lapse microscopy data sets, some of which are high-throughput, demonstrating promising results for the detection, segmentation and association of planar cells. Our results surpass the state of the art for the Fluo-C2DL-MSC data set of the Cell Tracking Challenge (Maška et al., 2014).

Automated Cell Segmentation for Quantitative Phase Microscopy

Automated cell segmentation and tracking is essential for dynamic studies of cellular morphology, movement, and interactions as well as other cellular behaviors. However, accurate, automated, and easy-to-use cell segmentation remains a challenge, especially in cases of high cell densities, where discrete boundaries are not easily discernable. Here, we present a fully automated segmentation algorithm that iteratively segments cells based on the observed distribution of optical cell volumes measured by quantitative phase microscopy. By fitting these distributions to known probability density functions, we are able to converge on volumetric thresholds that enable valid segmentation cuts. Since each threshold is determined from the observed data itself, virtually no input is needed from the user. We demonstrate the effectiveness of this approach over time using six cell types that display a range of morphologies, and evaluate these cultures over a range of confluencies. Facile dynamic measures of cell mobility and function revealed unique cellular behaviors that relate to tissue origins, state of differentiation, and real-time signaling. These will improve our understanding of multicellular communication and organization.

A Novel Methodology for Characterizing Cell Subpopulations in Automated Time-lapse Microscopy

Time-lapse imaging of cell colonies in microfluidic chambers provides time series of bioimages, i.e., biomovies. They show the behavior of cells over time under controlled conditions. One of the main remaining bottlenecks in this area of research is the analysis of experimental data and the extraction of cell growth characteristics, such as lineage information. The extraction of the cell line by human observers is time-consuming and error-prone. Previously proposed methods often fail because of their reliance on the accurate detection of a single cell, which is not possible for high density, high diversity of cell shapes and numbers, and high-resolution images with high noise. Our task is to characterize subpopulations in biomovies. In order to shift the analysis of the data from individual cell level to cellular groups with similar fluorescence or even subpopulations, we propose to represent the cells by two new abstractions: the particle and the patch. We use a three-step framework: preprocessing, particle tracking, and construction of the patch lineage. First, preprocessing improves the signal-to-noise ratio and spatially aligns the biomovie frames. Second, cell sampling is performed by assuming particles, which represent a part of a cell, cell or group of contiguous cells in space. Particle analysis includes the following: particle tracking, trajectory linking, filtering, and color information, respectively. Particle tracking consists of following the spatiotemporal position of a particle and gives rise to coherent particle trajectories over time. Typical tracking problems may occur (e.g., appearance or disappearance of cells, spurious artifacts). They are effectively processed using trajectory linking and filtering. Third, the construction of the patch lineage consists in joining particle trajectories that share common attributes (i.e., proximity and fluorescence intensity) and feature common ancestry. This step is based on patch finding, patching trajectory propagation, patch splitting, and patch merging. The main idea is to group together the trajectories of particles in order to gain spatial coherence. The final result of CYCASP is the complete graph of the patch lineage. Finally, the graph encodes the temporal and spatial coherence of the development of cellular colonies. We present results showing a computation time of less than 5 min for biomovies and simulated films. The method, presented here, allowed for the separation of colonies into subpopulations and allowed us to interpret the growth of colonies in a timely manner.

Evaluation of epidermal growth factor receptor signaling effects in gastric cancer cell lines by detailed motility-focused phenotypic characterization linked with molecular analysis

Background

Gastric cancers frequently overexpress the epidermal growth factor receptor (EGFR), which has been implicated in pathological processes including tumor cell motility, invasion and metastasis. Targeting EGFR with the inhibitory antibody cetuximab may affect the motile and invasive behavior of tumor cells. Here, we evaluated the effects of EGFR signaling in gastric cancer cell lines to link the phenotypic behavior of the cells with their molecular characteristics.

Methods

Phenotypic effects were analyzed in four gastric cancer cell lines (AGS, Hs746T, LMSU and MKN1) by time-lapse microscopy and transwell invasion assay. Effects on EGFR signaling were detected using Western blot and proteome profiler analyses. A network was constructed linking EGFR signaling to the regulation of cellular motility.

Results

The analysis of the effects of treatment with epidermal growth factor (EGF) and cetuximab revealed that only one cell line (MKN1) was sensitive to cetuximab treatment in all phenotypic assays, whereas the other cell lines were either not responsive (Hs746T, LMSU) or sensitive only in certain tests (AGS). Cetuximab inhibited EGFR, MAPK and AKT activity and associated components of the EGFR signaling pathway to different degrees in cetuximab-sensitive MKN1 cells. In contrast, no such changes were observed in Hs746T cells. Thus, the different phenotypic behaviors of the cells were linked to their molecular response to treatment. Genetic alterations had different associations with response to treatment: while PIK3CA mutations and KRAS mutation or amplification were not obstructive, the MET mutation was associated with non-response.

Conclusion

These results identify components of the EGFR signaling network as important regulators of the phenotypic and molecular response to cetuximab treatment.

Electronic supplementary material

The online version of this article (10.1186/s12885-017-3822-3) contains supplementary material, which is available to authorized users.

An objective comparison of cell-tracking algorithms

We present a combined report on the results of three editions of the Cell Tracking Challenge, an ongoing initiative aimed at promoting the development and objective evaluation of cell segmentation and tracking algorithms. With 21 participating algorithms and a data repository consisting of 13 data sets from various microscopy modalities, the challenge displays today's state-of-the-art methodology in the field. We analyzed the challenge results using performance measures for segmentation and tracking that rank all participating methods. We also analyzed the performance of all of the algorithms in terms of biological measures and practical usability. Although some methods scored high in all technical aspects, none obtained fully correct solutions. We found that methods that either take prior information into account using learning strategies or analyze cells in a global spatiotemporal video context performed better than other methods under the segmentation and tracking scenarios included in the challenge.

Quantitative profiling of innate immune activation by viral infection in single cells

Cells infected by viruses can exhibit diverse patterns of viral and cellular gene expression. The patterns arise in part from the stochastic or noisy reaction kinetics associated with the small number of genomes, enzymes, and other molecules that typically initiate virus replication and activate cellular anti-viral defenses. It is not known what features, if any, of the early viral or cellular gene expression correlate with later processes of viral replication or cell survival. Here we used two fluorescent reporters to visualize innate immune activation of human prostate cancer (PC3) cells against infection by vesicular stomatitis virus. The cells were engineered to express green-fluorescent protein under control of the promoter for IFIT2, an interferon-sensitive component of the anti-viral response, while red-fluorescent protein was expressed as a byproduct of virus infection. To isolate and quantitatively analyze single-cells, we used a unique microwell array device and open-source image processing software. Kinetic analysis of viral and cellular reporter profiles from hundreds of cells revealed novel relationships between gene expression and the outcome of infection. Specifically, the relative timing rather than the magnitude of the viral gene expression and innate immune activation correlated with the infection outcome. Earlier viral or anti-viral gene expression favored or hindered virus growth, respectively. Further, analysis of kinetic parameters estimated from these data suggests a trade-off between robust antiviral signaling and cell death, as indicated by a higher rate of detectable cell lysis in infected cells with a detectable immune response. In short, cells that activate an immune response lyse at a higher rate. More broadly, we demonstrate how the intrinsic heterogeneity of individual cell behaviors can be exploited to discover features of viral and host gene expression that correlate with single-cell outcomes, which will ultimately impact whether or not infections spread.

Trajectory energy minimization for cell growth tracking and genealogy analysis

Cell growth experiments with a microfluidic device produce large-scale time-lapse image data, which contain important information on cell growth and patterns in their genealogy. To extract such information, we propose a scheme to segment and track bacterial cells automatically. In contrast with most published approaches, which often split segmentation and tracking into two independent procedures, we focus on designing an algorithm that describes cell properties evolving between consecutive frames by feeding segmentation and tracking results from one frame to the next one. The cell boundaries are extracted by minimizing the distance regularized level set evolution (DRLSE) model. Each individual cell was identified and tracked by identifying cell septum and membrane as well as developing a trajectory energy minimization function along time-lapse series. Experiments show that by applying this scheme, cell growth and division can be measured automatically. The results show the efficiency of the approach when testing on different datasets while comparing with other existing algorithms. The proposed approach demonstrates great potential for large-scale bacterial cell growth analysis.

Network Flow Integer Programming to Track Elliptical Cells in Time-Lapse Sequences

We propose a novel approach to automatically tracking elliptical cell populations in time-lapse image sequences. Given an initial segmentation, we account for partial occlusions and overlaps by generating an over-complete set of competing detection hypotheses. To this end, we fit ellipses to portions of the initial regions and build a hierarchy of ellipses, which are then treated as cell candidates. We then select temporally consistent ones by solving to optimality an integer program with only one type of flow variables. This eliminates the need for heuristics to handle missed detections due to partial occlusions and complex morphology. We demonstrate the effectiveness of our approach on a range of challenging sequences consisting of clumped cells and show that it outperforms state-of-the-art techniques.

Machine learning applications in cell image analysis

Machine learning (ML) refers to a set of automatic pattern recognition methods that have been successfully applied across various problem domains, including biomedical image analysis. This review focuses on ML applications for image analysis in light microscopy experiments with typical tasks of segmenting and tracking individual cells, and modelling of reconstructed lineage trees. After describing a typical image analysis pipeline and highlighting challenges of automatic analysis (for example, variability in cell morphology, tracking in presence of clutters) this review gives a brief historical outlook of ML, followed by basic concepts and definitions required for understanding examples. This article then presents several example applications at various image processing stages, including the use of supervised learning methods for improving cell segmentation, and the application of active learning for tracking. The review concludes with remarks on parameter setting and future directions.

Mathematical imaging methods for mitosis analysis in live-cell phase contrast microscopy

In this paper we propose a workflow to detect and track mitotic cells in time-lapse microscopy image sequences. In order to avoid the requirement for cell lines expressing fluorescent markers and the associated phototoxicity, phase contrast microscopy is often preferred over fluorescence microscopy in live-cell imaging. However, common specific image characteristics complicate image processing and impede use of standard methods. Nevertheless, automated analysis is desirable due to manual analysis being subjective, biased and extremely time-consuming for large data sets. Here, we present the following workflow based on mathematical imaging methods. In the first step, mitosis detection is performed by means of the circular Hough transform. The obtained circular contour subsequently serves as an initialisation for the tracking algorithm based on variational methods. It is sub-divided into two parts: in order to determine the beginning of the whole mitosis cycle, a backwards tracking procedure is performed. After that, the cell is tracked forwards in time until the end of mitosis. As a result, the average of mitosis duration and ratios of different cell fates (cell death, no division, division into two or more daughter cells) can be measured and statistics on cell morphologies can be obtained. All of the tools are featured in the user-friendly MATLAB®Graphical User Interface MitosisAnalyser.

Lineage mapper: A versatile cell and particle tracker

The ability to accurately track cells and particles from images is critical to many biomedical problems. To address this, we developed Lineage Mapper, an open-source tracker for time-lapse images of biological cells, colonies, and particles. Lineage Mapper tracks objects independently of the segmentation method, detects mitosis in confluence, separates cell clumps mistakenly segmented as a single cell, provides accuracy and scalability even on terabyte-sized datasets, and creates division and/or fusion lineages. Lineage Mapper has been tested and validated on multiple biological and simulated problems. The software is available in ImageJ and Matlab at isg.nist.gov.

Measuring Process Dynamics and Nuclear Migration for Clones of Neural Progenitor Cells

Neural stem and progenitor cells (NPCs) generate processes that extend from the cell body in a dynamic manner. The NPC nucleus migrates along these processes with patterns believed to be tightly coupled to mechanisms of cell cycle regulation and cell fate determination. Here, we describe a new segmentation and tracking approach that allows NPC processes and nuclei to be reliably tracked across multiple rounds of cell division in phase-contrast microscopy images. Results are presented for mouse adult and embryonic NPCs from hundreds of clones, or lineage trees, containing tens of thousands of cells and millions of segmentations. New visualization approaches allow the NPC nuclear and process features to be effectively visualized for an entire clone. Significant differences in process and nuclear dynamics were found among type A and type C adult NPCs, and also between embryonic NPCs cultured from the anterior and posterior cerebral cortex.

Network Consistent Data Association

Existing data association techniques mostly focus on matching pairs of data-point sets and then repeating this process along space-time to achieve long term correspondences. However, in many problems such as person re-identification, a set of data-points may be observed at multiple spatio-temporal locations and/or by multiple agents in a network and simply combining the local pairwise association results between sets of data-points often leads to inconsistencies over the global space-time horizons. In this paper, we propose a Novel Network Consistent Data Association (NCDA) framework formulated as an optimization problem that not only maintains consistency in association results across the network, but also improves the pairwise data association accuracies. The proposed NCDA can be solved as a binary integer program leading to a globally optimal solution and is capable of handling the challenging data-association scenario where the number of data-points varies across different sets of instances in the network. We also present an online implementation of NCDA method that can dynamically associate new observations to already observed data-points in an iterative fashion, while maintaining network consistency. We have tested both the batch and the online NCDA in two application areas-person re-identification and spatio-temporal cell tracking and observed consistent and highly accurate data association results in all the cases.

A Hybrid Approach for Segmentation and Tracking of Myxococcus Xanthus Swarms

Cell segmentation and motion tracking in time-lapse images are fundamental problems in computer vision, and are also crucial for various biomedical studies. Myxococcus xanthus is a type of rod-like cells with highly coordinated motion. The segmentation and tracking of M. xanthus are challenging, because cells may touch tightly and form dense swarms that are difficult to identify individually in an accurate manner. The known cell tracking approaches mainly fall into two frameworks, detection association and model evolution, each having its own advantages and disadvantages. In this paper, we propose a new hybrid framework combining these two frameworks into one and leveraging their complementary advantages. Also, we propose an active contour model based on the Ribbon Snake, which is seamlessly integrated with our hybrid framework. Evaluated by 10 different datasets, our approach achieves considerable improvement over the state-of-the-art cell tracking algorithms on identifying complete cell trajectories, and higher segmentation accuracy than performing segmentation in individual 2D images.

Non-invasive single-cell biomechanical analysis using live-imaging datasets

The physiological state of a cell is governed by a multitude of processes and can be described by a combination of mechanical, spatial and temporal properties. Quantifying cell dynamics at multiple scales is essential for comprehensive studies of cellular function, and remains a challenge for traditional end-point assays. We introduce an efficient, non-invasive computational tool that takes time-lapse images as input to automatically detect, segment and analyze unlabeled live cells; the program then outputs kinematic cellular shape and migration parameters, while simultaneously measuring cellular stiffness and viscosity. We demonstrate the capabilities of the program by testing it on human mesenchymal stem cells (huMSCs) induced to differentiate towards the osteoblastic (huOB) lineage, and T-lymphocyte cells (T cells) of naïve and stimulated phenotypes. The program detected relative cellular stiffness differences in huMSCs and huOBs that were comparable to those obtained with studies that utilize atomic force microscopy; it further distinguished naïve from stimulated T cells, based on characteristics necessary to invoke an immune response. In summary, we introduce an integrated tool to decipher spatiotemporal and intracellular dynamics of cells, providing a new and alternative approach for cell characterization.

Deformable Graph Model for Tracking Epithelial Cell Sheets in Fluorescence Microscopy

We propose a novel method for tracking cells that are connected through a visible network of membrane junctions. Tissues of this form are common in epithelial cell sheets and resemble planar graphs where each face corresponds to a cell. We leverage this structure and develop a method to track the entire tissue as a deformable graph. This coupled model in which vertices inform the optimal placement of edges and vice versa captures global relationships between tissue components and leads to accurate and robust cell tracking. We compare the performance of our method with that of four reference tracking algorithms on four data sets that present unique tracking challenges. Our method exhibits consistently superior performance in tracking all cells accurately over all image frames, and is robust over a wide range of image intensity and cell shape profiles. This may be an important tool for characterizing tissues of this type especially in the field of developmental biology where automated cell analysis can help elucidate the mechanisms behind controlled cell-shape changes.

A computational framework for particle and whole cell tracking applied to a real biological dataset

Cell tracking is becoming increasingly important in cell biology as it provides a valuable tool for analysing experimental data and hence furthering our understanding of dynamic cellular phenomena. The advent of high-throughput, high-resolution microscopy and imaging techniques means that a wealth of large data is routinely generated in many laboratories. Due to the sheer magnitude of the data involved manual tracking is often cumbersome and the development of computer algorithms for automated cell tracking is thus highly desirable. In this work, we describe two approaches for automated cell tracking. Firstly, we consider particle tracking. We propose a few segmentation techniques for the detection of cells migrating in a non-uniform background, centroids of the segmented cells are then calculated and linked from frame to frame via a nearest-neighbour approach. Secondly, we consider the problem of whole cell tracking in which one wishes to reconstruct in time whole cell morphologies. Our approach is based on fitting a mathematical model to the experimental imaging data with the goal being that the physics encoded in the model is reflected in the reconstructed data. The resulting mathematical problem involves the optimal control of a phase-field formulation of a geometric evolution law. Efficient approximation of this challenging optimal control problem is achieved via advanced numerical methods for the solution of semilinear parabolic partial differential equations (PDEs) coupled with parallelisation and adaptive resolution techniques. Along with a detailed description of our algorithms, a number of simulation results are reported on. We focus on illustrating the effectivity of our approaches by applying the algorithms to the tracking of migrating cells in a dataset which reflects many of the challenges typically encountered in microscopy data.

Tools for Single-Cell Kinetic Analysis of Virus-Host Interactions

Measures of cellular gene expression or behavior, when performed on individual cells, inevitably reveal a diversity of behaviors and outcomes that can correlate with normal or diseased states. For virus infections, the potential diversity of outcomes are pushed to an extreme, where measures of infection reflect features of the specific infecting virus particle, the individual host cell, as well as interactions between viral and cellular components. Single-cell measures, while revealing, still often rely on specialized fluid handling capabilities, employ end-point measures, and remain labor-intensive to perform. To address these limitations, we consider a new microwell-based device that uses simple pipette-based fluid handling to isolate individual cells. Our design allows different experimental conditions to be implemented in a single device, permitting easier and more standardized protocols. Further, we utilize a recently reported dual-color fluorescent reporter system that provides dynamic readouts of viral and cellular gene expression during single-cell infections by vesicular stomatitis virus. In addition, we develop and show how free, open-source software can enable streamlined data management and batch image analysis. Here we validate the integration of the device and software using the reporter system to demonstrate unique single-cell dynamic measures of cellular responses to viral infection.

Robust Nucleus/Cell Detection and Segmentation in Digital Pathology and Microscopy Images: A Comprehensive Review

Digital pathology and microscopy image analysis is widely used for comprehensive studies of cell morphology or tissue structure. Manual assessment is labor intensive and prone to interobserver variations. Computer-aided methods, which can significantly improve the objectivity and reproducibility, have attracted a great deal of interest in recent literature. Among the pipeline of building a computer-aided diagnosis system, nucleus or cell detection and segmentation play a very important role to describe the molecular morphological information. In the past few decades, many efforts have been devoted to automated nucleus/cell detection and segmentation. In this review, we provide a comprehensive summary of the recent state-of-the-art nucleus/cell segmentation approaches on different types of microscopy images including bright-field, phase-contrast, differential interference contrast, fluorescence, and electron microscopies. In addition, we discuss the challenges for the current methods and the potential future work of nucleus/cell detection and segmentation.

Cell Tracking Accuracy Measurement Based on Comparison of Acyclic Oriented Graphs

Tracking motile cells in time-lapse series is challenging and is required in many biomedical applications. Cell tracks can be mathematically represented as acyclic oriented graphs. Their vertices describe the spatio-temporal locations of individual cells, whereas the edges represent temporal relationships between them. Such a representation maintains the knowledge of all important cellular events within a captured field of view, such as migration, division, death, and transit through the field of view. The increasing number of cell tracking algorithms calls for comparison of their performance. However, the lack of a standardized cell tracking accuracy measure makes the comparison impracticable. This paper defines and evaluates an accuracy measure for objective and systematic benchmarking of cell tracking algorithms. The measure assumes the existence of a ground-truth reference, and assesses how difficult it is to transform a computed graph into the reference one. The difficulty is measured as a weighted sum of the lowest number of graph operations, such as split, delete, and add a vertex and delete, add, and alter the semantics of an edge, needed to make the graphs identical. The measure behavior is extensively analyzed based on the tracking results provided by the participants of the first Cell Tracking Challenge hosted by the 2013 IEEE International Symposium on Biomedical Imaging. We demonstrate the robustness and stability of the measure against small changes in the choice of weights for diverse cell tracking algorithms and fluorescence microscopy datasets. As the measure penalizes all possible errors in the tracking results and is easy to compute, it may especially help developers and analysts to tune their algorithms according to their needs.

CellTracker (not only) for dummies

MOTIVATION

Time-lapse experiments play a key role in studying the dynamic behavior of cells. Single-cell tracking is one of the fundamental tools for such analyses. The vast majority of the recently introduced cell tracking methods are limited to fluorescently labeled cells. An equally important limitation is that most software cannot be effectively used by biologists without reasonable expertise in image processing. Here we present CellTracker, a user-friendly open-source software tool for tracking cells imaged with various imaging modalities, including fluorescent, phase contrast and differential interference contrast (DIC) techniques.

AVAILABILITY AND IMPLEMENTATION

CellTracker is written in MATLAB (The MathWorks, Inc., USA). It works with Windows, Macintosh and UNIX-based systems. Source code and graphical user interface (GUI) are freely available at: http://celltracker.website/

CONTACT

horvath.peter@brc.mta.hu

SUPPLEMENTARY INFORMATION

Supplementary data are available at Bioinformatics online.