Cell population tracking and lineage construction with spatiotemporal context

CellProfiler Tracer: exploring and validating high-throughput, time-lapse microscopy image data

Background

Time-lapse analysis of cellular images is an important and growing need in biology. Algorithms for cell tracking are widely available; what researchers have been missing is a single open-source software package to visualize standard tracking output (from software like CellProfiler) in a way that allows convenient assessment of track quality, especially for researchers tuning tracking parameters for high-content time-lapse experiments. This makes quality assessment and algorithm adjustment a substantial challenge, particularly when dealing with hundreds of time-lapse movies collected in a high-throughput manner.

Results

We present CellProfiler Tracer, a free and open-source tool that complements the object tracking functionality of the CellProfiler biological image analysis package. Tracer allows multi-parametric morphological data to be visualized on object tracks, providing visualizations that have already been validated within the scientific community for time-lapse experiments, and combining them with simple graph-based measures for highlighting possible tracking artifacts.

Conclusions

CellProfiler Tracer is a useful, free tool for inspection and quality control of object tracking data, available from http://www.cellprofiler.org/tracer/.

Electronic supplementary material

The online version of this article (doi:10.1186/s12859-015-0759-x) contains supplementary material, which is available to authorized users.

Survey statistics of automated segmentations applied to optical imaging of mammalian cells

BACKGROUND

The goal of this survey paper is to overview cellular measurements using optical microscopy imaging followed by automated image segmentation. The cellular measurements of primary interest are taken from mammalian cells and their components. They are denoted as two- or three-dimensional (2D or 3D) image objects of biological interest. In our applications, such cellular measurements are important for understanding cell phenomena, such as cell counts, cell-scaffold interactions, cell colony growth rates, or cell pluripotency stability, as well as for establishing quality metrics for stem cell therapies. In this context, this survey paper is focused on automated segmentation as a software-based measurement leading to quantitative cellular measurements.

METHODS

We define the scope of this survey and a classification schema first. Next, all found and manually filteredpublications are classified according to the main categories: (1) objects of interests (or objects to be segmented), (2) imaging modalities, (3) digital data axes, (4) segmentation algorithms, (5) segmentation evaluations, (6) computational hardware platforms used for segmentation acceleration, and (7) object (cellular) measurements. Finally, all classified papers are converted programmatically into a set of hyperlinked web pages with occurrence and co-occurrence statistics of assigned categories.

RESULTS

The survey paper presents to a reader: (a) the state-of-the-art overview of published papers about automated segmentation applied to optical microscopy imaging of mammalian cells, (b) a classification of segmentation aspects in the context of cell optical imaging, (c) histogram and co-occurrence summary statistics about cellular measurements, segmentations, segmented objects, segmentation evaluations, and the use of computational platforms for accelerating segmentation execution, and (d) open research problems to pursue.

CONCLUSIONS

The novel contributions of this survey paper are: (1) a new type of classification of cellular measurements and automated segmentation, (2) statistics about the published literature, and (3) a web hyperlinked interface to classification statistics of the surveyed papers at https://isg.nist.gov/deepzoomweb/resources/survey/index.html.

Computational Image Analysis Reveals Intrinsic Multigenerational Differences between Anterior and Posterior Cerebral Cortex Neural Progenitor Cells

Time-lapse microscopy can capture patterns of development through multiple divisions for an entire clone of proliferating cells. Images are taken every few minutes over many days, generating data too vast to process completely by hand. Computational analysis of this data can benefit from occasional human guidance. Here we combine improved automated algorithms with minimized human validation to produce fully corrected segmentation, tracking, and lineaging results with dramatic reduction in effort. A web-based viewer provides access to data and results. The improved approach allows efficient analysis of large numbers of clones. Using this method, we studied populations of progenitor cells derived from the anterior and posterior embryonic mouse cerebral cortex, each growing in a standardized culture environment. Progenitors from the anterior cortex were smaller, less motile, and produced smaller clones compared to those from the posterior cortex, demonstrating cell-intrinsic differences that may contribute to the areal organization of the cerebral cortex.

•

Open-source automated software designed to track stem/progenitor clones in time-lapse movies

•

Software tools for easy data validation and visualization greatly improve efficiency

•

Lineage tree reconstruction from hundreds of embryonic mouse forebrain clones

•

Intrinsic differences in progenitor behavior from anterior/posterior cerebral cortex

Cell Detection From Redundant Candidate Regions Under Nonoverlapping Constraints

Cell detection in microscopy images is essential for automated cell behavior analysis including cell shape analysis and cell tracking. Robust cell detection in high-density and low-contrast images is still challenging since cells often touch and partially overlap, forming a cell cluster with blurry intercellular boundaries. In such cases, current methods tend to detect multiple cells as a cluster. If the control parameters are adjusted to separate the touching cells, other problems often occur: a single cell may be segmented into several regions, and cells in low-intensity regions may not be detected. To solve these problems, we first detect redundant candidate regions, which include many false positives but in turn very few false negatives, by allowing candidate regions to overlap with each other. Next, the score for how likely the candidate region contains the main part of a single cell is computed for each cell candidate using supervised learning. Then we select an optimal set of cell regions from the redundant regions under nonoverlapping constraints, where each selected region looks like a single cell and the selected regions do not overlap. We formulate this problem of optimal region selection as a binary linear programming problem under nonoverlapping constraints. We demonstrated the effectiveness of our method for several types of cells in microscopy images. Our method performed better than five representative methods, achieving an F-measure of over 0.9 for all data sets. Experimental application of the proposed method to 3-D images demonstrated that also works well for 3-D cell detection.

Segmentation of the Clustered Cells with Optimized Boundary Detection in Negative Phase Contrast Images

Cell image segmentation plays a central role in numerous biology studies and clinical applications. As a result, the development of cell image segmentation algorithms with high robustness and accuracy is attracting more and more attention. In this study, an automated cell image segmentation algorithm is developed to get improved cell image segmentation with respect to cell boundary detection and segmentation of the clustered cells for all cells in the field of view in negative phase contrast images. A new method which combines the thresholding method and edge based active contour method was proposed to optimize cell boundary detection. In order to segment clustered cells, the geographic peaks of cell light intensity were utilized to detect numbers and locations of the clustered cells. In this paper, the working principles of the algorithms are described. The influence of parameters in cell boundary detection and the selection of the threshold value on the final segmentation results are investigated. At last, the proposed algorithm is applied to the negative phase contrast images from different experiments. The performance of the proposed method is evaluated. Results show that the proposed method can achieve optimized cell boundary detection and highly accurate segmentation for clustered cells.

Automated profiling of individual cell-cell interactions from high-throughput time-lapse imaging microscopy in nanowell grids (TIMING)

MOTIVATION

There is a need for effective automated methods for profiling dynamic cell-cell interactions with single-cell resolution from high-throughput time-lapse imaging data, especially, the interactions between immune effector cells and tumor cells in adoptive immunotherapy.

RESULTS

Fluorescently labeled human T cells, natural killer cells (NK), and various target cells (NALM6, K562, EL4) were co-incubated on polydimethylsiloxane arrays of sub-nanoliter wells (nanowells), and imaged using multi-channel time-lapse microscopy. The proposed cell segmentation and tracking algorithms account for cell variability and exploit the nanowell confinement property to increase the yield of correctly analyzed nanowells from 45% (existing algorithms) to 98% for wells containing one effector and a single target, enabling automated quantification of cell locations, morphologies, movements, interactions, and deaths without the need for manual proofreading. Automated analysis of recordings from 12 different experiments demonstrated automated nanowell delineation accuracy >99%, automated cell segmentation accuracy >95%, and automated cell tracking accuracy of 90%, with default parameters, despite variations in illumination, staining, imaging noise, cell morphology, and cell clustering. An example analysis revealed that NK cells efficiently discriminate between live and dead targets by altering the duration of conjugation. The data also demonstrated that cytotoxic cells display higher motility than non-killers, both before and during contact.

CONTACT

broysam@central.uh.edu or nvaradar@central.uh.edu

SUPPLEMENTARY INFORMATION

Supplementary data are available at Bioinformatics online.

Multi-Target Tracking With Time-Varying Clutter Rate and Detection Profile: Application to Time-Lapse Cell Microscopy Sequences

Quantitative analysis of the dynamics of tiny cellular and sub-cellular structures, known as particles, in time-lapse cell microscopy sequences requires the development of a reliable multi-target tracking method capable of tracking numerous similar targets in the presence of high levels of noise, high target density, complex motion patterns and intricate interactions. In this paper, we propose a framework for tracking these structures based on the random finite set Bayesian filtering framework. We focus on challenging biological applications where image characteristics such as noise and background intensity change during the acquisition process. Under these conditions, detection methods usually fail to detect all particles and are often followed by missed detections and many spurious measurements with unknown and time-varying rates. To deal with this, we propose a bootstrap filter composed of an estimator and a tracker. The estimator adaptively estimates the required meta parameters for the tracker such as clutter rate and the detection probability of the targets, while the tracker estimates the state of the targets. Our results show that the proposed approach can outperform state-of-the-art particle trackers on both synthetic and real data in this regime.

Associating approximate paths and temporal sequences of noisy detections: Application to the recovery of spatio-temporal cancer cell trajectories

In this paper we address the problem of recovering spatio-temporal trajectories of cancer cells in phase contrast video-microscopy where the user provides the paths on which the cells are moving. The paths are purely spatial, without temporal information. To recover the temporal information associated to a given path we propose an approach based on automatic cell detection and on a graph-based shortest path search. The nodes in the graph consist of the projections of the cell detections onto the geometrical cell path. The edges relate nodes which correspond to different frames of the sequence and potentially to the same cell and trajectory. In this directed graph we search for the shortest path and use it to define a temporal parametrization of the corresponding geometrical cell path. An evaluation based on 286 paths of 7 phase contrast microscopy videos shows that our algorithm allows to recover 92% of trajectory points with respect to the associated ground truth. We compare our method with a state-of-the-art algorithm for semi-automated cell tracking in phase contrast microscopy which requires interactively placed starting points for the cells to track. The comparison shows that supporting geometrical paths in combination with our algorithm allow us to obtain more reliable cell trajectories.

Cell-sensitive phase contrast microscopy imaging by multiple exposures

We propose a novel way of imaging live cells in a Petri dish by the phase contrast microscope. By taking multiple exposures of phase contrast microscopy images on the same cell dish, we estimate a cell-sensitive camera response function which responds to cells' irradiance signals but generates a constant on non-cell background signal. The result of this new microscopy imaging is visually superior quality, which reveals the appearance details of cells and suppresses background noise near zero. Using the cell-sensitive microscopy imaging, cells' original irradiance signals are restored from all exposures and the irradiance signals on non-cell background regions are restored as a uniform constant (i.e., the imaging system is sensitive to cells only but insensitive to non-cell background). The restored irradiance signals greatly facilitate the cell segmentation by simple thresholding. The experimental results validate that high quality cell segmentation can be achieved by our approach.

Global linking of cell tracks using the Viterbi algorithm

Automated tracking of living cells in microscopy image sequences is an important and challenging problem. With this application in mind, we propose a global track linking algorithm, which links cell outlines generated by a segmentation algorithm into tracks. The algorithm adds tracks to the image sequence one at a time, in a way which uses information from the complete image sequence in every linking decision. This is achieved by finding the tracks which give the largest possible increases to a probabilistically motivated scoring function, using the Viterbi algorithm. We also present a novel way to alter previously created tracks when new tracks are created, thus mitigating the effects of error propagation. The algorithm can handle mitosis, apoptosis, and migration in and out of the imaged area, and can also deal with false positives, missed detections, and clusters of jointly segmented cells. The algorithm performance is demonstrated on two challenging datasets acquired using bright-field microscopy, but in principle, the algorithm can be used with any cell type and any imaging technique, presuming there is a suitable segmentation algorithm.

Segmentation and tracking of stem cells in time lapse microscopy to quantify dynamic behavioral changes during spheroid formation

Dynamic behavior of stem cells during in vitro development is diverse. Previous cell tracking studies have focused more on cell proliferation than on cell aggregation. However, the enhancement of cell proliferation in association with cell aggregation has been reported. In a previous study, we also demonstrated that the aggregation of adult human mesenchymal stem cells to form three-dimensional (3D) cellular spheroids helped maintain the expression of stemness marker genes in the cells. However, the dynamic behavioral changes triggered by spheroid formation remain to be investigated. A scheme of image processing techniques is proposed to meet this need. A hybrid-thresholding technique was first developed for efficient segmentation of cell clusters, after which a cell tracking method based on pair-matching with topological constraints was designed. Two morphological indices were derived to track the timing of 3D spheroid formation during the cellular aggregation process. Five cell motility indices measured from single cells and 3D spheroids were then compared. After confirmation of more than 90% correspondence between the results obtained by manual tracking and the proposed methods, an analysis of cellular behavior reveals a significant increase in motility in association with spheroid formation, consistent with a previous report that used a gene expression approach. This study proposed a systematic image analysis method to quantify the dynamic behavior of stem cells for stemness evaluation during cell culturing in vitro. Results demonstrated the validity of the developed platform in investigation of the dynamic behavior of cell aggregation in stem cell cultures in vitro.

Tracking multiple particles in fluorescence time-lapse microscopy images via probabilistic data association

Tracking subcellular structures as well as viral structures displayed as 'particles' in fluorescence microscopy images yields quantitative information on the underlying dynamical processes. We have developed an approach for tracking multiple fluorescent particles based on probabilistic data association. The approach combines a localization scheme that uses a bottom-up strategy based on the spot-enhancing filter as well as a top-down strategy based on an ellipsoidal sampling scheme that uses the Gaussian probability distributions computed by a Kalman filter. The localization scheme yields multiple measurements that are incorporated into the Kalman filter via a combined innovation, where the association probabilities are interpreted as weights calculated using an image likelihood. To track objects in close proximity, we compute the support of each image position relative to the neighboring objects of a tracked object and use this support to recalculate the weights. To cope with multiple motion models, we integrated the interacting multiple model algorithm. The approach has been successfully applied to synthetic 2-D and 3-D images as well as to real 2-D and 3-D microscopy images, and the performance has been quantified. In addition, the approach was successfully applied to the 2-D and 3-D image data of the recent Particle Tracking Challenge at the IEEE International Symposium on Biomedical Imaging (ISBI) 2012.

Semi-automated object tracking methods in biological imaging

Time-lapse imaging is a rich data source offering potential kinetic information of cellular activity and behavior. Tracking and extracting measurements of objects from time-lapse datasets are challenges that result from the complexity and dynamics of each object's motion and intensity or the appearance of new objects in the field of view. A wide range of strategies for proper data sampling, object detection, image analysis, and post-analysis interpretation are available. Theory and methods for single-particle tracking, spot detection, and object linking are discussed in this unit, as well as examples with step-by-step procedures for utilizing semi-automated software and visualization tools for achieving tracking results and interpreting this output.

Cell Processing Engineering for Regenerative Medicine : Noninvasive Cell Quality Estimation and Automatic Cell Processing

The cell processing engineering including automatic cell processing and noninvasive cell quality estimation of adherent mammalian cells for regenerative medicine was reviewed. Automatic cell processing necessary for the industrialization of regenerative medicine was introduced. The cell quality such as cell heterogeneity should be noninvasively estimated before transplantation to patient, because cultured cells are usually not homogeneous but heterogeneous and most protocols of regenerative medicine are autologous system. The differentiation level could be estimated by two-dimensional cell morphology analysis using a conventional phase-contrast microscope. The phase-shifting laser microscope (PLM) could determine laser phase shift at all pixel in a view, which is caused by the transmitted laser through cell, and might be more noninvasive and more useful than the atomic force microscope and digital holographic microscope. The noninvasive determination of the laser phase shift of a cell using a PLM was carried out to determine the three-dimensional cell morphology and estimate the cell cycle phase of each adhesive cell and the mean proliferation activity of a cell population. The noninvasive discrimination of cancer cells from normal cells by measuring the phase shift was performed based on the difference in cytoskeleton density. Chemical analysis of the culture supernatant was also useful to estimate the differentiation level of a cell population. A probe beam, an infrared beam, and Raman spectroscopy are useful for diagnosing the viability, apoptosis, and differentiation of each adhesive cell.

Cell-sensitive microscopy imaging for cell image segmentation

We propose a novel cell segmentation approach by estimating a cell-sensitive camera response function based on variously exposed phase contrast microscopy images on the same cell dish. Using the cell-sensitive microscopy imaging, cells' original irradiance signals are restored from all exposures and the irradiance signals on non-cell background regions are restored as a uniform constant (i.e., the imaging system is sensitive to cells only but insensitive to non-cell background). Cell segmentation is then performed on the restored irradiance signals by simple thresholding. The experimental results validate that high quality cell segmentation can be achieved by our approach.

Context aware spatio-temporal cell tracking in densely packed multilayer tissues

Modern live imaging technique enables us to observe the internal part of a tissue over time by generating serial optical images containing spatio-temporal slices of hundreds of tightly packed cells. Automated tracking of plant and animal cells from such time lapse live-imaging datasets of a developing multicellular tissue is required for quantitative, high throughput analysis of cell division, migration and cell growth. In this paper, we present a novel cell tracking method that exploits the tight spatial topology of neighboring cells in a multicellular field as contextual information and combines it with physical features of individual cells for generating reliable cell lineages. The 2D image slices of multicellular tissues are modeled as a conditional random field and pairwise cell to cell similarities are obtained by estimating marginal probability distributions through loopy belief propagation on this CRF. These similarity scores are further used in a spatio-temporal graph labeling problem to obtain the optimal and feasible set of correspondences between individual cell slices across the 4D image dataset. We present results on (3D+t) confocal image stacks of Arabidopsis shoot meristem and show that the method is capable of handling many visual analysis challenges associated with such cell tracking problems, viz. poor feature quality of individual cells, low SNR in parts of images, variable number of cells across slices and cell division detection.

A method for automatic segmentation of nuclei in phase-contrast images based on intensity, convexity and texture

This paper presents a method for automatic segmentation of nuclei in phase-contrast images using the intensity, convexity and texture of the nuclei. The proposed method consists of three main stages: preprocessing, h-maxima transformation-based marker controlled watershed segmentation ( h-TMC), and texture analysis. In the preprocessing stage, a top-hat filter is used to increase the contrast and suppress the non-uniform illumination, shading, and other imaging artifacts in the input image. The nuclei segmentation stage consists of a distance transformation, h-maxima transformation and watershed segmentation. These transformations utilize the intensity information and the convexity property of the nucleus for the purpose of detecting a single marker in every nucleus; these markers are then used in the h-TMC watershed algorithm to obtain segments of the nuclei. However, dust particles, imaging artifacts, or prolonged cell cytoplasm may falsely be segmented as nuclei at this stage, and thus may lead to an inaccurate analysis of the cell image. In order to identify and remove these non-nuclei segments, in the third stage a texture analysis is performed, that uses six of the Haralick measures along with the AdaBoost algorithm. The novelty of the proposed method is that it introduces a systematic framework that utilizes intensity, convexity, and texture information to achieve a high accuracy for automatic segmentation of nuclei in the phase-contrast images. Extensive experiments are performed demonstrating the superior performance ( precision = 0.948; recall = 0.924; F1-measure = 0.936; validation based on  ∼ 4850 manually-labeled nuclei) of the proposed method.

Fast, accurate reconstruction of cell lineages from large-scale fluorescence microscopy data

The comprehensive reconstruction of cell lineages in complex multicellular organisms is a central goal of developmental biology. We present an open-source computational framework for the segmentation and tracking of cell nuclei with high accuracy and speed. We demonstrate its (i) generality by reconstructing cell lineages in four-dimensional, terabyte-sized image data sets of fruit fly, zebrafish and mouse embryos acquired with three types of fluorescence microscopes, (ii) scalability by analyzing advanced stages of development with up to 20,000 cells per time point at 26,000 cells min(-1) on a single computer workstation and (iii) ease of use by adjusting only two parameters across all data sets and providing visualization and editing tools for efficient data curation. Our approach achieves on average 97.0% linkage accuracy across all species and imaging modalities. Using our system, we performed the first cell lineage reconstruction of early Drosophila melanogaster nervous system development, revealing neuroblast dynamics throughout an entire embryo.

Improved and robust detection of cell nuclei from four dimensional fluorescence images

Segmentation-free direct methods are quite efficient for automated nuclei extraction from high dimensional images. A few such methods do exist but most of them do not ensure algorithmic robustness to parameter and noise variations. In this research, we propose a method based on multiscale adaptive filtering for efficient and robust detection of nuclei centroids from four dimensional (4D) fluorescence images. A temporal feedback mechanism is employed between the enhancement and the initial detection steps of a typical direct method. We estimate the minimum and maximum nuclei diameters from the previous frame and feed back them as filter lengths for multiscale enhancement of the current frame. A radial intensity-gradient function is optimized at positions of initial centroids to estimate all nuclei diameters. This procedure continues for processing subsequent images in the sequence. Above mechanism thus ensures proper enhancement by automated estimation of major parameters. This brings robustness and safeguards the system against additive noises and effects from wrong parameters. Later, the method and its single-scale variant are simplified for further reduction of parameters. The proposed method is then extended for nuclei volume segmentation. The same optimization technique is applied to final centroid positions of the enhanced image and the estimated diameters are projected onto the binary candidate regions to segment nuclei volumes.Our method is finally integrated with a simple sequential tracking approach to establish nuclear trajectories in the 4D space. Experimental evaluations with five image-sequences (each having 271 3D sequential images) corresponding to five different mouse embryos show promising performances of our methods in terms of nuclear detection, segmentation, and tracking. A detail analysis with a sub-sequence of 101 3D images from an embryo reveals that the proposed method can improve the nuclei detection accuracy by 9 over the previous methods, which used inappropriate large valued parameters. Results also confirm that the proposed method and its variants achieve high detection accuracies ( 98 mean F-measure) irrespective of the large variations of filter parameters and noise levels.

A semi-local neighborhood-based framework for probabilistic cell lineage tracing

BACKGROUND

Advances in fluorescence labeling and imaging have made it possible to acquire in vivo records of complex biological processes. Analysis has lagged behind acquisition in part because of the difficulty and computational expense of accurate cell tracking. In vivo analysis requires, at minimum, tracking hundreds of cells over hundreds of time points in complex three dimensional environments. We address this challenge with a computational framework capable of efficiently and accurately tracing entire cell lineages.

RESULTS

The bulk of the tracking problem-tracking cells during interphase-is straightforward and can be executed with simple and fast methods. Difficult cases originate from detection errors and relatively rare large motions. Therefore, our method focuses computational effort on difficult cases identified by local increases in cell number. We force these cases into tentative cell track bifurcations, which define natural semi-local neighborhoods that permit Bayesian judgment about the underlying cell behavior. The bifurcation judgment process not only correctly tracks through cell divisions and large movements, but also offers corrections to detection errors. We demonstrate that this method enables large scale analysis of Caenorhabditis elegans development, an ideal validation platform because of an invariant cell lineage.

CONCLUSION

The high accuracy achieved by our method suggests that a bifurcation-based semi-local neighborhood provides sufficient information to recognize dependencies between nearby tracking choices, and to interpret difficult tracking cases without reverting to global optimization. Our method makes large amounts of lineage data accessible and opens the door to new types of statistical analysis of complex in vivo processes.

Quantitative characterization of cell behaviors through cell cycle progression via automated cell tracking

Cell behaviors are reflections of intracellular tension dynamics and play important roles in many cellular processes. In this study, temporal variations in cell geometry and cell motion through cell cycle progression were quantitatively characterized via automated cell tracking for MCF-10A non-transformed breast cells, MCF-7 non-invasive breast cancer cells, and MDA-MB-231 highly metastatic breast cancer cells. A new cell segmentation method, which combines the threshold method and our modified edge based active contour method, was applied to optimize cell boundary detection for all cells in the field-of-view. An automated cell-tracking program was implemented to conduct live cell tracking over 40 hours for the three cell lines. The cell boundary and location information was measured and aligned with cell cycle progression with constructed cell lineage trees. Cell behaviors were studied in terms of cell geometry and cell motion. For cell geometry, cell area and cell axis ratio were investigated. For cell motion, instantaneous migration speed, cell motion type, as well as cell motion range were analyzed. We applied a cell-based approach that allows us to examine and compare temporal variations of cell behavior along with cell cycle progression at a single cell level. Cell body geometry along with distribution of peripheral protrusion structures appears to be associated with cell motion features. Migration speed together with motion type and motion ranges are required to distinguish the three cell-lines examined. We found that cells dividing or overlapping vertically are unique features of cell malignancy for both MCF-7 and MDA-MB-231 cells, whereas abrupt changes in cell body geometry and cell motion during mitosis are unique to highly metastatic MDA-MB-231 cells. Taken together, our live cell tracking system serves as an invaluable tool to identify cell behaviors that are unique to malignant and/or highly metastatic breast cancer cells.

Heterogeneous structure of stem cells dynamics: statistical models and quantitative predictions

Understanding stem cell (SC) population dynamics is essential for developing models that can be used in basic science and medicine, to aid in predicting cells fate. These models can be used as tools e.g. in studying patho-physiological events at the cellular and tissue level, predicting (mal)functions along the developmental course, and personalized regenerative medicine. Using time-lapsed imaging and statistical tools, we show that the dynamics of SC populations involve a heterogeneous structure consisting of multiple sub-population behaviors. Using non-Gaussian statistical approaches, we identify the co-existence of fast and slow dividing subpopulations, and quiescent cells, in stem cells from three species. The mathematical analysis also shows that, instead of developing independently, SCs exhibit a time-dependent fractal behavior as they interact with each other through molecular and tactile signals. These findings suggest that more sophisticated models of SC dynamics should view SC populations as a collective and avoid the simplifying homogeneity assumption by accounting for the presence of more than one dividing sub-population, and their multi-fractal characteristics.

Bioimage Informatics in the context of Drosophila research

Modern biological research relies heavily on microscopic imaging. The advanced genetic toolkit of Drosophila makes it possible to label molecular and cellular components with unprecedented level of specificity necessitating the application of the most sophisticated imaging technologies. Imaging in Drosophila spans all scales from single molecules to the entire populations of adult organisms, from electron microscopy to live imaging of developmental processes. As the imaging approaches become more complex and ambitious, there is an increasing need for quantitative, computer-mediated image processing and analysis to make sense of the imagery. Bioimage Informatics is an emerging research field that covers all aspects of biological image analysis from data handling, through processing, to quantitative measurements, analysis and data presentation. Some of the most advanced, large scale projects, combining cutting edge imaging with complex bioimage informatics pipelines, are realized in the Drosophila research community. In this review, we discuss the current research in biological image analysis specifically relevant to the type of systems level image datasets that are uniquely available for the Drosophila model system. We focus on how state-of-the-art computer vision algorithms are impacting the ability of Drosophila researchers to analyze biological systems in space and time. We pay particular attention to how these algorithmic advances from computer science are made usable to practicing biologists through open source platforms and how biologists can themselves participate in their further development.

A novel multiple hypothesis based particle tracking method for clathrin mediated endocytosis analysis using fluorescence microscopy

In order to quantitatively analyze biological images and study underlying mechanisms of the cellular and subcellular processes, it is often required to track a large number of particles involved in these processes. Manual tracking can be performed by the biologists, but the workload is very heavy. In this paper, we present an automatic particle tracking method for analyzing an essential subcellular process, namely clathrin mediated endocytosis. The framework of the tracking method is an extension of the classical multiple hypothesis tracking (MHT), and it is designed to manage trajectories, solve data association problems, and handle pseudo-splitting/merging events. In the extended MHT framework, particle tracking becomes evaluating two types of hypotheses. The first one is the trajectory-related hypothesis, to test whether a recovered trajectory is correct, and the second one is the observation-related hypothesis, to test whether an observation from an image belongs to a real particle. Here, an observation refers to a detected particle and its feature vector. To detect the particles in 2D fluorescence images taken using total internal reflection microscopy, the images are segmented into regions, and the features of the particles are obtained by fitting Gaussian mixture models into each of the image regions. Specific models are developed according to the properties of the particles. The proposed tracking method is demonstrated on synthetic data under different scenarios and applied to real data.

Active structured learning for cell tracking: algorithm, framework, and usability

One distinguishing property of life is its temporal dynamics, and it is hence only natural that time lapse experiments play a crucial role in modern biomedical research areas such as signaling pathways, drug discovery or developmental biology. Such experiments yield a very large number of images that encode complex cellular activities, and reliable automated cell tracking emerges naturally as a prerequisite for further quantitative analysis. However, many existing cell tracking methods are restricted to using only a small number of features to allow for manual tweaking. In this paper, we propose a novel cell tracking approach that embraces a powerful machine learning technique to optimize the tracking parameters based on user annotated tracks. Our approach replaces the tedious parameter tuning with parameter learning and allows for the use of a much richer set of complex tracking features, which in turn affords superior prediction accuracy. Furthermore, we developed an active learning approach for efficient training data retrieval, which reduces the annotation effort to only 17%. In practical terms, our approach allows life science researchers to inject their expertise in a more intuitive and direct manner. This process is further facilitated by using a glyph visualization technique for ground truth annotation and validation. Evaluation and comparison on several publicly available benchmark sequences show significant performance improvement over recently reported approaches. Code and software tools are provided to the public.

A benchmark for comparison of cell tracking algorithms

Motivation: Automatic tracking of cells in multidimensional time-lapse fluorescence microscopy is an important task in many biomedical applications. A novel framework for objective evaluation of cell tracking algorithms has been established under the auspices of the IEEE International Symposium on Biomedical Imaging 2013 Cell Tracking Challenge. In this article, we present the logistics, datasets, methods and results of the challenge and lay down the principles for future uses of this benchmark.

Results: The main contributions of the challenge include the creation of a comprehensive video dataset repository and the definition of objective measures for comparison and ranking of the algorithms. With this benchmark, six algorithms covering a variety of segmentation and tracking paradigms have been compared and ranked based on their performance on both synthetic and real datasets. Given the diversity of the datasets, we do not declare a single winner of the challenge. Instead, we present and discuss the results for each individual dataset separately.

Availability and implementation: The challenge Web site (http://www.codesolorzano.com/celltrackingchallenge) provides access to the training and competition datasets, along with the ground truth of the training videos. It also provides access to Windows and Linux executable files of the evaluation software and most of the algorithms that competed in the challenge.

Contact:codesolorzano@unav.es

Supplementary information:Supplementary data are available at Bioinformatics online.

Cell tracking in microscopic video using matching and linking of bipartite graphs

Automated visual tracking of cells from video microscopy has many important biomedical applications. In this paper, we track human monocyte cells in a fluorescent microscopic video using matching and linking of bipartite graphs. Tracking of cells over a pair of frames is modeled as a maximum cardinality minimum weight matching problem for a bipartite graph with a novel cost function. The tracking results are further refined using a rank-based filtering mechanism. Linking of cell trajectories over different frames are achieved through composition of bipartite matches. The proposed solution does not require any explicit motion model, is highly scalable, and, can effectively handle the entry and exit of cells. Our tracking accuracy of (97.97±0.94)% is superior than several existing methods [(95.66±2.39)%, (94.42±2.08)%, (81.22±5.75)%, (78.31±4.70)%] and is highly comparable (98.20±1.22)% to a recently published algorithm.

An analysis-synthesis approach for neurosphere modelisation under phase-contrast microscopy

The study of stem cells is one of the most important biomedical research. Understanding their development could allow multiple applications in regenerative medicine. For this purpose, automated solutions for the observation of stem cell development process are needed. This study introduces an on-line analysis method for the modelling of neurosphere evolution during the early time of their development under phase contrast microscopy. From the corresponding phase contrast time-lapse sequences, we extract information from the neurosphere using a combination of phase contrast physics deconvolution and curve detection for locate the cells inside the neurosphere. Then, based on prior biological knowledge, we generate possible and optimal 3-dimensional configuration using 2D to 3D registration methods and evolutionary optimisation algorithm.

Segmentation and shape tracking of whole fluorescent cells based on the Chan-Vese model

We present a fast and robust approach to tracking the evolving shape of whole fluorescent cells in time-lapse series. The proposed tracking scheme involves two steps. First, coherence-enhancing diffusion filtering is applied on each frame to reduce the amount of noise and enhance flow-like structures. Second, the cell boundaries are detected by minimizing the Chan-Vese model in the fast level set-like and graph cut frameworks. To allow simultaneous tracking of multiple cells over time, both frameworks have been integrated with a topological prior exploiting the object indication function. The potential of the proposed tracking scheme and the advantages and disadvantages of both frameworks are demonstrated on 2-D and 3-D time-lapse series of rat adipose-derived mesenchymal stem cells and human lung squamous cell carcinoma cells, respectively.

Towards comprehensive cell lineage reconstructions in complex organisms using light-sheet microscopy

Understanding the development of complex multicellular organisms as a function of the underlying cell behavior is one of the most fundamental goals of developmental biology. The ability to quantitatively follow cell dynamics in entire developing embryos is an indispensable step towards such a system-level understanding. In recent years, light-sheet fluorescence microscopy has emerged as a particularly promising strategy for recording the in vivo data required to realize this goal. Using light-sheet fluorescence microscopy, entire complex organisms can be rapidly imaged in three dimensions at sub-cellular resolution, achieving high temporal sampling and excellent signal-to-noise ratio without damaging the living specimen or bleaching fluorescent markers. The resulting datasets allow following individual cells in vertebrate and higher invertebrate embryos over up to several days of development. However, the complexity and size of these multi-terabyte recordings typically preclude comprehensive manual analyses. Thus, new computational approaches are required to automatically segment cell morphologies, accurately track cell identities and systematically analyze cell behavior throughout embryonic development. We review current efforts in light-sheet microscopy and bioimage informatics towards this goal, and argue that comprehensive cell lineage reconstructions are finally within reach for many key model organisms, including fruit fly, zebrafish and mouse.

Chapter 17: bioimage informatics for systems pharmacology

Recent advances in automated high-resolution fluorescence microscopy and robotic handling have made the systematic and cost effective study of diverse morphological changes within a large population of cells possible under a variety of perturbations, e.g., drugs, compounds, metal catalysts, RNA interference (RNAi). Cell population-based studies deviate from conventional microscopy studies on a few cells, and could provide stronger statistical power for drawing experimental observations and conclusions. However, it is challenging to manually extract and quantify phenotypic changes from the large amounts of complex image data generated. Thus, bioimage informatics approaches are needed to rapidly and objectively quantify and analyze the image data. This paper provides an overview of the bioimage informatics challenges and approaches in image-based studies for drug and target discovery. The concepts and capabilities of image-based screening are first illustrated by a few practical examples investigating different kinds of phenotypic changes caEditorsused by drugs, compounds, or RNAi. The bioimage analysis approaches, including object detection, segmentation, and tracking, are then described. Subsequently, the quantitative features, phenotype identification, and multidimensional profile analysis for profiling the effects of drugs and targets are summarized. Moreover, a number of publicly available software packages for bioimage informatics are listed for further reference. It is expected that this review will help readers, including those without bioimage informatics expertise, understand the capabilities, approaches, and tools of bioimage informatics and apply them to advance their own studies.

Quantifying stretching and rearrangement in epithelial sheet migration

Although understanding the collective migration of cells, such as that seen in epithelial sheets, is essential for understanding diseases such as metastatic cancer, this motion is not yet as well characterized as individual cell migration. Here we adapt quantitative metrics used to characterize the flow and deformation of soft matter to contrast different types of motion within a migrating sheet of cells. Using a Finite-Time Lyapunov Exponent (FTLE) analysis, we find that - in spite of large fluctuations - the flow field of an epithelial cell sheet is not chaotic. Stretching of a sheet of cells (i.e., positive FTLE) is localized at the leading edge of migration and increases when the cells are more highly stimulated. By decomposing the motion of the cells into affine and non-affine components using the metric D2min , we quantify local plastic rearrangements and describe the motion of a group of cells in a novel way. We find an increase in plastic rearrangements with increasing cell densities, whereas inanimate systems tend to exhibit less non-affine rearrangements with increasing density.

Bioimage informatics for understanding spatiotemporal dynamics of cellular processes

The inner environment of the cell is highly dynamic and heterogeneous yet exquisitely organized. Successful completion of cellular processes within this environment depends on the right molecules or molecular complexes to function at the right place at the right time. Understanding spatiotemporal behaviors of cellular processes is therefore essential to understanding their molecular mechanisms at the systems level. These behaviors are usually visualized and recorded using imaging techniques. However, to infer from them systems-level molecular mechanisms, computational analysis and understanding of recorded image data is crucial, not only for acquiring quantitative behavior measurements but also for comprehending complex interactions among the molecules or molecular complexes involved. The technology of computational analysis and understanding of biological images is often referred to simply as bioimage informatics. This article introduces fundamentals of bioimage informatics for understanding spatiotemporal dynamics of cellular processes and reviews recent advances on this topic. Basic bioimage informatics concepts and techniques for characterizing spatiotemporal cell dynamics are introduced first. Studies on specific cellular processes such as cell migration and signal transduction are then used as examples to analyze and summarize recent advances, with the focus on transforming quantitative measurements of spatiotemporal cellular behaviors into knowledge of underlying molecular mechanisms. Despite the advances made, substantial technological challenges remain, especially in representation of spatiotemporal cellular behaviors and inference of systems-level molecular mechanisms. These challenges are briefly discussed. Overall, understanding spatiotemporal cell dynamics will provide critical insights into how specific cellular processes as well as the entire inner cellular environment are dynamically organized and regulated.

Improving active contour methods for tracking endothelial cells by the removal of low-confidence edge segments

Previously we developed an active contour method for segmenting and tracking cells in phase-contrast microscopy images. Our method is capable of fine-grained segmentation on noisy image sequences. In this paper, we improve the active contour segmentation model to provide better accuracy, by selectively identifying areas of the contour with low confidence and removing them. The method is applied to HMEC-1 cells (human microvascular endothelial cells). The segmentation provided by the method is quantitavely compared with manually-drawn contours, showing close fit and capability to 'lock' on to cell boundaries for hundreds of frames.

Multi-object geodesic active contours (MOGAC)

An emerging topic is to build image segmentation systems that can segment hundreds to thousands of objects (i.e. cell segmentation\tracking, full brain parcellation, full body segmentation, etc.). Multi-object Level Set Methods (MLSM) perform this task with the benefit of sub-pixel precision. However, current implementations of MLSM are not as computationally or memory efficient as their region growing and graph cut counterparts which lack sub-pixel precision. To address this performance gap, we present a novel parallel implementation of MLSM that leverages the sparse properties of the algorithm to minimize its memory footprint for multiple objects. The new method, Multi-Object Geodesic Active Contours (MOGAC), can represent N objects with just two functions: a label mask image and unsigned distance field. The time complexity of the algorithm is shown to be O((M (power)d)/P) for M (power)d pixels and P processing units in dimension d = {2,3}, independent of the number of objects. Results are presented for 2D and 3D image segmentation problems.

Application of the IMM-JPDA filter to multiple target tracking in total internal reflection fluorescence microscopy images

We propose a multi-target tracking method using an Interacting Multiple Model Joint Probabilistic Data Association (IMM-JPDA) filter for tracking vesicles in total internal reflection fluorescence microscopy (TIRFM) sequences. We enhance the accuracy and reliability of the algorithm by tailoring an appropriate framework to this application. Evaluation of our algorithm is performed on both realistic synthetic data and real TIRFM data. Our results are compared against related methods and a commercial tracking software.

Live imaging, identifying, and tracking single cells in complex populations in vivo and ex vivo

Advances in optical imaging technologies combined with the use of genetically encoded fluorescent proteins have enabled the visualization of stem cells over extensive periods of time in vivo and ex vivo. The generation of genetically encoded fluorescent protein reporters that are fused with subcellularly localized proteins, such as human histone H2B, has made it possible to direct fluorescent protein reporters to specific subcellular structures and identify single cells in complex populations. This facilitates the visualization of cellular behaviors such as division, movement, and apoptosis at a single-cell resolution and, in principle, allows the prospective and retrospective tracking towards determining the lineage of each cell.

Label-free detection and dynamic monitoring of drug-induced intracellular vesicle formation enabled using a 2-dimensional matched filter

Analysis of vesicle formation and degradation is a central issue in autophagy research and microscopy imaging is revolutionizing the study of such dynamic events inside living cells. A limiting factor is the need for labeling techniques that are labor intensive, expensive, and not always completely reliable. To enable label-free analyses we introduced a generic computational algorithm, the label-free vesicle detector (LFVD), which relies on a matched filter designed to identify circular vesicles within cells using only phase-contrast microscopy images. First, the usefulness of the LFVD is illustrated by presenting successful detections of autophagy modulating drugs found by analyzing the human colorectal carcinoma cell line HCT116 exposed to each substance among 1266 pharmacologically active compounds. Some top hits were characterized with respect to their activity as autophagy modulators using independent in vitro labeling of acidic organelles, detection of LC3-II protein, and analysis of the autophagic flux. Selected detection results for 2 additional cell lines (DLD1 and RKO) demonstrate the generality of the method. In a second experiment, label-free monitoring of dose-dependent vesicle formation kinetics is demonstrated by recorded detection of vesicles over time at different drug concentrations. In conclusion, label-free detection and dynamic monitoring of vesicle formation during autophagy is enabled using the LFVD approach introduced.

Bioprinting for stem cell research

Recently, there has been growing interest in applying bioprinting techniques to stem cell research. Several bioprinting methods have been developed utilizing acoustics, piezoelectricity, and lasers to deposit living cells onto receiving substrates. Using these technologies, spatially defined gradients of immobilized biomolecules can be engineered to direct stem cell differentiation into multiple subpopulations of different lineages. Stem cells can also be patterned in a high-throughput manner onto flexible implementation patches for tissue regeneration or onto substrates with the goal of accessing encapsulated stem cells of interest for genomic analysis. Here, we review recent achievements with bioprinting technologies in stem cell research, and identify future challenges and potential applications including tissue engineering and regenerative medicine, wound healing, and genomics.

Assessment of automated analyses of cell migration on flat and nanostructured surfaces

Motility studies of cells often rely on computer software that analyzes time-lapse recorded movies and establishes cell trajectories fully automatically. This raises the question of reproducibility of results, since different programs could yield significantly different results of such automated analysis. The fact that the segmentation routines of such programs are often challenged by nanostructured surfaces makes the question more pertinent. Here we illustrate how it is possible to track cells on bright field microscopy images with image analysis routines implemented in an open-source cell tracking program, PACT (Program for Automated Cell Tracking). We compare the automated motility analysis of three cell tracking programs, PACT, Autozell, and TLA, using the same movies as input for all three programs. We find that different programs track overlapping, but different subsets of cells due to different segmentation methods. Unfortunately, population averages based on such different cell populations, differ significantly in some cases. Thus, results obtained with one software package are not necessarily reproducible by other software.

Identifying virus-cell fusion in two-channel fluorescence microscopy image sequences based on a layered probabilistic approach

The entry process of virus particles into cells is decisive for infection. In this work, we investigate fusion of virus particles with the cell membrane via time-lapse fluorescence microscopy. To automatically identify fusion for single particles based on their intensity over time, we have developed a layered probabilistic approach. The approach decomposes the action of a single particle into three abstractions: the intensity over time, the underlying temporal intensity model, as well as a high level behavior. Each abstraction corresponds to a layer and these layers are represented via stochastic hybrid systems and hidden Markov models. We use a maxbelief strategy to efficiently combine both representations. To compute estimates for the abstractions we use a hybrid particle filter and the Viterbi algorithm. Based on synthetic image sequences, we characterize the performance of the approach as a function of the image noise. We also characterize the performance as a function of the tracking error. We have also successfully applied the approach to real image sequences displaying pseudotyped HIV-1 particles in contact with host cells and compared the experimental results with ground truth obtained by manual analysis.

Morphology-based features for adaptive mitosis detection of in vitro stem cell tracking data

OBJECTIVES

The cultivation of adherently growing cell populations is a major task in the field of adult stem cell production used for drug discovery and in the field of regenerative medicine. To assessthe quality of a cell population, a crucial event is the mitotic cell division: the precise knowledge of these events enables the reconstruction of lineages and accurate proliferation curves as well as a detailed analysis of cell cycles. To serve in an autonomous cell farming framework, such a detector requires to work reliably and unsupervised.

METHODS

We introduce a mitosis detector that is using a maximum likelihood (ML) estimator based on morphological cell features (cell area, brightness, length, compactness). It adapts to the 3 phases of cell growth (lag, log and stationary phase). As a concurrent model, we compared ML with kernel SVMs using linear, quadratic and Gaussian kernel functions. All approaches are evaluated for their ability to distinguish between mitotic and non-mitotic events. The large, publicly available benchmark data CeTReS (reference data set A with >240,000 segmented cells, >2,000 mitotic events) is used for this evaluation.

RESULTS

The adaptive (unsupervised) ML approach clearly outperforms previously published non-adaptive approaches and the linear SVM. Furthermore, it robustly reaches a performance comparable to quadratic and Gaussian SVM.

CONCLUSIONS

The proposed simple and label free adaptive variant might be the method of choice when it comes to autonomous cell farming. Hereby, it is essential to have reliable and unsupervised mitosis detection that covers all phases of cell growth.