Automated and semi-automated cell tracking: addressing portability challenges

Adaptive tracking algorithm for trajectory analysis of cells and layer-by-layer assessment of motility dynamics

Tracking biological objects such as cells or subcellular components imaged with time-lapse microscopy enables us to understand the molecular principles about the dynamics of cell behaviors. However, automatic object detection, segmentation and extracting trajectories remain as a rate-limiting step due to intrinsic challenges of video processing. This paper presents an adaptive tracking algorithm (Adtari) that automatically finds the optimum search radius and cell linkages to determine trajectories in consecutive frames. A critical assumption in most tracking studies is that displacement remains unchanged throughout the movie and cells in a few frames are usually analyzed to determine its magnitude. Tracking errors and inaccurate association of cells may occur if the user does not correctly evaluate the value or prior knowledge is not present on cell movement. The key novelty of our method is that minimum intercellular distance and maximum displacement of cells between frames are dynamically computed and used to determine the threshold distance. Since the space between cells is highly variable in a given frame, our software recursively alters the magnitude to determine all plausible matches in the trajectory analysis. Our method therefore eliminates a major preprocessing step where a constant distance was used to determine the neighbor cells in tracking methods. Cells having multiple overlaps and splitting events were further evaluated by using the shape attributes including perimeter, area, ellipticity and distance. The features were applied to determine the closest matches by minimizing the difference in their magnitudes. Finally, reporting section of our software were used to generate instant maps by overlaying cell features and trajectories. Adtari was validated by using videos with variable signal-to-noise, contrast ratio and cell density. We compared the adaptive tracking with constant distance and other methods to evaluate performance and its efficiency. Our algorithm yields reduced mismatch ratio, increased ratio of whole cell track, higher frame tracking efficiency and allows layer-by-layer assessment of motility to characterize single-cells. Adaptive tracking provides a reliable, accurate, time efficient and user-friendly open source software that is well suited for analysis of 2D fluorescence microscopy video datasets.

No Cell Left behind: Automated, Stochastic, Physics-Based Tracking of Every Cell in a Dense, Growing Colony

Motivation: Precise tracking of individual cells—especially tracking the family lineage, for example in a developing embryo—has widespread applications in biology and medicine. Due to significant noise in microscope images, existing methods have difficulty precisely tracking cell activities. These difficulties often require human intervention to resolve. Humans are helpful because our brain naturally and automatically builds a simulation “model” of any scene that we observe. Because we understand simple truths about the world—for example cells can move and divide, but they cannot instantaneously move vast distances—this model “in our heads” helps us to severely constrain the possible interpretations of what we see, allowing us to easily distinguish signal from noise, and track the motion of cells even in the presence of extreme levels of noise that would completely confound existing automated methods. Results: Here, we mimic the ability of the human brain by building an explicit computer simulation model of the scene. Our simulated cells are programmed to allow movement and cell division consistent with reality. At each video frame, we stochastically generate millions of nearby “Universes” and evolve them stochastically to the next frame. We then find and fit the best universes to reality by minimizing the residual between the real image frame and a synthetic image of the simulation. The rule-based simulation puts extremely stringent constraints on possible interpretations of the data, allowing our system to perform far better than existing methods even in the presense of extreme levels of image noise. We demonstrate the viability of this method by accurately tracking every cell in a colony that grows from 4 to over 300 individuals, doing about as well as a human can in the difficult task of tracking cell lineages.

Fully-automatic deep learning-based analysis for determination of the invasiveness of breast cancer cells in an acoustic trap

A single-beam acoustic trapping technique has been shown to be very useful for determining the invasiveness of suspended breast cancer cells in an acoustic trap with a manual calcium analysis method. However, for the rapid translation of the technology into the clinic, the development of an efficient/accurate analytical method is needed. We, therefore, develop a fully-automatic deep learning-based calcium image analysis algorithm for determining the invasiveness of suspended breast cancer cells using a single-beam acoustic trapping system. The algorithm allows to segment cells, find trapped cells, and quantify their calcium changes over time. For better segmentation of calcium fluorescent cells even with vague boundaries, a novel deep learning architecture with multi-scale/multi-channel convolution operations (MM-Net) is devised and constructed by a target inversion training method. The MM-Net outperforms other deep learning models in the cell segmentation. Also, a detection/quantification algorithm is developed and implemented to automatically determine the invasiveness of a trapped cell. For the evaluation of the algorithm, it is applied to quantify the invasiveness of breast cancer cells. The results show that the algorithm offers similar performance to the manual calcium analysis method for determining the invasiveness of cancer cells, suggesting that it may serve as a novel tool to automatically determine the invasiveness of cancer cells with high-efficiency.

A probabilistic approach to joint cell tracking and segmentation in high-throughput microscopy videos

We present a novel computational framework for the analysis of high-throughput microscopy videos of living cells. The proposed framework is generally useful and can be applied to different datasets acquired in a variety of laboratory settings. This is accomplished by tying together two fundamental aspects of cell lineage construction, namely cell segmentation and tracking, via a Bayesian inference of dynamic models. In contrast to most existing approaches, which aim to be general, no assumption of cell shape is made. Spatial, temporal, and cross-sectional variation of the analysed data are accommodated by two key contributions. First, time series analysis is exploited to estimate the temporal cell shape uncertainty in addition to cell trajectory. Second, a fast marching (FM) algorithm is used to integrate the inferred cell properties with the observed image measurements in order to obtain image likelihood for cell segmentation, and association. The proposed approach has been tested on eight different time-lapse microscopy data sets, some of which are high-throughput, demonstrating promising results for the detection, segmentation and association of planar cells. Our results surpass the state of the art for the Fluo-C2DL-MSC data set of the Cell Tracking Challenge (Maška et al., 2014).

Microrheology, advances in methods and insights

Microrheology is an emerging technique that probes mechanical response of soft material at micro-scale. Generally, microrheology technique can be divided into active and passive versions. During last two decades, extensive efforts have been paid to improve both the experiment techniques and data analysis methods, especially about how to link consequential particle positions into trajectories. We review the recent advances in microrheology, including improvements in labeling, imaging, data acquiring, data processing and data interpretation. Some of the recent insights in soft matter and living systems gained by using this technique are given. Before these, we also give a very brief description of the basic principles of both active and passive microrheology techniques, and some details about optical particle tracking and DWS.

Quantitative profiling of innate immune activation by viral infection in single cells

Cells infected by viruses can exhibit diverse patterns of viral and cellular gene expression. The patterns arise in part from the stochastic or noisy reaction kinetics associated with the small number of genomes, enzymes, and other molecules that typically initiate virus replication and activate cellular anti-viral defenses. It is not known what features, if any, of the early viral or cellular gene expression correlate with later processes of viral replication or cell survival. Here we used two fluorescent reporters to visualize innate immune activation of human prostate cancer (PC3) cells against infection by vesicular stomatitis virus. The cells were engineered to express green-fluorescent protein under control of the promoter for IFIT2, an interferon-sensitive component of the anti-viral response, while red-fluorescent protein was expressed as a byproduct of virus infection. To isolate and quantitatively analyze single-cells, we used a unique microwell array device and open-source image processing software. Kinetic analysis of viral and cellular reporter profiles from hundreds of cells revealed novel relationships between gene expression and the outcome of infection. Specifically, the relative timing rather than the magnitude of the viral gene expression and innate immune activation correlated with the infection outcome. Earlier viral or anti-viral gene expression favored or hindered virus growth, respectively. Further, analysis of kinetic parameters estimated from these data suggests a trade-off between robust antiviral signaling and cell death, as indicated by a higher rate of detectable cell lysis in infected cells with a detectable immune response. In short, cells that activate an immune response lyse at a higher rate. More broadly, we demonstrate how the intrinsic heterogeneity of individual cell behaviors can be exploited to discover features of viral and host gene expression that correlate with single-cell outcomes, which will ultimately impact whether or not infections spread.

Machine learning applications in cell image analysis

Machine learning (ML) refers to a set of automatic pattern recognition methods that have been successfully applied across various problem domains, including biomedical image analysis. This review focuses on ML applications for image analysis in light microscopy experiments with typical tasks of segmenting and tracking individual cells, and modelling of reconstructed lineage trees. After describing a typical image analysis pipeline and highlighting challenges of automatic analysis (for example, variability in cell morphology, tracking in presence of clutters) this review gives a brief historical outlook of ML, followed by basic concepts and definitions required for understanding examples. This article then presents several example applications at various image processing stages, including the use of supervised learning methods for improving cell segmentation, and the application of active learning for tracking. The review concludes with remarks on parameter setting and future directions.

Comparative evaluation of performance measures for shading correction in time-lapse fluorescence microscopy

Time-lapse fluorescence microscopy is a valuable technology in cell biology, but it suffers from the inherent problem of intensity inhomogeneity due to uneven illumination or camera nonlinearity, known as shading artefacts. This will lead to inaccurate estimates of single-cell features such as average and total intensity. Numerous shading correction methods have been proposed to remove this effect. In order to compare the performance of different methods, many quantitative performance measures have been developed. However, there is little discussion about which performance measure should be generally applied for evaluation on real data, where the ground truth is absent. In this paper, the state-of-the-art shading correction methods and performance evaluation methods are reviewed. We implement 10 popular shading correction methods on two artificial datasets and four real ones. In order to make an objective comparison between those methods, we employ a number of quantitative performance measures. Extensive validation demonstrates that the coefficient of joint variation (CJV) is the most applicable measure in time-lapse fluorescence images. Based on this measure, we have proposed a novel shading correction method that performs better compared to well-established methods for a range of real data tested.

Tools for Single-Cell Kinetic Analysis of Virus-Host Interactions

Measures of cellular gene expression or behavior, when performed on individual cells, inevitably reveal a diversity of behaviors and outcomes that can correlate with normal or diseased states. For virus infections, the potential diversity of outcomes are pushed to an extreme, where measures of infection reflect features of the specific infecting virus particle, the individual host cell, as well as interactions between viral and cellular components. Single-cell measures, while revealing, still often rely on specialized fluid handling capabilities, employ end-point measures, and remain labor-intensive to perform. To address these limitations, we consider a new microwell-based device that uses simple pipette-based fluid handling to isolate individual cells. Our design allows different experimental conditions to be implemented in a single device, permitting easier and more standardized protocols. Further, we utilize a recently reported dual-color fluorescent reporter system that provides dynamic readouts of viral and cellular gene expression during single-cell infections by vesicular stomatitis virus. In addition, we develop and show how free, open-source software can enable streamlined data management and batch image analysis. Here we validate the integration of the device and software using the reporter system to demonstrate unique single-cell dynamic measures of cellular responses to viral infection.

Cell Tracking Accuracy Measurement Based on Comparison of Acyclic Oriented Graphs

Tracking motile cells in time-lapse series is challenging and is required in many biomedical applications. Cell tracks can be mathematically represented as acyclic oriented graphs. Their vertices describe the spatio-temporal locations of individual cells, whereas the edges represent temporal relationships between them. Such a representation maintains the knowledge of all important cellular events within a captured field of view, such as migration, division, death, and transit through the field of view. The increasing number of cell tracking algorithms calls for comparison of their performance. However, the lack of a standardized cell tracking accuracy measure makes the comparison impracticable. This paper defines and evaluates an accuracy measure for objective and systematic benchmarking of cell tracking algorithms. The measure assumes the existence of a ground-truth reference, and assesses how difficult it is to transform a computed graph into the reference one. The difficulty is measured as a weighted sum of the lowest number of graph operations, such as split, delete, and add a vertex and delete, add, and alter the semantics of an edge, needed to make the graphs identical. The measure behavior is extensively analyzed based on the tracking results provided by the participants of the first Cell Tracking Challenge hosted by the 2013 IEEE International Symposium on Biomedical Imaging. We demonstrate the robustness and stability of the measure against small changes in the choice of weights for diverse cell tracking algorithms and fluorescence microscopy datasets. As the measure penalizes all possible errors in the tracking results and is easy to compute, it may especially help developers and analysts to tune their algorithms according to their needs.

Associating approximate paths and temporal sequences of noisy detections: Application to the recovery of spatio-temporal cancer cell trajectories

In this paper we address the problem of recovering spatio-temporal trajectories of cancer cells in phase contrast video-microscopy where the user provides the paths on which the cells are moving. The paths are purely spatial, without temporal information. To recover the temporal information associated to a given path we propose an approach based on automatic cell detection and on a graph-based shortest path search. The nodes in the graph consist of the projections of the cell detections onto the geometrical cell path. The edges relate nodes which correspond to different frames of the sequence and potentially to the same cell and trajectory. In this directed graph we search for the shortest path and use it to define a temporal parametrization of the corresponding geometrical cell path. An evaluation based on 286 paths of 7 phase contrast microscopy videos shows that our algorithm allows to recover 92% of trajectory points with respect to the associated ground truth. We compare our method with a state-of-the-art algorithm for semi-automated cell tracking in phase contrast microscopy which requires interactively placed starting points for the cells to track. The comparison shows that supporting geometrical paths in combination with our algorithm allow us to obtain more reliable cell trajectories.

A benchmark for comparison of cell tracking algorithms

Motivation: Automatic tracking of cells in multidimensional time-lapse fluorescence microscopy is an important task in many biomedical applications. A novel framework for objective evaluation of cell tracking algorithms has been established under the auspices of the IEEE International Symposium on Biomedical Imaging 2013 Cell Tracking Challenge. In this article, we present the logistics, datasets, methods and results of the challenge and lay down the principles for future uses of this benchmark.

Results: The main contributions of the challenge include the creation of a comprehensive video dataset repository and the definition of objective measures for comparison and ranking of the algorithms. With this benchmark, six algorithms covering a variety of segmentation and tracking paradigms have been compared and ranked based on their performance on both synthetic and real datasets. Given the diversity of the datasets, we do not declare a single winner of the challenge. Instead, we present and discuss the results for each individual dataset separately.

Availability and implementation: The challenge Web site (http://www.codesolorzano.com/celltrackingchallenge) provides access to the training and competition datasets, along with the ground truth of the training videos. It also provides access to Windows and Linux executable files of the evaluation software and most of the algorithms that competed in the challenge.

Contact:codesolorzano@unav.es

Supplementary information:Supplementary data are available at Bioinformatics online.

Labour-efficient in vitro lymphocyte population tracking and fate prediction using automation and manual review

Interest in cell heterogeneity and differentiation has recently led to increased use of time-lapse microscopy. Previous studies have shown that cell fate may be determined well in advance of the event. We used a mixture of automation and manual review of time-lapse live cell imaging to track the positions, contours, divisions, deaths and lineage of 44 B-lymphocyte founders and their 631 progeny in vitro over a period of 108 hours. Using this data to train a Support Vector Machine classifier, we were retrospectively able to predict the fates of individual lymphocytes with more than 90% accuracy, using only time-lapse imaging captured prior to mitosis or death of 90% of all cells. The motivation for this paper is to explore the impact of labour-efficient assistive software tools that allow larger and more ambitious live-cell time-lapse microscopy studies. After training on this data, we show that machine learning methods can be used for realtime prediction of individual cell fates. These techniques could lead to realtime cell culture segregation for purposes such as phenotype screening. We were able to produce a large volume of data with less effort than previously reported, due to the image processing, computer vision, tracking and human-computer interaction tools used. We describe the workflow of the software-assisted experiments and the graphical interfaces that were needed. To validate our results we used our methods to reproduce a variety of published data about lymphocyte populations and behaviour. We also make all our data publicly available, including a large quantity of lymphocyte spatio-temporal dynamics and related lineage information.

Cell tracking in microscopic video using matching and linking of bipartite graphs

Automated visual tracking of cells from video microscopy has many important biomedical applications. In this paper, we track human monocyte cells in a fluorescent microscopic video using matching and linking of bipartite graphs. Tracking of cells over a pair of frames is modeled as a maximum cardinality minimum weight matching problem for a bipartite graph with a novel cost function. The tracking results are further refined using a rank-based filtering mechanism. Linking of cell trajectories over different frames are achieved through composition of bipartite matches. The proposed solution does not require any explicit motion model, is highly scalable, and, can effectively handle the entry and exit of cells. Our tracking accuracy of (97.97±0.94)% is superior than several existing methods [(95.66±2.39)%, (94.42±2.08)%, (81.22±5.75)%, (78.31±4.70)%] and is highly comparable (98.20±1.22)% to a recently published algorithm.

Automated cell tracking and analysis in phase-contrast videos (iTrack4U): development of Java software based on combined mean-shift processes

Cell migration is a key biological process with a role in both physiological and pathological conditions. Locomotion of cells during embryonic development is essential for their correct positioning in the organism; immune cells have to migrate and circulate in response to injury. Failure of cells to migrate or an inappropriate acquisition of migratory capacities can result in severe defects such as altered pigmentation, skull and limb abnormalities during development, and defective wound repair, immunosuppression or tumor dissemination. The ability to accurately analyze and quantify cell migration is important for our understanding of development, homeostasis and disease. In vitro cell tracking experiments, using primary or established cell cultures, are often used to study migration as cells can quickly and easily be genetically or chemically manipulated. Images of the cells are acquired at regular time intervals over several hours using microscopes equipped with CCD camera. The locations (x,y,t) of each cell on the recorded sequence of frames then need to be tracked. Manual computer-assisted tracking is the traditional method for analyzing the migratory behavior of cells. However, this processing is extremely tedious and time-consuming. Most existing tracking algorithms require experience in programming languages that are unfamiliar to most biologists. We therefore developed an automated cell tracking program, written in Java, which uses a mean-shift algorithm and ImageJ as a library. iTrack4U is a user-friendly software. Compared to manual tracking, it saves considerable amount of time to generate and analyze the variables characterizing cell migration, since they are automatically computed with iTrack4U. Another major interest of iTrack4U is the standardization and the lack of inter-experimenter differences. Finally, iTrack4U is adapted for phase contrast and fluorescent cells.