Advanced level-set-based cell tracking in time-lapse fluorescence microscopy

A Hybrid Approach for Segmentation and Tracking of Myxococcus Xanthus Swarms

Cell segmentation and motion tracking in time-lapse images are fundamental problems in computer vision, and are also crucial for various biomedical studies. Myxococcus xanthus is a type of rod-like cells with highly coordinated motion. The segmentation and tracking of M. xanthus are challenging, because cells may touch tightly and form dense swarms that are difficult to identify individually in an accurate manner. The known cell tracking approaches mainly fall into two frameworks, detection association and model evolution, each having its own advantages and disadvantages. In this paper, we propose a new hybrid framework combining these two frameworks into one and leveraging their complementary advantages. Also, we propose an active contour model based on the Ribbon Snake, which is seamlessly integrated with our hybrid framework. Evaluated by 10 different datasets, our approach achieves considerable improvement over the state-of-the-art cell tracking algorithms on identifying complete cell trajectories, and higher segmentation accuracy than performing segmentation in individual 2D images.

Robust Individual-Cell/Object Tracking via PCANet Deep Network in Biomedicine and Computer Vision

Tracking individual-cell/object over time is important in understanding drug treatment effects on cancer cells and video surveillance. A fundamental problem of individual-cell/object tracking is to simultaneously address the cell/object appearance variations caused by intrinsic and extrinsic factors. In this paper, inspired by the architecture of deep learning, we propose a robust feature learning method for constructing discriminative appearance models without large-scale pretraining. Specifically, in the initial frames, an unsupervised method is firstly used to learn the abstract feature of a target by exploiting both classic principal component analysis (PCA) algorithms with recent deep learning representation architectures. We use learned PCA eigenvectors as filters and develop a novel algorithm to represent a target by composing of a PCA-based filter bank layer, a nonlinear layer, and a patch-based pooling layer, respectively. Then, based on the feature representation, a neural network with one hidden layer is trained in a supervised mode to construct a discriminative appearance model. Finally, to alleviate the tracker drifting problem, a sample update scheme is carefully designed to keep track of the most representative and diverse samples during tracking. We test the proposed tracking method on two standard individual cell/object tracking benchmarks to show our tracker's state-of-the-art performance.

Joint level-set and spatio-temporal motion detection for cell segmentation

BACKGROUND

Cell segmentation is a critical step for quantification and monitoring of cell cycle progression, cell migration, and growth control to investigate cellular immune response, embryonic development, tumorigenesis, and drug effects on live cells in time-lapse microscopy images.

METHODS

In this study, we propose a joint spatio-temporal diffusion and region-based level-set optimization approach for moving cell segmentation. Moving regions are initially detected in each set of three consecutive sequence images by numerically solving a system of coupled spatio-temporal partial differential equations. In order to standardize intensities of each frame, we apply a histogram transformation approach to match the pixel intensities of each processed frame with an intensity distribution model learned from all frames of the sequence during the training stage. After the spatio-temporal diffusion stage is completed, we compute the edge map by nonparametric density estimation using Parzen kernels. This process is followed by watershed-based segmentation and moving cell detection. We use this result as an initial level-set function to evolve the cell boundaries, refine the delineation, and optimize the final segmentation result.

RESULTS

We applied this method to several datasets of fluorescence microscopy images with varying levels of difficulty with respect to cell density, resolution, contrast, and signal-to-noise ratio. We compared the results with those produced by Chan and Vese segmentation, a temporally linked level-set technique, and nonlinear diffusion-based segmentation. We validated all segmentation techniques against reference masks provided by the international Cell Tracking Challenge consortium. The proposed approach delineated cells with an average Dice similarity coefficient of 89 % over a variety of simulated and real fluorescent image sequences. It yielded average improvements of 11 % in segmentation accuracy compared to both strictly spatial and temporally linked Chan-Vese techniques, and 4 % compared to the nonlinear spatio-temporal diffusion method.

CONCLUSIONS

Despite the wide variation in cell shape, density, mitotic events, and image quality among the datasets, our proposed method produced promising segmentation results. These results indicate the efficiency and robustness of this method especially for mitotic events and low SNR imaging, enabling the application of subsequent quantification tasks.

Size-Invariant Detection of Cell Nuclei in Microscopy Images

Accurate detection of individual cell nuclei in microscopy images is an essential and fundamental task for many biological studies. In particular, multivariate fluorescence microscopy is used to observe different aspects of cells in cultures. Manual detection of individual cell nuclei by visual inspection is time consuming, and prone to induce subjective bias. This makes automatic detection of cell nuclei essential for large-scale, objective studies of cell cultures. Blur, clutter, bleed-through, imaging noise and touching and partially overlapping nuclei with varying sizes and shapes make automated detection of individual cell nuclei a challenging task using image analysis. In this paper we propose a new automated method for fast and robust detection of individual cell nuclei based on their radial symmetric nature in fluorescence in-situ hybridization (FISH) images obtained via confocal microscopy. The main contributions are two-fold. 1) This work presents a more accurate cell nucleus detection system using the fast radial symmetry transform (FRST). 2) The proposed cell nucleus detection system is robust against most occlusions and variations in size and moderate shape deformations. We evaluate the performance of the proposed algorithm using precision/recall rates, Fβ-score and root-mean-squared distance (RMSD) and show that our algorithm provides improved detection accuracy compared to existing algorithms.

Deformable Graph Model for Tracking Epithelial Cell Sheets in Fluorescence Microscopy

We propose a novel method for tracking cells that are connected through a visible network of membrane junctions. Tissues of this form are common in epithelial cell sheets and resemble planar graphs where each face corresponds to a cell. We leverage this structure and develop a method to track the entire tissue as a deformable graph. This coupled model in which vertices inform the optimal placement of edges and vice versa captures global relationships between tissue components and leads to accurate and robust cell tracking. We compare the performance of our method with that of four reference tracking algorithms on four data sets that present unique tracking challenges. Our method exhibits consistently superior performance in tracking all cells accurately over all image frames, and is robust over a wide range of image intensity and cell shape profiles. This may be an important tool for characterizing tissues of this type especially in the field of developmental biology where automated cell analysis can help elucidate the mechanisms behind controlled cell-shape changes.

Curvelet initialized level set cell segmentation for touching cells in low contrast images

Cell segmentation is an important element of automatic cell analysis. This paper proposes a method to extract the cell nuclei and the cell boundaries of touching cells in low contrast images. First, the contrast of the low contrast cell images is improved by a combination of multiscale top hat filter and h-maxima. Then, a curvelet initialized level set method has been proposed to detect the cell nuclei and the boundaries. The image enhancement results have been verified using PSNR (Peak Signal to noise ratio) and the segmentation results have been verified using accuracy, sensitivity and precision metrics. The results show improved values of the performance metrics with the proposed method.

An Automatic Learning-Based Framework for Robust Nucleus Segmentation

Computer-aided image analysis of histopathology specimens could potentially provide support for early detection and improved characterization of diseases such as brain tumor, pancreatic neuroendocrine tumor (NET), and breast cancer. Automated nucleus segmentation is a prerequisite for various quantitative analyses including automatic morphological feature computation. However, it remains to be a challenging problem due to the complex nature of histopathology images. In this paper, we propose a learning-based framework for robust and automatic nucleus segmentation with shape preservation. Given a nucleus image, it begins with a deep convolutional neural network (CNN) model to generate a probability map, on which an iterative region merging approach is performed for shape initializations. Next, a novel segmentation algorithm is exploited to separate individual nuclei combining a robust selection-based sparse shape model and a local repulsive deformable model. One of the significant benefits of the proposed framework is that it is applicable to different staining histopathology images. Due to the feature learning characteristic of the deep CNN and the high level shape prior modeling, the proposed method is general enough to perform well across multiple scenarios. We have tested the proposed algorithm on three large-scale pathology image datasets using a range of different tissue and stain preparations, and the comparative experiments with recent state of the arts demonstrate the superior performance of the proposed approach.

Inferring intra-cellular mechanics using geometric metamorphosis: A preliminary study

Mechanotransduction plays an important role in sub-cellular processes and is an active area of research. Determining the forces/strains that the intra-cellular structures experience is vital for developing quantitative models of cellular behavior. Established techniques such as traction force microscopy, digital image correlation etc. track surface forces and kinematics of intra-cellular structures. However, difficulties arise when cells cannot be seeded on micro-patterned substrates or the intra-cellular structures vary (unstable landmarks). Here, we applied geometric metamorphosis, a global image registration method, to determine the kinematic profile of a cell during cell division. The method does not require stable landmarks, the registration is non-local in nature and constraints such as volume conservation can be enforced. The cell wall was tracked over time and a sequence of transformations relating the cell wall at the start of cytokinesis to the configuration prior to the daughters completely separate was determined. These transformations are associated with a scalar metric and a statistical atlas describing the wall kinematics from multiple tracking's of the wall shape is constructed. Using these transformations, the cellular kinematics can be described using a Lagrangian frame of reference and the evolution of a material point property can be easily modeled. To demonstrate this, we use the kinematic data derived from the atlas along with a model of stress-fiber (de)formation dynamics to simulate the stress-fiber configuration as the cell domain deforms.

Robust Nucleus/Cell Detection and Segmentation in Digital Pathology and Microscopy Images: A Comprehensive Review

Digital pathology and microscopy image analysis is widely used for comprehensive studies of cell morphology or tissue structure. Manual assessment is labor intensive and prone to interobserver variations. Computer-aided methods, which can significantly improve the objectivity and reproducibility, have attracted a great deal of interest in recent literature. Among the pipeline of building a computer-aided diagnosis system, nucleus or cell detection and segmentation play a very important role to describe the molecular morphological information. In the past few decades, many efforts have been devoted to automated nucleus/cell detection and segmentation. In this review, we provide a comprehensive summary of the recent state-of-the-art nucleus/cell segmentation approaches on different types of microscopy images including bright-field, phase-contrast, differential interference contrast, fluorescence, and electron microscopies. In addition, we discuss the challenges for the current methods and the potential future work of nucleus/cell detection and segmentation.

An Automatic Segmentation Method Combining an Active Contour Model and a Classification Technique for Detecting Polycomb-group Proteinsin High-Throughput Microscopy Images

The large amount of data generated in biological experiments that rely on advanced microscopy can be handled only with automated image analysis. Most analyses require a reliable cell image segmentation eventually capable of detecting subcellular structures.We present an automatic segmentation method to detect Polycomb group (PcG) proteins areas isolated from nuclei regions in high-resolution fluorescent cell image stacks. It combines two segmentation algorithms that use an active contour model and a classification technique serving as a tool to better understand the subcellular three-dimensional distribution of PcG proteins in live cell image sequences. We obtained accurate results throughout several cell image datasets, coming from different cell types and corresponding to different fluorescent labels, without requiring elaborate adjustments to each dataset.

Cell Tracking Accuracy Measurement Based on Comparison of Acyclic Oriented Graphs

Tracking motile cells in time-lapse series is challenging and is required in many biomedical applications. Cell tracks can be mathematically represented as acyclic oriented graphs. Their vertices describe the spatio-temporal locations of individual cells, whereas the edges represent temporal relationships between them. Such a representation maintains the knowledge of all important cellular events within a captured field of view, such as migration, division, death, and transit through the field of view. The increasing number of cell tracking algorithms calls for comparison of their performance. However, the lack of a standardized cell tracking accuracy measure makes the comparison impracticable. This paper defines and evaluates an accuracy measure for objective and systematic benchmarking of cell tracking algorithms. The measure assumes the existence of a ground-truth reference, and assesses how difficult it is to transform a computed graph into the reference one. The difficulty is measured as a weighted sum of the lowest number of graph operations, such as split, delete, and add a vertex and delete, add, and alter the semantics of an edge, needed to make the graphs identical. The measure behavior is extensively analyzed based on the tracking results provided by the participants of the first Cell Tracking Challenge hosted by the 2013 IEEE International Symposium on Biomedical Imaging. We demonstrate the robustness and stability of the measure against small changes in the choice of weights for diverse cell tracking algorithms and fluorescence microscopy datasets. As the measure penalizes all possible errors in the tracking results and is easy to compute, it may especially help developers and analysts to tune their algorithms according to their needs.

Analysis of in vivo single cell behavior by high throughput, human-in-the-loop segmentation of three-dimensional images

BACKGROUND

Analysis of single cells in their native environment is a powerful method to address key questions in developmental systems biology. Confocal microscopy imaging of intact tissues, followed by automatic image segmentation, provides a means to conduct cytometric studies while at the same time preserving crucial information about the spatial organization of the tissue and morphological features of the cells. This technique is rapidly evolving but is still not in widespread use among research groups that do not specialize in technique development, perhaps in part for lack of tools that automate repetitive tasks while allowing experts to make the best use of their time in injecting their domain-specific knowledge.

RESULTS

Here we focus on a well-established stem cell model system, the C. elegans gonad, as well as on two other model systems widely used to study cell fate specification and morphogenesis: the pre-implantation mouse embryo and the developing mouse olfactory epithelium. We report a pipeline that integrates machine-learning-based cell detection, fast human-in-the-loop curation of these detections, and running of active contours seeded from detections to segment cells. The procedure can be bootstrapped by a small number of manual detections, and outperforms alternative pieces of software we benchmarked on C. elegans gonad datasets. Using cell segmentations to quantify fluorescence contents, we report previously-uncharacterized cell behaviors in the model systems we used. We further show how cell morphological features can be used to identify cell cycle phase; this provides a basis for future tools that will streamline cell cycle experiments by minimizing the need for exogenous cell cycle phase labels.

CONCLUSIONS

High-throughput 3D segmentation makes it possible to extract rich information from images that are routinely acquired by biologists, and provides insights - in particular with respect to the cell cycle - that would be difficult to derive otherwise.

Survey statistics of automated segmentations applied to optical imaging of mammalian cells

BACKGROUND

The goal of this survey paper is to overview cellular measurements using optical microscopy imaging followed by automated image segmentation. The cellular measurements of primary interest are taken from mammalian cells and their components. They are denoted as two- or three-dimensional (2D or 3D) image objects of biological interest. In our applications, such cellular measurements are important for understanding cell phenomena, such as cell counts, cell-scaffold interactions, cell colony growth rates, or cell pluripotency stability, as well as for establishing quality metrics for stem cell therapies. In this context, this survey paper is focused on automated segmentation as a software-based measurement leading to quantitative cellular measurements.

METHODS

We define the scope of this survey and a classification schema first. Next, all found and manually filteredpublications are classified according to the main categories: (1) objects of interests (or objects to be segmented), (2) imaging modalities, (3) digital data axes, (4) segmentation algorithms, (5) segmentation evaluations, (6) computational hardware platforms used for segmentation acceleration, and (7) object (cellular) measurements. Finally, all classified papers are converted programmatically into a set of hyperlinked web pages with occurrence and co-occurrence statistics of assigned categories.

RESULTS

The survey paper presents to a reader: (a) the state-of-the-art overview of published papers about automated segmentation applied to optical microscopy imaging of mammalian cells, (b) a classification of segmentation aspects in the context of cell optical imaging, (c) histogram and co-occurrence summary statistics about cellular measurements, segmentations, segmented objects, segmentation evaluations, and the use of computational platforms for accelerating segmentation execution, and (d) open research problems to pursue.

CONCLUSIONS

The novel contributions of this survey paper are: (1) a new type of classification of cellular measurements and automated segmentation, (2) statistics about the published literature, and (3) a web hyperlinked interface to classification statistics of the surveyed papers at https://isg.nist.gov/deepzoomweb/resources/survey/index.html.

Automated profiling of individual cell-cell interactions from high-throughput time-lapse imaging microscopy in nanowell grids (TIMING)

MOTIVATION

There is a need for effective automated methods for profiling dynamic cell-cell interactions with single-cell resolution from high-throughput time-lapse imaging data, especially, the interactions between immune effector cells and tumor cells in adoptive immunotherapy.

RESULTS

Fluorescently labeled human T cells, natural killer cells (NK), and various target cells (NALM6, K562, EL4) were co-incubated on polydimethylsiloxane arrays of sub-nanoliter wells (nanowells), and imaged using multi-channel time-lapse microscopy. The proposed cell segmentation and tracking algorithms account for cell variability and exploit the nanowell confinement property to increase the yield of correctly analyzed nanowells from 45% (existing algorithms) to 98% for wells containing one effector and a single target, enabling automated quantification of cell locations, morphologies, movements, interactions, and deaths without the need for manual proofreading. Automated analysis of recordings from 12 different experiments demonstrated automated nanowell delineation accuracy >99%, automated cell segmentation accuracy >95%, and automated cell tracking accuracy of 90%, with default parameters, despite variations in illumination, staining, imaging noise, cell morphology, and cell clustering. An example analysis revealed that NK cells efficiently discriminate between live and dead targets by altering the duration of conjugation. The data also demonstrated that cytotoxic cells display higher motility than non-killers, both before and during contact.

CONTACT

broysam@central.uh.edu or nvaradar@central.uh.edu

SUPPLEMENTARY INFORMATION

Supplementary data are available at Bioinformatics online.

Segmentation and Tracking of Lymphocytes Based on Modified Active Contour Models in Phase Contrast Microscopy Images

The paper proposes an improved active contour model for segmenting and tracking accurate boundaries of the single lymphocyte in phase-contrast microscopic images. Active contour models have been widely used in object segmentation and tracking. However, current external-force-inspired methods are weak at handling low-contrast edges and suffer from initialization sensitivity. In order to segment low-contrast boundaries, we combine the region information of the object, extracted by morphology gray-scale reconstruction, and the edge information, extracted by the Laplacian of Gaussian filter, to obtain an improved feature map to compute the external force field for the evolution of active contours. To alleviate initial location sensitivity, we set the initial contour close to the real boundaries by performing morphological image processing. The proposed method was tested on live lymphocyte images acquired through the phase-contrast microscope from the blood samples of mice, and comparative experimental results showed the advantages of the proposed method in terms of the accuracy and the speed. Tracking experiments showed that the proposed method can accurately segment and track lymphocyte boundaries in microscopic images over time even in the presence of low-contrast edges, which will provide a good prerequisite for the quantitative analysis of lymphocyte morphology and motility.

Rapid 3-D delineation of cell nuclei for high-content screening platforms

High-resolution three-dimensional (3-D) microscopy combined with multiplexing of fluorescent labels allows high-content analysis of large numbers of cell nuclei. The full automation of 3-D screening platforms necessitates image processing algorithms that can accurately and robustly delineate nuclei in images with little to no human intervention. Imaging-based high-content screening was originally developed as a powerful tool for drug discovery. However, cell confluency, complexity of nuclear staining as well as poor contrast between nuclei and background result in slow and unreliable 3-D image processing and therefore negatively affect the performance of studying a drug response. Here, we propose a new method, 3D-RSD, to delineate nuclei by means of 3-D radial symmetries and test it on high-resolution image data of human cancer cells treated by drugs. The nuclei detection performance was evaluated by means of manually generated ground truth from 2351 nuclei (27 confocal stacks). When compared to three other nuclei segmentation methods, 3D-RSD possessed a better true positive rate of 83.3% and F-score of 0.895±0.045 (p-value=0.047). Altogether, 3D-RSD is a method with a very good overall segmentation performance. Furthermore, implementation of radial symmetries offers good processing speed, and makes 3D-RSD less sensitive to staining patterns. In particular, the 3D-RSD method performs well in cell lines, which are often used in imaging-based HCS platforms and are afflicted by nuclear crowding and overlaps that hinder feature extraction.

Global linking of cell tracks using the Viterbi algorithm

Automated tracking of living cells in microscopy image sequences is an important and challenging problem. With this application in mind, we propose a global track linking algorithm, which links cell outlines generated by a segmentation algorithm into tracks. The algorithm adds tracks to the image sequence one at a time, in a way which uses information from the complete image sequence in every linking decision. This is achieved by finding the tracks which give the largest possible increases to a probabilistically motivated scoring function, using the Viterbi algorithm. We also present a novel way to alter previously created tracks when new tracks are created, thus mitigating the effects of error propagation. The algorithm can handle mitosis, apoptosis, and migration in and out of the imaged area, and can also deal with false positives, missed detections, and clusters of jointly segmented cells. The algorithm performance is demonstrated on two challenging datasets acquired using bright-field microscopy, but in principle, the algorithm can be used with any cell type and any imaging technique, presuming there is a suitable segmentation algorithm.

Segmentation and tracking of stem cells in time lapse microscopy to quantify dynamic behavioral changes during spheroid formation

Dynamic behavior of stem cells during in vitro development is diverse. Previous cell tracking studies have focused more on cell proliferation than on cell aggregation. However, the enhancement of cell proliferation in association with cell aggregation has been reported. In a previous study, we also demonstrated that the aggregation of adult human mesenchymal stem cells to form three-dimensional (3D) cellular spheroids helped maintain the expression of stemness marker genes in the cells. However, the dynamic behavioral changes triggered by spheroid formation remain to be investigated. A scheme of image processing techniques is proposed to meet this need. A hybrid-thresholding technique was first developed for efficient segmentation of cell clusters, after which a cell tracking method based on pair-matching with topological constraints was designed. Two morphological indices were derived to track the timing of 3D spheroid formation during the cellular aggregation process. Five cell motility indices measured from single cells and 3D spheroids were then compared. After confirmation of more than 90% correspondence between the results obtained by manual tracking and the proposed methods, an analysis of cellular behavior reveals a significant increase in motility in association with spheroid formation, consistent with a previous report that used a gene expression approach. This study proposed a systematic image analysis method to quantify the dynamic behavior of stem cells for stemness evaluation during cell culturing in vitro. Results demonstrated the validity of the developed platform in investigation of the dynamic behavior of cell aggregation in stem cell cultures in vitro.

A robust algorithm for segmenting and tracking clustered cells in time-lapse fluorescent microscopy

We present herein a robust algorithm for cell tracking in a sequence of time-lapse 2-D fluorescent microscopy images. Tracking is performed automatically via a multiphase active contours algorithm adapted to the segmentation of clustered nuclei with obscure boundaries. An ellipse fitting method is applied to avoid problems typically associated with clustered, overlapping, or dying cells, and to obtain more accurate segmentation and tracking results. We provide quantitative validation of results obtained with this new algorithm by comparing them to the results obtained from the established CellProfiler, MTrack2 (plugin for Fiji), and LSetCellTracker software.

Comprehensive single cell-resolution analysis of the role of chromatin regulators in early C. elegans embryogenesis

Chromatin regulators are widely expressed proteins with diverse roles in gene expression, nuclear organization, cell cycle regulation, pluripotency, physiology and development, and are frequently mutated in human diseases such as cancer. Their inhibition often results in pleiotropic effects that are difficult to study using conventional approaches. We have developed a semi-automated nuclear tracking algorithm to quantify the divisions, movements and positions of all nuclei during the early development of Caenorhabditis elegans and have used it to systematically study the effects of inhibiting chromatin regulators. The resulting high dimensional datasets revealed that inhibition of multiple regulators, including F55A3.3 (encoding FACT subunit SUPT16H), lin-53 (RBBP4/7), rba-1 (RBBP4/7), set-16 (MLL2/3), hda-1 (HDAC1/2), swsn-7 (ARID2), and let-526 (ARID1A/1B) affected cell cycle progression and caused chromosome segregation defects. In contrast, inhibition of cir-1 (CIR1) accelerated cell division timing in specific cells of the AB lineage. The inhibition of RNA polymerase II also accelerated these division timings, suggesting that normal gene expression is required to delay cell cycle progression in multiple lineages in the early embryo. Quantitative analyses of the dataset suggested the existence of at least two functionally distinct SWI/SNF chromatin remodeling complex activities in the early embryo, and identified a redundant requirement for the egl-27 and lin-40 MTA orthologs in the development of endoderm and mesoderm lineages. Moreover, our dataset also revealed a characteristic rearrangement of chromatin to the nuclear periphery upon the inhibition of multiple general regulators of gene expression. Our systematic, comprehensive and quantitative datasets illustrate the power of single cell-resolution quantitative tracking and high dimensional phenotyping to investigate gene function. Furthermore, the results provide an overview of the functions of essential chromatin regulators during the early development of an animal.

A multistaged automatic restoration of noisy microscopy cell images

Automated cell segmentation for microscopy cell images has recently become an initial step for further image analysis in cell biology. However, microscopy cell images are easily degraded by noise during the readout procedure via optical-electronic imaging systems. Such noise degradations result in low signal-to-noise ratio (SNR) and poor image quality for cell identification. In order to improve SNR for subsequent segmentation and image-based quantitative analysis, the commonly used state-of-art restoration techniques are applied but few of them are suitable for corrupted microscopy cell images. In this paper, we propose a multistaged method based on a novel integration of trend surface analysis, quantile-quantile plot, bootstrapping, and the Gaussian spatial kernel for the restoration of noisy microscopy cell images. We show this multistaged approach achieves higher performance compared with other state-of-art restoration techniques in terms of peak signal-to-noise ratio and structure similarity in synthetic noise experiments. This paper also reports an experiment on real noisy microscopy data which demonstrated the advantages of the proposed restoration method for improving segmentation performance.

Context aware spatio-temporal cell tracking in densely packed multilayer tissues

Modern live imaging technique enables us to observe the internal part of a tissue over time by generating serial optical images containing spatio-temporal slices of hundreds of tightly packed cells. Automated tracking of plant and animal cells from such time lapse live-imaging datasets of a developing multicellular tissue is required for quantitative, high throughput analysis of cell division, migration and cell growth. In this paper, we present a novel cell tracking method that exploits the tight spatial topology of neighboring cells in a multicellular field as contextual information and combines it with physical features of individual cells for generating reliable cell lineages. The 2D image slices of multicellular tissues are modeled as a conditional random field and pairwise cell to cell similarities are obtained by estimating marginal probability distributions through loopy belief propagation on this CRF. These similarity scores are further used in a spatio-temporal graph labeling problem to obtain the optimal and feasible set of correspondences between individual cell slices across the 4D image dataset. We present results on (3D+t) confocal image stacks of Arabidopsis shoot meristem and show that the method is capable of handling many visual analysis challenges associated with such cell tracking problems, viz. poor feature quality of individual cells, low SNR in parts of images, variable number of cells across slices and cell division detection.

A novel framework for cellular tracking and mitosis detection in dense phase contrast microscopy images

The aim of this paper is to detail the development of a novel tracking framework that is able to extract the cell motility indicators and to determine the cellular division (mitosis) events in large time-lapse phase-contrast image sequences. To address the challenges induced by nonstructured (random) motion, cellular agglomeration, and cellular mitosis, the process of automatic (unsupervised) cell tracking is carried out in a sequential manner, where the interframe cell association is achieved by assessing the variation in the local cellular structures in consecutive frames of the image sequence. In our study, a strong emphasis has been placed on the robust use of the topological information in the cellular tracking process and in the development of targeted pattern recognition techniques that were designed to redress the problems caused by segmentation errors, and to precisely identify mitosis using a backward (reversed) tracking strategy. The proposed algorithm has been evaluated on dense phase-contrast cellular data and the experimental results indicate that the proposed algorithm is able to accurately track epithelial and endothelial cells in time-lapse image sequences that are characterized by low contrast and high level of noise. Our algorithm achieved 86.10% overall tracking accuracy and 90.12% mitosis detection accuracy.

Quantitative assessment of the relationship between cellular morphodynamics and signaling events by stochastic analysis of fluorescent images

Cell motility involves a number of strategies that cells use in order to seek nutrients, escape danger, and fulfill morphogenetic roles. Here we present a methodology to quantify morphological changes and their relationship with signaling events from time-lapse imaging microscopy experiments, in order to characterize physiological and pathological processes. To this aim, the stationary spatial pattern of signaling events is determined through an intracellular fluorescent probe, and it is related with the frequency and entity of morphodynamic events, which are in turn quantified through a stochastic approach: two pseudoimages are obtained from a time series of moving cells that describe the probability that a pixel belongs to the cell, and the probability that a pixel is subject to a dynamic event. The simultaneous construction of these maps permits visualization of hot spots of dynamic events, i.e., zones of formation of membrane protrusions and retractions and their relationship with the signaling events reported by the specific probe employed. The method is tested on spontaneous movement of cells, trasfected with redox-sensitive yellow fluorescent protein, in which the distribution of the hot spots and its change upon expression of constitutively active Rac (V12-Rac), is related to the distribution of oxidized spots.

Improved and robust detection of cell nuclei from four dimensional fluorescence images

Segmentation-free direct methods are quite efficient for automated nuclei extraction from high dimensional images. A few such methods do exist but most of them do not ensure algorithmic robustness to parameter and noise variations. In this research, we propose a method based on multiscale adaptive filtering for efficient and robust detection of nuclei centroids from four dimensional (4D) fluorescence images. A temporal feedback mechanism is employed between the enhancement and the initial detection steps of a typical direct method. We estimate the minimum and maximum nuclei diameters from the previous frame and feed back them as filter lengths for multiscale enhancement of the current frame. A radial intensity-gradient function is optimized at positions of initial centroids to estimate all nuclei diameters. This procedure continues for processing subsequent images in the sequence. Above mechanism thus ensures proper enhancement by automated estimation of major parameters. This brings robustness and safeguards the system against additive noises and effects from wrong parameters. Later, the method and its single-scale variant are simplified for further reduction of parameters. The proposed method is then extended for nuclei volume segmentation. The same optimization technique is applied to final centroid positions of the enhanced image and the estimated diameters are projected onto the binary candidate regions to segment nuclei volumes.Our method is finally integrated with a simple sequential tracking approach to establish nuclear trajectories in the 4D space. Experimental evaluations with five image-sequences (each having 271 3D sequential images) corresponding to five different mouse embryos show promising performances of our methods in terms of nuclear detection, segmentation, and tracking. A detail analysis with a sub-sequence of 101 3D images from an embryo reveals that the proposed method can improve the nuclei detection accuracy by 9 over the previous methods, which used inappropriate large valued parameters. Results also confirm that the proposed method and its variants achieve high detection accuracies ( 98 mean F-measure) irrespective of the large variations of filter parameters and noise levels.

A simple optical configuration for cell tracking by dark-field microscopy

We describe a simple optical configuration for dark-field microscopy at low magnification, realized with the use of standard microscope components. An inherent high contrast makes this method attractive for computer-assisted tracking and counting of microorganisms. We applied this setup for dark-field microscopy to measure the speed of migrating Dictyostelium amoebae.

Active structured learning for cell tracking: algorithm, framework, and usability

One distinguishing property of life is its temporal dynamics, and it is hence only natural that time lapse experiments play a crucial role in modern biomedical research areas such as signaling pathways, drug discovery or developmental biology. Such experiments yield a very large number of images that encode complex cellular activities, and reliable automated cell tracking emerges naturally as a prerequisite for further quantitative analysis. However, many existing cell tracking methods are restricted to using only a small number of features to allow for manual tweaking. In this paper, we propose a novel cell tracking approach that embraces a powerful machine learning technique to optimize the tracking parameters based on user annotated tracks. Our approach replaces the tedious parameter tuning with parameter learning and allows for the use of a much richer set of complex tracking features, which in turn affords superior prediction accuracy. Furthermore, we developed an active learning approach for efficient training data retrieval, which reduces the annotation effort to only 17%. In practical terms, our approach allows life science researchers to inject their expertise in a more intuitive and direct manner. This process is further facilitated by using a glyph visualization technique for ground truth annotation and validation. Evaluation and comparison on several publicly available benchmark sequences show significant performance improvement over recently reported approaches. Code and software tools are provided to the public.

A benchmark for comparison of cell tracking algorithms

Motivation: Automatic tracking of cells in multidimensional time-lapse fluorescence microscopy is an important task in many biomedical applications. A novel framework for objective evaluation of cell tracking algorithms has been established under the auspices of the IEEE International Symposium on Biomedical Imaging 2013 Cell Tracking Challenge. In this article, we present the logistics, datasets, methods and results of the challenge and lay down the principles for future uses of this benchmark.

Results: The main contributions of the challenge include the creation of a comprehensive video dataset repository and the definition of objective measures for comparison and ranking of the algorithms. With this benchmark, six algorithms covering a variety of segmentation and tracking paradigms have been compared and ranked based on their performance on both synthetic and real datasets. Given the diversity of the datasets, we do not declare a single winner of the challenge. Instead, we present and discuss the results for each individual dataset separately.

Availability and implementation: The challenge Web site (http://www.codesolorzano.com/celltrackingchallenge) provides access to the training and competition datasets, along with the ground truth of the training videos. It also provides access to Windows and Linux executable files of the evaluation software and most of the algorithms that competed in the challenge.

Contact:codesolorzano@unav.es

Supplementary information:Supplementary data are available at Bioinformatics online.

An efficient, scalable, and adaptable framework for solving generic systems of level-set PDEs

In the last decade, level-set methods have been actively developed for applications in image registration, segmentation, tracking, and reconstruction. However, the development of a wide variety of level-set PDEs and their numerical discretization schemes, coupled with hybrid combinations of PDE terms, stopping criteria, and reinitialization strategies, has created a software logistics problem. In the absence of an integrative design, current toolkits support only specific types of level-set implementations which restrict future algorithm development since extensions require significant code duplication and effort. In the new NIH/NLM Insight Toolkit (ITK) v4 architecture, we implemented a level-set software design that is flexible to different numerical (continuous, discrete, and sparse) and grid representations (point, mesh, and image-based). Given that a generic PDE is a summation of different terms, we used a set of linked containers to which level-set terms can be added or deleted at any point in the evolution process. This container-based approach allows the user to explore and customize terms in the level-set equation at compile-time in a flexible manner. The framework is optimized so that repeated computations of common intensity functions (e.g., gradient and Hessians) across multiple terms is eliminated. The framework further enables the evolution of multiple level-sets for multi-object segmentation and processing of large datasets. For doing so, we restrict level-set domains to subsets of the image domain and use multithreading strategies to process groups of subdomains or level-set functions. Users can also select from a variety of reinitialization policies and stopping criteria. Finally, we developed a visualization framework that shows the evolution of a level-set in real-time to help guide algorithm development and parameter optimization. We demonstrate the power of our new framework using confocal microscopy images of cells in a developing zebrafish embryo.

A novel framework for segmentation of secretory granules in electron micrographs

It is still a standard practice for biologists to manually analyze transmission electron microscopy images. This is not only time consuming but also not reproducible and prone to induce subjective bias. For large-scale studies of insulin granules inside beta cells of the islet of Langerhans, an automated method for analysis is essential. Due to the complex structure of the images, standard microscopy segmentation techniques cannot be applied. We present a new approach to segment and measure transmission electron microscopy images of insulin granule cores and membranes from beta cells of rat islets of Langerhans. The algorithm is separated into two broad components, core segmentation and membrane segmentation. Core segmentation proceeds through three steps: pre-segmentation using a novel level-set active contour, morphological cleaning and a refining segmentation on each granule using a novel dual level-set active contour. Membrane segmentation is achieved in four steps: morphological cleaning, membrane sampling and scaling, vector field convolution for gap filling and membrane verification using a novel convergence filter. We show results from our algorithm alongside popular microscopy segmentation methods; the advantages of our method are demonstrated. Our algorithm is validated by comparing automated results to a manually defined ground truth. When the number of granules detected is compared to the number of granules in the ground truth a precision of 91% and recall of 87% is observed. The average granule areas differ by 13.35% and 6.08% for core and membranes respectively, when compared to the average areas of the ground truth. These results compare favorably to previously published data.

Cell tracking in microscopic video using matching and linking of bipartite graphs

Automated visual tracking of cells from video microscopy has many important biomedical applications. In this paper, we track human monocyte cells in a fluorescent microscopic video using matching and linking of bipartite graphs. Tracking of cells over a pair of frames is modeled as a maximum cardinality minimum weight matching problem for a bipartite graph with a novel cost function. The tracking results are further refined using a rank-based filtering mechanism. Linking of cell trajectories over different frames are achieved through composition of bipartite matches. The proposed solution does not require any explicit motion model, is highly scalable, and, can effectively handle the entry and exit of cells. Our tracking accuracy of (97.97±0.94)% is superior than several existing methods [(95.66±2.39)%, (94.42±2.08)%, (81.22±5.75)%, (78.31±4.70)%] and is highly comparable (98.20±1.22)% to a recently published algorithm.

Reconstruction of missing cells by a Killing energy minimizing nonrigid image registration

Fluorescent microscopy has been a popular and important tool for studying live cells. One challenge of analyzing cell images obtained from fluorescent microscopy is that cells in fluorescent images frequently disappear and reappear, making cell tracking difficult. In this paper, we present an image registration approach which can reconstruct both the cell appearance and location of the missing cells from the image frames where the cells become invisible. The idea is to perform an image registration on the images before and after a cell disappears. The missing image frames between these two images are given by the intermediate registration results. The formulation is based on the nonrigid particle registration model, which captures soft deformation of the cells. In addition, to obtain natural and more rigid cell movements such as translation and rotation, we propose a new registration technique which is Killing energy minimizing, motivated by the fact that a Killing vector field with zero Killing energy will generate an isometric deformation. We will present reconstruction results of C2C12 cells in fluorescent images to illustrate the effectiveness of our model by different numerical examples.

Cell tracking using phase-adaptive shape prior

Automated tracking of cell population is very crucial for quantitative measurements of dynamic cell-cycle behaviour of individual cells. This problem involves several subproblems and a high accuracy of each step is essential to avoid error propagation. In this paper, we propose a holistic three-component system to tackle this problem. For each phase, we first learn a mean shape as well as a model of the temporal dynamics of transformation, which are used for estimating a shape prior for the cell in the current frame. We then segment the cell using a level set-based shape prior model. Finally, we identify its phase based on the goodness-of-fit of the data to the segmentation model. This phase information is also used for fine-tuning the segmentation result. We evaluate the performance of our method empirically in various aspects and in tracking individual cells from HeLa H2B-GFP cell population. Highly accurate validation results confirm the robustness of our method in many realistic scenarios and the essentiality of each component of our integrating system.

Region-based progressive localization of cell nuclei in microscopic images with data adaptive modeling

Background

Segmenting cell nuclei in microscopic images has become one of the most important routines in modern biological applications. With the vast amount of data, automatic localization, i.e. detection and segmentation, of cell nuclei is highly desirable compared to time-consuming manual processes. However, automated segmentation is challenging due to large intensity inhomogeneities in the cell nuclei and the background.

Results

We present a new method for automated progressive localization of cell nuclei using data-adaptive models that can better handle the inhomogeneity problem. We perform localization in a three-stage approach: first identify all interest regions with contrast-enhanced salient region detection, then process the clusters to identify true cell nuclei with probability estimation via feature-distance profiles of reference regions, and finally refine the contours of detected regions with regional contrast-based graphical model. The proposed region-based progressive localization (RPL) method is evaluated on three different datasets, with the first two containing grayscale images, and the third one comprising of color images with cytoplasm in addition to cell nuclei. We demonstrate performance improvement over the state-of-the-art. For example, compared to the second best approach, on the first dataset, our method achieves 2.8 and 3.7 reduction in Hausdorff distance and false negatives; on the second dataset that has larger intensity inhomogeneity, our method achieves 5% increase in Dice coefficient and Rand index; on the third dataset, our method achieves 4% increase in object-level accuracy.

Conclusions

To tackle the intensity inhomogeneities in cell nuclei and background, a region-based progressive localization method is proposed for cell nuclei localization in fluorescence microscopy images. The RPL method is demonstrated highly effective on three different public datasets, with on average 3.5% and 7% improvement of region- and contour-based segmentation performance over the state-of-the-art.

A new way of tracking motion, shape, and divisions

The process of detecting and tracking biological features such as bacteria and nuclei is complicated by the fact that they constantly change their shape. Shape changes happen both continuously as the biological features grow and discontinuously as they divide or die. In this paper I present a new method of tracking such features for the case that they can be reasonably approximated by a relatively simple mathematical shape such as a cylinder or an ellipse. Using contour plots with multiple levels to detect the features and their shapes, rather than the commonly used single contour detection technique, this method can efficiently detect multiple features even if they have large differences in brightness, as well as reliably track divisions when both brightness and size drop dramatically.

Segmentation and shape tracking of whole fluorescent cells based on the Chan-Vese model

We present a fast and robust approach to tracking the evolving shape of whole fluorescent cells in time-lapse series. The proposed tracking scheme involves two steps. First, coherence-enhancing diffusion filtering is applied on each frame to reduce the amount of noise and enhance flow-like structures. Second, the cell boundaries are detected by minimizing the Chan-Vese model in the fast level set-like and graph cut frameworks. To allow simultaneous tracking of multiple cells over time, both frameworks have been integrated with a topological prior exploiting the object indication function. The potential of the proposed tracking scheme and the advantages and disadvantages of both frameworks are demonstrated on 2-D and 3-D time-lapse series of rat adipose-derived mesenchymal stem cells and human lung squamous cell carcinoma cells, respectively.

A flexible and robust approach for segmenting cell nuclei from 2D microscopy images using supervised learning and template matching

We describe a new supervised learning-based template matching approach for segmenting cell nuclei from microscopy images. The method uses examples selected by a user for building a statistical model that captures the texture and shape variations of the nuclear structures from a given dataset to be segmented. Segmentation of subsequent, unlabeled, images is then performed by finding the model instance that best matches (in the normalized cross correlation sense) local neighborhood in the input image. We demonstrate the application of our method to segmenting nuclei from a variety of imaging modalities, and quantitatively compare our results to several other methods. Quantitative results using both simulated and real image data show that, while certain methods may work well for certain imaging modalities, our software is able to obtain high accuracy across several imaging modalities studied. Results also demonstrate that, relative to several existing methods, the template-based method we propose presents increased robustness in the sense of better handling variations in illumination, variations in texture from different imaging modalities, providing more smooth and accurate segmentation borders, as well as handling better cluttered nuclei.

Chapter 17: bioimage informatics for systems pharmacology

Recent advances in automated high-resolution fluorescence microscopy and robotic handling have made the systematic and cost effective study of diverse morphological changes within a large population of cells possible under a variety of perturbations, e.g., drugs, compounds, metal catalysts, RNA interference (RNAi). Cell population-based studies deviate from conventional microscopy studies on a few cells, and could provide stronger statistical power for drawing experimental observations and conclusions. However, it is challenging to manually extract and quantify phenotypic changes from the large amounts of complex image data generated. Thus, bioimage informatics approaches are needed to rapidly and objectively quantify and analyze the image data. This paper provides an overview of the bioimage informatics challenges and approaches in image-based studies for drug and target discovery. The concepts and capabilities of image-based screening are first illustrated by a few practical examples investigating different kinds of phenotypic changes caEditorsused by drugs, compounds, or RNAi. The bioimage analysis approaches, including object detection, segmentation, and tracking, are then described. Subsequently, the quantitative features, phenotype identification, and multidimensional profile analysis for profiling the effects of drugs and targets are summarized. Moreover, a number of publicly available software packages for bioimage informatics are listed for further reference. It is expected that this review will help readers, including those without bioimage informatics expertise, understand the capabilities, approaches, and tools of bioimage informatics and apply them to advance their own studies.