Cell population tracking and lineage construction with spatiotemporal context

An active particle-based tracking framework for 2D and 3D time-lapse microscopy images

The process required to track cellular structures is a key task in the study of cell migration. This allows the accurate estimation of motility indicators that help in the understanding of mechanisms behind various biological processes. This paper reports a particle-based fully automatic tracking framework that is able to quantify the motility of living cells in time-lapse images. Contrary to the standard tracking methods based on predefined motion models, in this paper we reformulate the tracking mechanism as a data driven optimization process to remove its reliance on a priory motion models. The proposed method has been evaluated using 2D and 3D deconvolved epifluorescent in-vivo image sequences that describe the development of the quail embryo.

Automated cell identification and tracking using nanoparticle moving-light-displays

An automated technique for the identification, tracking and analysis of biological cells is presented. It is based on the use of nanoparticles, enclosed within intra-cellular vesicles, to produce clusters of discrete, point-like fluorescent, light sources within the cells. Computational analysis of these light ensembles in successive time frames of a movie sequence, using k-means clustering and particle tracking algorithms, provides robust and automated discrimination of live cells and their motion and a quantitative measure of their proliferation. This approach is a cytometric version of the moving light display technique which is widely used for analyzing the biological motion of humans and animals. We use the endocytosis of CdTe/ZnS, core-shell quantum dots to produce the light displays within an A549, epithelial, lung cancer cell line, using time-lapse imaging with frame acquisition every 5 minutes over a 40 hour time period. The nanoparticle moving light displays provide simultaneous collection of cell motility data, resolution of mitotic traversal dynamics and identification of familial relationships allowing construction of multi-parameter lineage trees.

Ensemble analysis of angiogenic growth in three-dimensional microfluidic cell cultures

We demonstrate ensemble three-dimensional cell cultures and quantitative analysis of angiogenic growth from uniform endothelial monolayers. Our approach combines two key elements: a micro-fluidic assay that enables parallelized angiogenic growth instances subject to common extracellular conditions, and an automated image acquisition and processing scheme enabling high-throughput, unbiased quantification of angiogenic growth. Because of the increased throughput of the assay in comparison to existing three-dimensional morphogenic assays, statistical properties of angiogenic growth can be reliably estimated. We used the assay to evaluate the combined effects of vascular endothelial growth factor (VEGF) and the signaling lipid sphingoshine-1-phosphate (S1P). Our results show the importance of S1P in amplifying the angiogenic response in the presence of VEGF gradients. Furthermore, the application of S1P with VEGF gradients resulted in angiogenic sprouts with higher aspect ratio than S1P with background levels of VEGF, despite reduced total migratory activity. This implies a synergistic effect between the growth factors in promoting angiogenic activity. Finally, the variance in the computed angiogenic metrics (as measured by ensemble standard deviation) was found to increase linearly with the ensemble mean. This finding is consistent with stochastic agent-based mathematical models of angiogenesis that represent angiogenic growth as a series of independent stochastic cell-level decisions.

Enabling user-guided segmentation and tracking of surface-labeled cells in time-lapse image sets of living tissues

To study the process of morphogenesis, one often needs to collect and segment time-lapse images of living tissues to accurately track changing cellular morphology. This task typically involves segmenting and tracking tens to hundreds of individual cells over hundreds of image frames, a scale that would certainly benefit from automated routines; however, any automated routine would need to reliably handle a large number of sporadic, and yet typical problems (e.g., illumination inconsistency, photobleaching, rapid cell motions, and drift of focus or of cells moving through the imaging plane). Here, we present a segmentation and cell tracking approach based on the premise that users know their data best-interpreting and using image features that are not accounted for in any a priori algorithm design. We have developed a program, SeedWater Segmenter, that combines a parameter-less and fast automated watershed algorithm with a suite of manual intervention tools that enables users with little to no specialized knowledge of image processing to efficiently segment images with near-perfect accuracy based on simple user interactions.

Cell cycle dependent TN-C promoter activity determined by live cell imaging

The extracellular matrix protein tenascin-C plays a critical role in development, wound healing, and cancer progression, but how it is controlled and how it exerts its physiological responses remain unclear. By quantifying the behavior of live cells with phase contrast and fluorescence microscopy, the dynamic regulation of TN-C promoter activity is examined. We employ an NIH 3T3 cell line stably transfected with the TN-C promoter ligated to the gene sequence for destabilized green fluorescent protein (GFP). Fully automated image analysis routines, validated by comparison with data derived from manual segmentation and tracking of single cells, are used to quantify changes in the cellular GFP in hundreds of individual cells throughout their cell cycle during live cell imaging experiments lasting 62 h. We find that individual cells vary substantially in their expression patterns over the cell cycle, but that on average TN-C promoter activity increases during the last 40% of the cell cycle. We also find that the increase in promoter activity is proportional to the activity earlier in the cell cycle. This work illustrates the application of live cell microscopy and automated image analysis of a promoter-driven GFP reporter cell line to identify subtle gene regulatory mechanisms that are difficult to uncover using population averaged measurements.

Image-based characterization of thrombus formation in time-lapse DIC microscopy

The characterization of thrombus formation in time-lapse DIC microscopy is of increased interest for identifying genes which account for atherothrombosis and coronary artery diseases (CADs). In particular, we are interested in large-scale studies on zebrafish, which result in large amount of data, and require automatic processing. In this work, we present an image-based solution for the automatized extraction of parameters quantifying the temporal development of thrombotic plugs. Our system is based on the joint segmentation of thrombotic and aortic regions over time. This task is made difficult by the low contrast and the high dynamic conditions observed in vivo DIC microscopic scenes. Our key idea is to perform this segmentation by distinguishing the different motion patterns in image time series rather than by solving standard image segmentation tasks in each image frame. Thus, we are able to compensate for the poor imaging conditions. We model motion patterns by energies based on the idea of dynamic textures, and regularize the model by two prior energies on the shape of the aortic region and on the topological relationship between the thrombus and the aorta. We demonstrate the performance of our segmentation algorithm by qualitative and quantitative experiments on synthetic examples as well as on real in vivo microscopic sequences.

Understanding the phase contrast optics to restore artifact-free microscopy images for segmentation

Phase contrast, a noninvasive microscopy imaging technique, is widely used to capture time-lapse images to monitor the behavior of transparent cells without staining or altering them. Due to the optical principle, phase contrast microscopy images contain artifacts such as the halo and shade-off that hinder image segmentation, a critical step in automated microscopy image analysis. Rather than treating phase contrast microscopy images as general natural images and applying generic image processing techniques on them, we propose to study the optical properties of the phase contrast microscope to model its image formation process. The phase contrast imaging system can be approximated by a linear imaging model. Based on this model and input image properties, we formulate a regularized quadratic cost function to restore artifact-free phase contrast images that directly correspond to the specimen's optical path length. With artifacts removed, high quality segmentation can be achieved by simply thresholding the restored images. The imaging model and restoration method are quantitatively evaluated on microscopy image sequences with thousands of cells captured over several days. We also demonstrate that accurate restoration lays the foundation for high performance in cell detection and tracking.

A semi-Markov model for mitosis segmentation in time-lapse phase contrast microscopy image sequences of stem cell populations

We propose a semi-Markov model trained in a max-margin learning framework for mitosis event segmentation in large-scale time-lapse phase contrast microscopy image sequences of stem cell populations. Our method consists of three steps. First, we apply a constrained optimization based microscopy image segmentation method that exploits phase contrast optics to extract candidate subsequences in the input image sequence that contains mitosis events. Then, we apply a max-margin hidden conditional random field (MM-HCRF) classifier learned from human-annotated mitotic and nonmitotic sequences to classify each candidate subsequence as a mitosis or not. Finally, a max-margin semi-Markov model (MM-SMM) trained on manually-segmented mitotic sequences is utilized to reinforce the mitosis classification results, and to further segment each mitosis into four predefined temporal stages. The proposed method outperforms the event-detection CRF model recently reported by Huh as well as several other competing methods in very challenging image sequences of multipolar-shaped C3H10T1/2 mesenchymal stem cells. For mitosis detection, an overall precision of 95.8% and a recall of 88.1% were achieved. For mitosis segmentation, the mean and standard deviation for the localization errors of the start and end points of all mitosis stages were well below 1 and 2 frames, respectively. In particular, an overall temporal location error of 0.73 ± 1.29 frames was achieved for locating daughter cell birth events.

Cell tracking and mitosis detection using splitting flow networks in phase-contrast imaging

Cell tracking is a crucial component of many biomedical image analysis applications. Many available cell tracking systems assume high precision of the cell detection module. Therefore low performance in cell detection can heavily affect the tracking results. Unfortunately cell segmentation modules often have significant errors, especially in the case of phase-contrast imaging. In this paper we propose a tracking method that does not rely on perfect cell segmentation and can deal with uncertainties by exploiting temporal information and aggregating the results of many frames. Our tracking algorithm is fully automated and can handle common challenges of tracking such as cells entering/exiting the screen and mitosis events. To handle the latter, we modify the standard flow network and introduce the concept of a splitting node into it. Experiment results show that adding temporal information from the video microscopy improves the cell/mitosis detection and results in a better tracking system.

Electrotaxis of oral squamous cell carcinoma cells in a multiple-electric-field chip with uniform flow field

We report a new design of microfluidic chip (Multiple electric Field with Uniform Flow chip, MFUF chip) to create multiple electric field strengths (EFSs) while providing a uniform flow field simultaneously. MFUF chip was fabricated from poly-methyl methacrylates (PMMA) substrates by using CO2 laser micromachining. A microfluidic network with interconnecting segments was utilized to de-couple the flow field and the electric field (EF). Using our special design, different EFSs were obtained in channel segments that had an identical cross-section and therefore a uniform flow field. Four electric fields with EFS ratio of 7.9:2.8:1:0 were obtained with flow velocity variation of only 7.8% CV (coefficient of variation). Possible biological effect of shear force can therefore be avoided. Cell behavior under three EFSs and the control condition, where there is no EF, was observed in a single experiment. We validated MFUF chip performance using lung adenocarcinoma cell lines and then used the chip to study the electrotaxis of HSC-3, an oral squamous cell carcinoma cell line. The MFUF chip has high throughput capability for studying the EF-induced cell behavior under various EFSs, including the control condition (EFS = 0).

Detecting and tracking motion of Myxococcus xanthus bacteria in swarms

Automatically detecting and tracking the motion of Myxococcus xanthus bacteria provide essential information for studying bacterial cell motility mechanisms and collective behaviors. However, this problem is difficult due to the low contrast of microscopy images, cell clustering and colliding behaviors, etc. To overcome these difficulties, our approach starts with a level set based pre-segmentation of cell clusters, followed by an enhancement of the rod-like cell features and detection of individual bacterium within each cluster. A novel method based on "spikes" of the outer medial axis is applied to divide touching (colliding) cells. The tracking of cell motion is accomplished by a non-crossing bipartite graph matching scheme that matches not only individual cells but also the neighboring structures around each cell. Our approach was evaluated on image sequences of moving M. xanthus bacteria close to the edge of their swarms, achieving high accuracy on the test data sets.

Software for precise tracking of cell proliferation

We have developed a multi-target cell tracking program TADOR, which we applied to a series of fluorescence images. TADOR is based on an active contour model that is modified in order to be free of the problem of locally optimal solutions, and thus is resistant to signal fluctuation and morphological changes. Due to adoption of backward tracing and addition of user-interactive correction functions, TADOR is used in an off-line and semi-automated mode, but enables precise tracking of cell division. By applying TADOR to the analysis of cultured cells whose nuclei had been fluorescently labeled, we tracked cell division and cell-cycle progression on coverslips over an extended period of time.

Extracting fluorescent reporter time courses of cell lineages from high-throughput microscopy at low temporal resolution

The extraction of fluorescence time course data is a major bottleneck in high-throughput live-cell microscopy. Here we present an extendible framework based on the open-source image analysis software ImageJ, which aims in particular at analyzing the expression of fluorescent reporters through cell divisions. The ability to track individual cell lineages is essential for the analysis of gene regulatory factors involved in the control of cell fate and identity decisions. In our approach, cell nuclei are identified using Hoechst, and a characteristic drop in Hoechst fluorescence helps to detect dividing cells. We first compare the efficiency and accuracy of different segmentation methods and then present a statistical scoring algorithm for cell tracking, which draws on the combination of various features, such as nuclear intensity, area or shape, and importantly, dynamic changes thereof. Principal component analysis is used to determine the most significant features, and a global parameter search is performed to determine the weighting of individual features. Our algorithm has been optimized to cope with large cell movements, and we were able to semi-automatically extract cell trajectories across three cell generations. Based on the MTrackJ plugin for ImageJ, we have developed tools to efficiently validate tracks and manually correct them by connecting broken trajectories and reassigning falsely connected cell positions. A gold standard consisting of two time-series with 15,000 validated positions will be released as a valuable resource for benchmarking. We demonstrate how our method can be applied to analyze fluorescence distributions generated from mouse stem cells transfected with reporter constructs containing transcriptional control elements of the Msx1 gene, a regulator of pluripotency, in mother and daughter cells. Furthermore, we show by tracking zebrafish PAC2 cells expressing FUCCI cell cycle markers, our framework can be easily adapted to different cell types and fluorescent markers.

An interacting multiple model filter-based autofocus strategy for confocal time-lapse microscopy

Gene expression and other cellular processes are stochastic, thus their study requires observing multiple events in multiple cells. Therefore, confocal microscopy cell imaging has recently gained much interest. In time-lapse imaging, adjustments are needed at short intervals to compensate for focus drift. There are several automated methods for this purpose. In general, before acquiring higher resolution images, software-based autofocus algorithms require a set of low-resolution images along the z-axis to determine the plane for which a predefined focusing function is maximized. These algorithms require 10-100 z-slices each time, and there is no fixed number or upper limit of required z-slices that ensures optimal focusing. The higher is this number, the stronger is photo bleaching, hampering the feasibility of long-time series measurements. We propose a new focusing strategy in time-lapse imaging. The algorithm relies on the nature and predictability of the focus drift. We first show that the focus drift curve is predictable within a small error bound in standard experimental setups. We, then, exploit the interacting multiple model filter algorithm to predict the drift at time, t, based on the measurement at time t-1. This allows a drastic reduction of the number of required z-slices for focus drift correction, largely overcoming the problem of photo bleaching. In addition, we propose a new set of functions for focusing in time-lapse imaging, derived from preexisting ones. We demonstrate the method's efficiency in time-lapse imaging of Escherichia coli cells expressing MS2d-GFP tagged RNA molecules.

Vertebrate neural stem cell segmentation, tracking and lineaging with validation and editing

This protocol and the accompanying software program called LEVER (lineage editing and validation) enable quantitative automated analysis of phase-contrast time-lapse images of cultured neural stem cells. Images are captured at 5-min intervals over a period of 5-15 d as the cells proliferate and differentiate. LEVER automatically segments, tracks and generates lineage trees of the stem cells from the image sequence. In addition to generating lineage trees capturing the population dynamics of clonal development, LEVER extracts quantitative phenotypic measurements of cell location, shape, movement and size. When available, the system can include biomolecular markers imaged using fluorescence. It then displays the results to the user for highly efficient inspection and editing to correct any errors in the segmentation, tracking or lineaging. To enable high-throughput inspection, LEVER incorporates features for rapid identification of errors and for learning from user-supplied corrections to automatically identify and correct related errors.

An engineered approach to stem cell culture: automating the decision process for real-time adaptive subculture of stem cells

Current cell culture practices are dependent upon human operators and remain laborious and highly subjective, resulting in large variations and inconsistent outcomes, especially when using visual assessments of cell confluency to determine the appropriate time to subculture cells. Although efforts to automate cell culture with robotic systems are underway, the majority of such systems still require human intervention to determine when to subculture. Thus, it is necessary to accurately and objectively determine the appropriate time for cell passaging. Optimal stem cell culturing that maintains cell pluripotency while maximizing cell yields will be especially important for efficient, cost-effective stem cell-based therapies. Toward this goal we developed a real-time computer vision-based system that monitors the degree of cell confluency with a precision of 0.791±0.031 and recall of 0.559±0.043. The system consists of an automated phase-contrast time-lapse microscope and a server. Multiple dishes are sequentially imaged and the data is uploaded to the server that performs computer vision processing, predicts when cells will exceed a pre-defined threshold for optimal cell confluency, and provides a Web-based interface for remote cell culture monitoring. Human operators are also notified via text messaging and e-mail 4 hours prior to reaching this threshold and immediately upon reaching this threshold. This system was successfully used to direct the expansion of a paradigm stem cell population, C2C12 cells. Computer-directed and human-directed control subcultures required 3 serial cultures to achieve the theoretical target cell yield of 50 million C2C12 cells and showed no difference for myogenic and osteogenic differentiation. This automated vision-based system has potential as a tool toward adaptive real-time control of subculturing, cell culture optimization and quality assurance/quality control, and it could be integrated with current and developing robotic cell cultures systems to achieve complete automation.

A novel validation algorithm allows for automated cell tracking and the extraction of biologically meaningful parameters

Automated microscopy is currently the only method to non-invasively and label-free observe complex multi-cellular processes, such as cell migration, cell cycle, and cell differentiation. Extracting biological information from a time-series of micrographs requires each cell to be recognized and followed through sequential microscopic snapshots. Although recent attempts to automatize this process resulted in ever improving cell detection rates, manual identification of identical cells is still the most reliable technique. However, its tedious and subjective nature prevented tracking from becoming a standardized tool for the investigation of cell cultures. Here, we present a novel method to accomplish automated cell tracking with a reliability comparable to manual tracking. Previously, automated cell tracking could not rival the reliability of manual tracking because, in contrast to the human way of solving this task, none of the algorithms had an independent quality control mechanism; they missed validation. Thus, instead of trying to improve the cell detection or tracking rates, we proceeded from the idea to automatically inspect the tracking results and accept only those of high trustworthiness, while rejecting all other results. This validation algorithm works independently of the quality of cell detection and tracking through a systematic search for tracking errors. It is based only on very general assumptions about the spatiotemporal contiguity of cell paths. While traditional tracking often aims to yield genealogic information about single cells, the natural outcome of a validated cell tracking algorithm turns out to be a set of complete, but often unconnected cell paths, i.e. records of cells from mitosis to mitosis. This is a consequence of the fact that the validation algorithm takes complete paths as the unit of rejection/acceptance. The resulting set of complete paths can be used to automatically extract important biological parameters with high reliability and statistical significance. These include the distribution of life/cycle times and cell areas, as well as of the symmetry of cell divisions and motion analyses. The new algorithm thus allows for the quantification and parameterization of cell culture with unprecedented accuracy. To evaluate our validation algorithm, two large reference data sets were manually created. These data sets comprise more than 320,000 unstained adult pancreatic stem cells from rat, including 2592 mitotic events. The reference data sets specify every cell position and shape, and assign each cell to the correct branch of its genealogic tree. We provide these reference data sets for free use by others as a benchmark for the future improvement of automated tracking methods.

TimeLapseAnalyzer: multi-target analysis for live-cell imaging and time-lapse microscopy

The direct observation of cells over time using time-lapse microscopy can provide deep insights into many important biological processes. Reliable analyses of motility, proliferation, invasive potential or mortality of cells are essential to many studies involving live cell imaging and can aid in biomarker discovery and diagnostic decisions. Given the vast amount of image- and time-series data produced by modern microscopes, automated analysis is a key feature to capitalize the potential of time-lapse imaging devices. To provide fast and reproducible analyses of multiple aspects of cell behaviour, we developed TimeLapseAnalyzer. Apart from general purpose image enhancements and segmentation procedures, this extensible, self-contained, modular cross-platform package provides dedicated modalities for fast and reliable analysis of multi-target cell tracking, scratch wound healing analysis, cell counting and tube formation analysis in high throughput screening of live-cell experiments. TimeLapseAnalyzer is freely available (MATLAB, Open Source) at http://www.informatik.uni-ulm.de/ni/mitarbeiter/HKestler/tla.

High-resolution cell outline segmentation and tracking from phase-contrast microscopy images

Accurate extraction of cell outlines from microscopy images is essential for analysing the dynamics of migrating cells. Phase-contrast microscopy is one of the most common and convenient imaging modalities for observing cell motility because it does not require exogenous labelling and uses only moderate light levels with generally negligible phototoxicity effects. Automatic extraction and tracking of high-resolution cell outlines from phase-contrast images, however, is difficult due to complex and non-uniform edge intensity. We present a novel image-processing method based on refined level-set segmentation for accurate extraction of cell outlines from high-resolution phase-contrast images. The algorithm is validated on synthetic images of defined noise levels and applied to real image sequences of polarizing and persistently migrating keratocyte cells. We demonstrate that the algorithm is able to reliably reveal fine features in the cell edge dynamics.

Automated and semi-automated cell tracking: addressing portability challenges

Cell tracking is a key task in the high-throughput quantitative study of important biological processes, such as immune system regulation and neurogenesis. Variability in cell density and dynamics in different videos, hampers portability of existing trackers across videos. We address these potability challenges in order to develop a portable cell tracking algorithm. Our algorithm can handle noise in cell segmentation as well as divisions and deaths of cells. We also propose a parameter-free variation of our tracker. In the tracker, we employ a novel method for recovering the distribution of cell displacements. Further, we present a mathematically justified procedure for determining the gating distance in relation to tracking performance. For the range of real videos tested, our tracker correctly recovers on average 96% of cell moves, and outperforms an advanced probabilistic tracker when the cell detection quality is high. The scalability of our tracker was tested on synthetic videos with up to 200 cells per frame. For more challenging tracking conditions, we propose a novel semi-automated framework that can increase the ratio of correctly recovered tracks by 12%, through selective manual inspection of only 10% of all frames in a video.

A new method to address unmet needs for extracting individual cell migration features from a large number of cells embedded in 3D volumes

Background

In vitro cell observation has been widely used by biologists and pharmacologists for screening molecule-induced effects on cancer cells. Computer-assisted time-lapse microscopy enables automated live cell imaging in vitro, enabling cell behavior characterization through image analysis, in particular regarding cell migration. In this context, 3D cell assays in transparent matrix gels have been developed to provide more realistic in vitro 3D environments for monitoring cell migration (fundamentally different from cell motility behavior observed in 2D), which is related to the spread of cancer and metastases.

Methodology/Principal Findings

In this paper we propose an improved automated tracking method that is designed to robustly and individually follow a large number of unlabeled cells observed under phase-contrast microscopy in 3D gels. The method automatically detects and tracks individual cells across a sequence of acquired volumes, using a template matching filtering method that in turn allows for robust detection and mean-shift tracking. The robustness of the method results from detecting and managing the cases where two cell (mean-shift) trackers converge to the same point. The resulting trajectories quantify cell migration through statistical analysis of 3D trajectory descriptors. We manually validated the method and observed efficient cell detection and a low tracking error rate (6%). We also applied the method in a real biological experiment where the pro-migratory effects of hyaluronic acid (HA) were analyzed on brain cancer cells. Using collagen gels with increased HA proportions, we were able to evidence a dose-response effect on cell migration abilities.

Conclusions/Significance

The developed method enables biomedical researchers to automatically and robustly quantify the pro- or anti-migratory effects of different experimental conditions on unlabeled cell cultures in a 3D environment.

Chemical functionalization of polysilicon microparticles for single-cell studies

In this work, two types of polycrystalline silicon (polysilicon) microparticles were modified with specific ligands in order to be selectively attached to chemical residues located at the plasma membrane and thus to be applied to study individual cells in culture. Two different functionalization approaches based on adsorption and covalent attachment were assayed. A comparative study of the efficiency of the ligand immobilization and stability of the modified particle in the culture medium was carried out using the selected ligands labeled with a fluorophore. Cylindrical microparticles (nonencoded microparticles) and shape-encoded microparticles (bar codes) were used with the aim of demonstrating the nondependence of the particle size and shape on the efficiency of the immobilization protocol. Fluorescence imaging and statistical analysis of the recorded fluorescence intensity showed that the covalent attachment of the ligand to the surface of the microparticle, previously modified with an aldehyde-terminated silane, gave the best results. As a proof of concept, Vero cells in culture were labeled with the covalently modified bar codes and successfully tracked for up to 1 week without observing any alteration in the viability of the cells. Bar code numbers could be easily read by eye using a bright-field optical microscope. It is anticipated that such modified microparticles could be feasible platforms for the introduction of other analytical functions of interest in single-cell monitoring and cell sorting in automatic analysis systems.

3-D active meshes: fast discrete deformable models for cell tracking in 3-D time-lapse microscopy

Variational deformable models have proven over the past decades a high efficiency for segmentation and tracking in 2-D sequences. Yet, their application to 3-D time-lapse images has been hampered by discretization issues, heavy computational loads and lack of proper user visualization and interaction, limiting their use for routine analysis of large data-sets. We propose here to address these limitations by reformulating the problem entirely in the discrete domain using 3-D active meshes, which express a surface as a discrete triangular mesh, and minimize the energy functional accordingly. By performing computations in the discrete domain, computational costs are drastically reduced, whilst the mesh formalism allows to benefit from real-time 3-D rendering and other GPU-based optimizations. Performance evaluations on both simulated and real biological data sets show that this novel framework outperforms current state-of-the-art methods, constituting a light and fast alternative to traditional variational models for segmentation and tracking applications.

Quantitation of cellular dynamics in growing Arabidopsis roots with light sheet microscopy

To understand dynamic developmental processes, living tissues have to be imaged frequently and for extended periods of time. Root development is extensively studied at cellular resolution to understand basic mechanisms underlying pattern formation and maintenance in plants. Unfortunately, ensuring continuous specimen access, while preserving physiological conditions and preventing photo-damage, poses major barriers to measurements of cellular dynamics in growing organs such as plant roots. We present a system that integrates optical sectioning through light sheet fluorescence microscopy with hydroponic culture that enables us to image, at cellular resolution, a vertically growing Arabidopsis root every few minutes and for several consecutive days. We describe novel automated routines to track the root tip as it grows, to track cellular nuclei and to identify cell divisions. We demonstrate the system's capabilities by collecting data on divisions and nuclear dynamics.

Automated live cell imaging systems reveal dynamic cell behavior

Automated time-lapsed microscopy provides unique research opportunities to visualize cells and subcellular components in experiments with time-dependent parameters. As accessibility to these systems is increasing, we review here their use in cell science with a focus on stem cell research. Although the use of time-lapsed imaging to answer biological questions dates back nearly 150 years, only recently have the use of an environmentally controlled chamber and robotic stage controllers allowed for high-throughput continuous imaging over long periods at the cell and subcellular levels. Numerous automated imaging systems are now available from both companies that specialize in live cell imaging and from major microscope manufacturers. We discuss the key components of robots used for time-lapsed live microscopic imaging, and the unique data that can be obtained from image analysis. We show how automated features enhance experimentation by providing examples of uniquely quantified proliferation and migration live cell imaging data. In addition to providing an efficient system that drastically reduces man-hours and consumes fewer laboratory resources, this technology greatly enhances cell science by providing a unique dataset of temporal changes in cell activity.

Tracking colliding cells in vivo microscopy

Leukocyte motion represents an important component in the innate immune response to infection. Intravital microscopy is a powerful tool as it enables in vivo imaging of leukocyte motion. Under inflammatory conditions, leukocytes may exhibit various motion behaviors, such as flowing, rolling, and adhering. With many leukocytes moving at a wide range of speeds, collisions occur. These collisions result in abrupt changes in the motion and appearance of leukocytes. Manual analysis is tedious, error prone,time consuming, and could introduce technician-related bias. Automatic tracking is also challenging due to the noise inherent in in vivo images and abrupt changes in motion and appearance due to collision. This paper presents a method to automatically track multiple cells undergoing collisions by modeling the appearance and motion for each collision state and testing collision hypotheses of possible transitions between states. The tracking results are demonstrated using in vivo intravital microscopy image sequences.We demonstrate that 1)71% of colliding cells are correctly tracked; (2) the improvement of the proposed method is enhanced when the duration of collision increases; and (3) given good detection results, the proposed method can correctly track 88% of colliding cells. The method minimizes the tracking failures under collisions and, therefore, allows more robust analysis in the study of leukocyte behaviors responding to inflammatory conditions.

A global spatial similarity optimization scheme to track large numbers of dendritic spines in time-lapse confocal microscopy

Dendritic spines form postsynaptic contact sites in the central nervous system. The rapid and spontaneous morphology changes of spines have been widely observed by neurobiologists. Determining the relationship between dendritic spine morphology change and its functional properties such as memory learning is a fundamental yet challenging problem in neurobiology research. In this paper, we propose a novel algorithm to track the morphology change of multiple spines simultaneously in time-lapse neuronal images based on nonrigid registration and integer programming. We also propose a robust scheme to link disappearing-and-reappearing spines. Performance comparisons with other state-of-the-art cell and spine tracking algorithms, and the ground truth show that our approach is more accurate and robust, and it is capable of tracking a large number of neuronal spines in time-lapse confocal microscopy images.

Overlap-Based Cell Tracker

In order to facilitate the extraction of quantitative data from live cell image sets, automated image analysis methods are needed. This paper presents an introduction to the general principle of an overlap cell tracking software developed by the National Institute of Standards and Technology (NIST). This cell tracker has the ability to track cells across a set of time lapse images acquired at high rates based on the amount of overlap between cellular regions in consecutive frames. It is designed to be highly flexible, requires little user parameterization, and has a fast execution time.

An arrayed high-content chemotaxis assay for patient diagnosis

Chemotaxis assays are essential tools for the study of gradient sensing and directed cell migration, and have the potential to aid in the diagnosis and characterization of patients with immune disorders. Current methods are limited in their ability to meet the more demanding requirements for clinical applications. Because patient samples have a short lifespan and sometimes a limited volume (e.g. pediatrics), the operational requirements for an efficient chemotaxis assay are increased in the clinical setting. Here we describe a microscale assay platform for gradient generation that overcomes these limitations. Passive fluidic methods are leveraged to provide a reliable microfluidic gradient generation device, operable in only three pipetting steps. In addition, arrayed imaging and advanced cell tracking algorithms enabled a 50-fold increase in throughput over current methods. These methods were employed to aid in the diagnostic evaluation of an infant who presented with severe, recurrent bacterial infections. Analysis of the infant's neutrophils revealed impaired cell polarization and chemotaxis in a gradient of the chemoattractant fMLP. The patient was subsequently diagnosed with an inhibitory mutation in the Rho GTPase, Rac2. The approach also enabled a microenvironmental screen of human primary neutrophil chemotaxis on fibronectin, fibrinogen and laminin with results suggesting that fibronectin, although commonly used, may not be the most appropriate matrix protein for chemotaxis assays. Together, these findings demonstrate the use of arrayed micro-devices to aid in the diagnosis of a primary immunodeficiency disorder, and illustrate the capability for increased throughput microenvironmental studies and screening targeted to specific human diseases.

Tracking and quantitative analysis of dynamic movements of cells and particles

Understanding complex cellular processes requires investigating the underlying mechanisms within a spatiotemporal context. Although cellular processes are dynamic in nature, most studies in molecular cell biology are based on fixed specimens, for example, using immunocytochemistry or fluorescence in situ hybridization (FISH). However, breakthroughs in fluorescence microscopy imaging techniques, in particular, the discovery of green fluorescent protein (GFP) and its spectral variants, have facilitated the study of a wide range of dynamic processes by allowing nondestructive labeling of target structures in living cells. In addition, the tremendous improvements in spatial and temporal resolution of light microscopes now allow cellular processes to be analyzed in unprecedented detail. These state-of-the-art imaging technologies, however, provide a huge amount of digital image data. To cope with the enormous amount of image data and to extract reproducible as well as quantitative information, computer-based image analysis is required. In this article, we describe methods for computer-based analysis of multidimensional live cell microscopy images and their application to study the dynamics of cells and particles. First, we sketch a general workflow for quantitative analysis of live cell images. Then, we detail computational methods for automatic image analysis comprising image preprocessing, segmentation, registration, tracking, and classification. We conclude with a discussion of quantitative analysis and systems biology.

Automated analysis of time-lapse fluorescence microscopy images: from live cell images to intracellular foci

MOTIVATION

Complete, accurate and reproducible analysis of intracellular foci from fluorescence microscopy image sequences of live cells requires full automation of all processing steps involved: cell segmentation and tracking followed by foci segmentation and pattern analysis. Integrated systems for this purpose are lacking.

RESULTS

Extending our previous work in cell segmentation and tracking, we developed a new system for performing fully automated analysis of fluorescent foci in single cells. The system was validated by applying it to two common tasks: intracellular foci counting (in DNA damage repair experiments) and cell-phase identification based on foci pattern analysis (in DNA replication experiments). Experimental results show that the system performs comparably to expert human observers. Thus, it may replace tedious manual analyses for the considered tasks, and enables high-content screening.

AVAILABILITY AND IMPLEMENTATION

The described system was implemented in MATLAB (The MathWorks, Inc., USA) and compiled to run within the MATLAB environment. The routines together with four sample datasets are available at http://celmia.bigr.nl/. The software is planned for public release, free of charge for non-commercial use, after publication of this article.

Multi-environment model estimation for motility analysis of Caenorhabditis elegans

The nematode Caenorhabditis elegans is a well-known model organism used to investigate fundamental questions in biology. Motility assays of this small roundworm are designed to study the relationships between genes and behavior. Commonly, motility analysis is used to classify nematode movements and characterize them quantitatively. Over the past years, C. elegans' motility has been studied across a wide range of environments, including crawling on substrates, swimming in fluids, and locomoting through microfluidic substrates. However, each environment often requires customized image processing tools relying on heuristic parameter tuning. In the present study, we propose a novel Multi-Environment Model Estimation (MEME) framework for automated image segmentation that is versatile across various environments. The MEME platform is constructed around the concept of Mixture of Gaussian (MOG) models, where statistical models for both the background environment and the nematode appearance are explicitly learned and used to accurately segment a target nematode. Our method is designed to simplify the burden often imposed on users; here, only a single image which includes a nematode in its environment must be provided for model learning. In addition, our platform enables the extraction of nematode ‘skeletons’ for straightforward motility quantification. We test our algorithm on various locomotive environments and compare performances with an intensity-based thresholding method. Overall, MEME outperforms the threshold-based approach for the overwhelming majority of cases examined. Ultimately, MEME provides researchers with an attractive platform for C. elegans' segmentation and ‘skeletonizing’ across a wide range of motility assays.

Automated image analysis for high-content screening and analysis

The field of high-content screening and analysis consists of a set of methodologies for automated discovery in cell biology and drug development using large amounts of image data. In most cases, imaging is carried out by automated microscopes, often assisted by automated liquid handling and cell culture. Image processing, computer vision, and machine learning are used to automatically process high-dimensional image data into meaningful cell biological results. The key is creating automated analysis pipelines typically consisting of 4 basic steps: (1) image processing (normalization, segmentation, tracing, tracking), (2) spatial transformation to bring images to a common reference frame (registration), (3) computation of image features, and (4) machine learning for modeling and interpretation of data. An overview of these image analysis tools is presented here, along with brief descriptions of a few applications.

Live-cell tracking using SIFT features in DIC microscopic videos

In this paper, a novel motion-tracking scheme using scale-invariant features is proposed for automatic cell motility analysis in gray-scale microscopic videos, particularly for the live-cell tracking in low-contrast differential interference contrast (DIC) microscopy. In the proposed approach, scale-invariant feature transform (SIFT) points around live cells in the microscopic image are detected, and a structure locality preservation (SLP) scheme using Laplacian Eigenmap is proposed to track the SIFT feature points along successive frames of low-contrast DIC videos. Experiments on low-contrast DIC microscopic videos of various live-cell lines shows that in comparison with principal component analysis (PCA) based SIFT tracking, the proposed Laplacian-SIFT can significantly reduce the error rate of SIFT feature tracking. With this enhancement, further experimental results demonstrate that the proposed scheme is a robust and accurate approach to tackling the challenge of live-cell tracking in DIC microscopy.

Multilevel space-time aggregation for bright field cell microscopy segmentation and tracking

A multilevel aggregation method is applied to the problem of segmenting live cell bright field microscope images. The method employed is a variant of the so-called “Segmentation by Weighted Aggregation” technique, which itself is based on Algebraic Multigrid methods. The variant of the method used is described in detail, and it is explained how it is tailored to the application at hand. In particular, a new scale-invariant “saliency measure” is proposed for deciding when aggregates of pixels constitute salient segments that should not be grouped further. It is shown how segmentation based on multilevel intensity similarity alone does not lead to satisfactory results for bright field cells. However, the addition of multilevel intensity variance (as a measure of texture) to the feature vector of each aggregate leads to correct cell segmentation. Preliminary results are presented for applying the multilevel aggregation algorithm in space time to temporal sequences of microscope images, with the goal of obtaining space-time segments (“object tunnels”) that track individual cells. The advantages and drawbacks of the space-time aggregation approach for segmentation and tracking of live cells in sequences of bright field microscope images are presented, along with a discussion on how this approach may be used in the future work as a building block in a complete and robust segmentation and tracking system.

Cell segmentation in time-lapse fluorescence microscopy with temporally varying sub-cellular fusion protein patterns

Fluorescently tagged proteins such as GFP-PCNA produce rich dynamically varying textural patterns of foci distributed in the nucleus. This enables the behavioral study of sub-cellular structures during different phases of the cell cycle. The varying punctuate patterns of fluorescence, drastic changes in SNR, shape and position during mitosis and abundance of touching cells, however, require more sophisticated algorithms for reliable automatic cell segmentation and lineage analysis. Since the cell nuclei are non-uniform in appearance, a distribution-based modeling of foreground classes is essential. The recently proposed graph partitioning active contours (GPAC) algorithm supports region descriptors and flexible distance metrics. We extend GPAC for fluorescence-based cell segmentation using regional density functions and dramatically improve its efficiency for segmentation from O(N(4)) to O(N(2)), for an image with N(2) pixels, making it practical and scalable for high throughput microscopy imaging studies.

Quantification of fluorescence intensity of labeled human mesenchymal stem cells and cell counting of unlabeled cells in phase-contrast imaging: an open-source-based algorithm

Assessment of cell fate is indispensable to evaluate cell-based therapies in regenerative medicine. Therefore, a widely used technique is fluorescence labeling. A major problem still is the standardized, noninvasive, and reliable quantification of fluorescence intensity of adherent cell populations on single-cell level, since total fluorescence intensity must be correlated to the cell number. Consequently, the aim of the present study was to produce and validate an open-source-based algorithm, capable of measuring the total fluorescence intensity of cell populations and assessing the total cell number in phase-contrast images. To verify the algorithms' capacity to assess fluorescence intensity, human mesenchymal stem cells were transduced to stably express enhanced green fluorescent protein and results produced by the algorithm were compared to flow cytometry analysis. No significant differences could be observed at any time (p ≥ 0.443). For validation of the algorithm for cell counting in phase-contrast images, adherent human mesenchymal stem cells were manually counted and compared to results produced by the algorithm (correlation coefficient [CC] r = 0.975), nuclei staining (CC r = 0.997), and hemocytometer (CC r = 0.629). We conclude that applying the developed algorithm in routine practice allows robust, fast, and reproducible assessment of fluorescence intensity and cell numbers in simple large-scale microscopy. The method is easy to perform and open source based.

Significantly improved precision of cell migration analysis in time-lapse video microscopy through use of a fully automated tracking system

Background

Cell motility is a critical parameter in many physiological as well as pathophysiological processes. In time-lapse video microscopy, manual cell tracking remains the most common method of analyzing migratory behavior of cell populations. In addition to being labor-intensive, this method is susceptible to user-dependent errors regarding the selection of "representative" subsets of cells and manual determination of precise cell positions.

Results

We have quantitatively analyzed these error sources, demonstrating that manual cell tracking of pancreatic cancer cells lead to mis-calculation of migration rates of up to 410%. In order to provide for objective measurements of cell migration rates, we have employed multi-target tracking technologies commonly used in radar applications to develop fully automated cell identification and tracking system suitable for high throughput screening of video sequences of unstained living cells.

Conclusion

We demonstrate that our automatic multi target tracking system identifies cell objects, follows individual cells and computes migration rates with high precision, clearly outperforming manual procedures.

Advanced level-set-based cell tracking in time-lapse fluorescence microscopy

Cell segmentation and tracking in time-lapse fluorescence microscopy images is a task of fundamental importance in many biological studies on cell migration and proliferation. In recent years, level sets have been shown to provide a very appropriate framework for this purpose, as they are well suited to capture topological changes occurring during mitosis, and they easily extend to higher dimensional image data. This model evolution approach has also been extended to deal with many cells concurrently. Notwithstanding its high potential, the multiple-level-set method suffers from a number of shortcomings, which limit its applicability to a larger variety of cell biological imaging studies. In this paper, we propose several modifications and extensions to the coupled-active-surfaces algorithm, which considerably improve its robustness and applicability. Our algorithm was validated by comparing it to the original algorithm and two other cell segmentation algorithms. For the evaluation, four real fluorescence microscopy image datasets were used, involving different cell types and labelings that are representative of a large range of biological experiments. Improved tracking performance in terms of precision (up to 11%), recall (up to 8%), ability to correctly capture all cell division events, and computation time (up to nine times reduction) is achieved.