A Monte Carlo approach to rolling leukocyte tracking in vivo

Tracking Neutrophil Migration in Zebrafish Model Using Multi-Channel Feature Learning

Tracking cells over time is crucial in the fields of computer vision and biomedical science. Studying neutrophils and their migratory profile is the highly topical fields in inflammation research due to determining role of these cells during immune responses. As neutrophils generally are of various shapes and motion, it remains challenging to track and describe their behaviours from multi-dimensional microscopy datasets. In this study, we propose a robust novel multi-channel feature learning (MCFL) model inspired by deep learning to extract the complex behaviour of neutrophils moved in time lapse images. In this model, the convolutional neural networks along with cell relocation distance and orientation channels learn the robust significant spatial and temporal features of an individual neutrophil. Additionally, we also proposed a new cell tracking framework to detect and track neutrophils in the original time-laps microscopy images, entails sampling, observation, and visualisation functions. Our proposed cell tracking-based-multi channel feature learning method has remarkable performance in rectifying common cell tracking problem compared with state-of the-art methods.

Detecting cells in intravital video microscopy using a deep convolutional neural network

The analysis of leukocyte recruitment in intravital video microscopy (IVM) is essential to the understanding of inflammatory processes. However, because IVM images often present a large variety of visual characteristics, it is hard for an expert human or even conventional machine learning techniques to detect and count the massive amount of cells and extract statistical measures precisely. Convolutional neural networks are a promising approach to overcome this problem, but due to the difficulty of labeling cells, large data sets with ground truth are rare. The present work explores an adaptation of the RetinaNet model with a suite of augmentation techniques and transfer learning for detecting leukocytes in IVM data. The augmentation techniques include simulating the Airy pattern and motion artifacts present in microscopy imaging, followed by traditional photometric, geometric and smooth elastic transformations to reproduce color and shape changes in cells. In addition, we analyzed the use of different network backbones, feature pyramid levels, and image input scales. We have found that even with limited data, our strategy not only enables training without overfitting but also boosts generalization performance. Among several experiments, the model reached a value of 94.84 for the average precision (AP) metric as our best outcome when using data from different image modalities. We also compared our results with conventional image processing techniques and open-source tools. The results showed an outstanding precision of the method compared with other approaches, presenting low error rates for cell counting and centroid distances. Code is available at: https://github.com/brunoggregorio/retinanet-cell-detection.

SITUP: Scale Invariant Tracking using Average Peak-to-Correlation Energy

Robust and accurate scale estimation of a target object is a challenging task in visual object tracking. Most existing tracking methods cannot accommodate large scale variation in complex image sequences and thus result in inferior performance. In this paper, we propose to incorporate a novel criterion called the average peak-to-correlation energy into the multi-resolution translation filter framework to obtain robust and accurate scale estimation. The resulting system is named SITUP: Scale Invariant Tracking using Average Peak-to-Correlation Energy. SITUP effectively tackles the problem of fixed template size in standard discriminative correlation filter based trackers. Extensive empirical evaluation on the publicly available tracking benchmark datasets demonstrates that the proposed scale searching framework meets the demands of scale variation challenges effectively while providing superior performance over other scale adaptive variants of standard discriminative correlation filter based trackers. Also, SITUP obtains favorable performance compared to state-of-the-art trackers for various scenarios while operating in real-time on a single CPU.

An extended Kalman filter for mouse tracking

Animal tracking is an important tool for observing behavior, which is useful in various research areas. Animal specimens can be tracked using dynamic models and observation models that require several types of data. Tracking mouse has several barriers due to the physical characteristics of the mouse, their unpredictable movement, and cluttered environments. Therefore, we propose a reliable method that uses a detection stage and a tracking stage to successfully track mouse. The detection stage detects the surface area of the mouse skin, and the tracking stage implements an extended Kalman filter to estimate the state variables of a nonlinear model. The changes in the overall shape of the mouse are tracked using an oval-shaped tracking model to estimate the parameters for the ellipse. An experiment is conducted to demonstrate the performance of the proposed tracking algorithm using six video images showing various types of movement, and the ground truth values for synthetic images are compared to the values generated by the tracking algorithm. A conventional manual tracking method is also applied to compare across eight experimenters. Furthermore, the effectiveness of the proposed tracking method is also demonstrated by applying the tracking algorithm with actual images of mouse. Graphical abstract.

CASA in the medical laboratory: CASA in diagnostic andrology and assisted conception

CASA has been used in reproductive medicine and pathology laboratories for over 25 years, yet the ‘fertility industry’ generally remains sceptical and has avoided automation, despite clear weaknesses in manual semen analysis. Early implementers had difficulty in validating CASA-Mot instruments against recommended manual methods (haemocytometer) due to the interference of seminal debris and non-sperm cells, which also affects the accuracy of grading motility. Both the inability to provide accurate sperm counts and a lack of consensus as to the value of sperm kinematic parameters appear to have continued to have a negative effect on CASA-Mot’s reputation. One positive interpretation from earlier work is that at least one or more measures of sperm velocity adds clinical value to the semen analysis, and these are clearly more objective than any manual motility analysis. Moreover, recent CASA-Mot systems offer simple solutions to earlier problems in eliminating artefacts and have been successfully validated for sperm concentration; as a result, they should be viewed with more confidence in relation to motility grading. Sperm morphology and DNA testing both require an evidence-based consensus and a well-validated (reliable, reproducible) assay to be developed before automation of either can be of real clinical benefit.

A Novel Multiobject Tracking Approach in the Presence of Collision and Division

This paper aims to develop a general framework for accurately tracking and quantitatively characterizing multiple cells (objects) when collision and division between cells arise. Through introducing three types of interaction events among cells, namely, independence, collision, and division, the corresponding dynamic models are defined and an augmented interacting multiple model particle filter tracking algorithm is first proposed for spatially adjacent cells with varying size. In addition, to reduce the ambiguity of correspondence between frames, both the estimated cell dynamic parameters and cell size are further utilized to identify cells of interest. The experiments have been conducted on two real cell image sequences characterized with cells collision, division, or number variation, and the resulting dynamic parameters such as instant velocity, turn rate were obtained and analyzed.

Tracking multiple particles in fluorescence time-lapse microscopy images via probabilistic data association

Tracking subcellular structures as well as viral structures displayed as 'particles' in fluorescence microscopy images yields quantitative information on the underlying dynamical processes. We have developed an approach for tracking multiple fluorescent particles based on probabilistic data association. The approach combines a localization scheme that uses a bottom-up strategy based on the spot-enhancing filter as well as a top-down strategy based on an ellipsoidal sampling scheme that uses the Gaussian probability distributions computed by a Kalman filter. The localization scheme yields multiple measurements that are incorporated into the Kalman filter via a combined innovation, where the association probabilities are interpreted as weights calculated using an image likelihood. To track objects in close proximity, we compute the support of each image position relative to the neighboring objects of a tracked object and use this support to recalculate the weights. To cope with multiple motion models, we integrated the interacting multiple model algorithm. The approach has been successfully applied to synthetic 2-D and 3-D images as well as to real 2-D and 3-D microscopy images, and the performance has been quantified. In addition, the approach was successfully applied to the 2-D and 3-D image data of the recent Particle Tracking Challenge at the IEEE International Symposium on Biomedical Imaging (ISBI) 2012.

PhagoSight: an open-source MATLAB® package for the analysis of fluorescent neutrophil and macrophage migration in a zebrafish model

Neutrophil migration in zebrafish larvae is increasingly used as a model to study the response of these leukocytes to different determinants of the cellular inflammatory response. However, it remains challenging to extract comprehensive information describing the behaviour of neutrophils from the multi-dimensional data sets acquired with widefield or confocal microscopes. Here, we describe PhagoSight, an open-source software package for the segmentation, tracking and visualisation of migrating phagocytes in three dimensions. The algorithms in PhagoSight extract a large number of measurements that summarise the behaviour of neutrophils, but that could potentially be applied to any moving fluorescent cells. To derive a useful panel of variables quantifying aspects of neutrophil migratory behaviour, and to demonstrate the utility of PhagoSight, we evaluated changes in the volume of migrating neutrophils. Cell volume increased as neutrophils migrated towards the wound region of injured zebrafish. PhagoSight is openly available as MATLAB® m-files under the GNU General Public License. Synthetic data sets and a comprehensive user manual are available from http://www.phagosight.org.

Tracking colliding cells in vivo microscopy

Leukocyte motion represents an important component in the innate immune response to infection. Intravital microscopy is a powerful tool as it enables in vivo imaging of leukocyte motion. Under inflammatory conditions, leukocytes may exhibit various motion behaviors, such as flowing, rolling, and adhering. With many leukocytes moving at a wide range of speeds, collisions occur. These collisions result in abrupt changes in the motion and appearance of leukocytes. Manual analysis is tedious, error prone,time consuming, and could introduce technician-related bias. Automatic tracking is also challenging due to the noise inherent in in vivo images and abrupt changes in motion and appearance due to collision. This paper presents a method to automatically track multiple cells undergoing collisions by modeling the appearance and motion for each collision state and testing collision hypotheses of possible transitions between states. The tracking results are demonstrated using in vivo intravital microscopy image sequences.We demonstrate that 1)71% of colliding cells are correctly tracked; (2) the improvement of the proposed method is enhanced when the duration of collision increases; and (3) given good detection results, the proposed method can correctly track 88% of colliding cells. The method minimizes the tracking failures under collisions and, therefore, allows more robust analysis in the study of leukocyte behaviors responding to inflammatory conditions.

Tracking in cell and developmental biology

The past decade has seen an unprecedented data explosion in biology. It has become evident that in order to take full advantage of the potential wealth of information hidden in the data produced by even a single experiment, visual inspection and manual analysis are no longer adequate. To ensure efficiency, consistency, and completeness in data processing and analysis, computational tools are essential. Of particular importance to many modern live-cell imaging experiments is the ability to automatically track and analyze the motion of objects in time-lapse microscopy images. This article surveys the recent literature in this area. Covering all scales of microscopic observation, from cells, down to molecules, and up to entire organisms, it discusses the latest trends and successes in the development and application of computerized tracking methods in cell and developmental biology.

Deterministic and probabilistic approaches for tracking virus particles in time-lapse fluorescence microscopy image sequences

Modern developments in time-lapse fluorescence microscopy enable the observation of a variety of processes exhibited by viruses. The dynamic nature of these processes requires the tracking of viruses over time to explore spatial-temporal relationships. In this work, we developed deterministic and probabilistic approaches for multiple virus tracking in multi-channel fluorescence microscopy images. The deterministic approaches follow a traditional two-step paradigm comprising particle localization based on either the spot-enhancing filter or 2D Gaussian fitting, as well as motion correspondence based on a global nearest neighbor scheme. Our probabilistic approaches are based on particle filters. We describe approaches based on a mixture of particle filters and based on independent particle filters. For the latter, we have developed a penalization strategy that prevents the problem of filter coalescence (merging) in cases where objects lie in close proximity. A quantitative comparison based on synthetic image sequences is carried out to evaluate the performance of our approaches. In total, eight different tracking approaches have been evaluated. We have also applied these approaches to real microscopy images of HIV-1 particles and have compared the tracking results with ground truth obtained from manual tracking. It turns out that the probabilistic approaches based on independent particle filters are superior to the deterministic schemes as well as to the approaches based on a mixture of particle filters.

An affine transformation invariance approach to cell tracking

Accurate and robust methods for automatically tracking rolling leukocytes facilitate inflammation research as leukocyte motion is a primary indicator of inflammatory response in the microvasculature. This paper reports on an affine transformation invariance approach we proposed to track rolling leukocyte in intravital microscopy image sequences. The method is based on the affine transformation invariance property, which enables the accommodation of linear affine transformations (translation, rotation, and/or scaling) of the target, and a particle filter that overcomes the effect of image clutter. In our data set of 50 sequences, we compared the new approach with an active contour tracking method and a Monte Carlo tracker. With the manual tracking result provided by an operator as the reference, the root mean square errors for the active contour tracking method, the Monte Carlo tracker and the affine transformation invariance approach were 0.95 microm, 0.79 microm and 0.74 microm, respectively, and the percentage of frames tracked were 72%, 75% and 89%, respectively. The affine transformation invariance approach demonstrated more robust (being able to successfully locate target leukocyte in more frames) and more accurate (lower root mean square error) tracking performance. We also separately studied the ability of the affine transformation invariance approach to track a dark target leukocyte and a bright target leukocyte by using the number of frames tracked as an evaluation measure. Dark target leukocyte possesses similar image intensity to the background, making it difficult to be located. In 20 sequences where the target leukocyte was dark, the affine transformation invariance approach tracked more frames in 18 sequences and fewer frames in 2 sequences when compared with the active contour tracking method. In comparison with the Monte Carlo tracker, the affine invariance method tracked more frames in 9 sequences, the same number of frames in 7 sequences and fewer frames in 4 sequences. In tracking a bright target leukocyte in 30 sequences, the affine transformation invariance approach demonstrated superior performance in 7 sequences and inferior performance in 1 sequence when compared with the active contour tracking method. It outperformed the Monte Carlo tracker in 15 sequences and underperformed in 1 sequence.

Measuring the velocity of fluorescently labelled red blood cells with a keyhole tracking algorithm

In this paper we propose a tracking algorithm to measure the velocity of fluorescently labelled red blood cells travelling through microvessels of tumours, growing in dorsal skin flap window chambers, implanted on mice. Preprocessing removed noise and artefacts from the images and then segmented cells from background. The tracking algorithm is based on a 'keyhole' model that describes the probable movement of a segmented cell between contiguous frames of a video sequence. When a history of cell movement exists, past, present and a predicted landing position of the cells will define two regions of probability that resemble the shape of a keyhole. This keyhole model was used to determine if cells in contiguous frames should be linked to form tracks and also as a postprocessing tool to join split tracks and discard links that could have been formed due to noise or uncertainty. When there was no history, a circular region around the centroid of the parent cell was used as a region of probability. Outliers were removed based on the distribution of the average velocities of the tracks. Since the position and time of each cell is recorded, a wealth of statistical measures can be obtained from the tracks. The algorithm was tested on two sets of experiments. First, the vasculatures of eight tumours with different geometries were analyzed; average velocities ranged from 86 to 372 microm s(-1), with minimum and maximum track velocities 7 and 1212 microm s(-1), respectively. Second, a longitudinal study of velocities was performed after administering a vascular disrupting agent to two tumours and the time behaviour was analyzed over 24 h. In one of the tumours there is a complete shutdown of the vasculature whereas in the other there is a clear decrease of velocity at 30 min, with subsequent recovery by 6 h. The tracking algorithm enabled the simultaneous measurement of red blood cell velocity in multiple vessels within an intravital video sequence, enabling analysis of heterogeneity of flow and response to treatment in mouse models of cancer.