Automatic extraction and measurement of leukocyte motion in microvessels using spatiotemporal image analysis

ASIST: Annotation-free synthetic instance segmentation and tracking by adversarial simulations

BACKGROUND

The quantitative analysis of microscope videos often requires instance segmentation and tracking of cellular and subcellular objects. The traditional method consists of two stages: (1) performing instance object segmentation of each frame, and (2) associating objects frame-by-frame. Recently, pixel-embedding-based deep learning approaches these two steps simultaneously as a single stage holistic solution. Pixel-embedding-based learning forces similar feature representation of pixels from the same object, while maximizing the difference of feature representations from different objects. However, such deep learning methods require consistent annotations not only spatially (for segmentation), but also temporally (for tracking). In computer vision, annotated training data with consistent segmentation and tracking is resource intensive, the severity of which is multiplied in microscopy imaging due to (1) dense objects (e.g., overlapping or touching), and (2) high dynamics (e.g., irregular motion and mitosis). Adversarial simulations have provided successful solutions to alleviate the lack of such annotations in dynamics scenes in computer vision, such as using simulated environments (e.g., computer games) to train real-world self-driving systems.

METHODS

In this paper, we propose an annotation-free synthetic instance segmentation and tracking (ASIST) method with adversarial simulation and single-stage pixel-embedding based learning.

CONTRIBUTION

The contribution of this paper is three-fold: (1) the proposed method aggregates adversarial simulations and single-stage pixel-embedding based deep learning (2) the method is assessed with both the cellular (i.e., HeLa cells); and subcellular (i.e., microvilli) objects; and (3) to the best of our knowledge, this is the first study to explore annotation-free instance segmentation and tracking study for microscope videos.

RESULTS

The ASIST method achieved an important step forward, when compared with fully supervised approaches: ASIST shows 7%-11% higher segmentation, detection and tracking performance on microvilli relative to fully supervised methods, and comparable performance on Hela cell videos.

Quantitative nailfold capillaroscopy-update and possible next steps

We review the exciting potential (and challenges) of quantitative nailfold capillaroscopy, focusing on its role in systemic sclerosis. Quantifying abnormality, including automated analysis of nailfold images, overcomes the subjectivity of qualitative/descriptive image interpretation. First we consider the rationale for quantitative analysis, including the potential for precise discrimination between normal and abnormal capillaries and for reliable measurement of disease progression and treatment response. We discuss nailfold image acquisition and interpretation, and describe how early work on semi-quantitative and quantitative analysis paved the way for semi-automated and automated analysis. Measurement of red blood cell velocity is described briefly. Finally we give a personal view on 'next steps'. From a clinical perspective, increased uptake of nailfold capillaroscopy by general rheumatologists could be achieved via low-cost hand-held devices with cloud-based automated analysis. From a research perspective, automated analysis could facilitate large-scale prospective studies using capillaroscopic parameters as possible biomarkers of systemic sclerosis-spectrum disorders.

In-vivo full-field measurement of microcirculatory blood flow velocity based on intelligent object identification

Microcirculation plays a crucial role in delivering oxygen and nutrients to living tissues and in removing metabolic wastes from the human body. Monitoring the velocity of blood flow in microcirculation is essential for assessing various diseases, such as diabetes, cancer, and critical illnesses. Because of the complex morphological pattern of the capillaries, both In-vivo capillary identification and blood flow velocity measurement by conventional optical capillaroscopy are challenging. Thus, we focused on developing an In-vivo optical microscope for capillary imaging, and we propose an In-vivo full-field flow velocity measurement method based on intelligent object identification. The proposed method realizes full-field blood flow velocity measurements in microcirculation by employing a deep neural network to automatically identify and distinguish capillaries from images. In addition, a spatiotemporal diagram analysis is used for flow velocity calculation. In-vivo experiments were conducted, and the images and videos of capillaries were collected for analysis. We demonstrated that the proposed method is highly accurate in performing full-field blood flow velocity measurements in microcirculation. Further, because this method is simple and inexpensive, it can be effectively employed in clinics.

Quantitative Analysis of Intracellular Motility Based on Optical Flow Model

Analysis of cell mobility is a key issue for abnormality identification and classification in cell biology research. However, since cell deformation induced by various biological processes is random and cell protrusion is irregular, it is difficult to measure cell morphology and motility in microscopic images. To address this dilemma, we propose an improved variation optical flow model for quantitative analysis of intracellular motility, which not only extracts intracellular motion fields effectively but also deals with optical flow computation problem at the border by taking advantages of the formulation based on L₁ and L₂ norm, respectively. In the energy functional of our proposed optical flow model, the data term is in the form of L₂ norm; the smoothness of the data changes with regional features through an adaptive parameter, using L₁ norm near the edge of the cell and L₂ norm away from the edge. We further extract histograms of oriented optical flow (HOOF) after optical flow field of intracellular motion is computed. Then distances of different HOOFs are calculated as the intracellular motion features to grade the intracellular motion. Experimental results show that the features extracted from HOOFs provide new insights into the relationship between the cell motility and the special pathological conditions.

Parafoveal retinal vascular response to pattern visual stimulation assessed with OCT angiography

We used optical coherence tomography (OCT) angiography with a high-speed swept-source OCT system to investigate retinal blood flow changes induced by visual stimulation with a reversing checkerboard pattern. The split-spectrum amplitude-decorrelation angiography (SSADA) algorithm was used to quantify blood flow as measured with parafoveal flow index (PFI), which is proportional to the density of blood vessels and the velocity of blood flow in the parafoveal region of the macula. PFI measurements were taken in 15 second intervals during a 4 minute period consisting of 1 minute of baseline, 2 minutes with an 8 Hz reversing checkerboard pattern stimulation, and 1 minute without stimulation. PFI measurements increased 6.1±4.7% (p = .001) during the first minute of stimulation, with the most significant increase in PFI occurring 30 seconds into stimulation (p<0.001). These results suggest that pattern stimulation induces a change to retinal blood flow that can be reliably measured with OCT angiography.

Quantitative analysis of lymphocytes morphology and motion in intravital microscopic images

Studying the morphology and interior movement of lymphocytes in intravital microscopic images is essential to understanding and treating various biological processes and pathological situations. A method combing features of shape, deformation, and intracellular motion for quantitatively characterizing the dynamic behavior of a single lymphocyte is proposed in this paper. The method is tested on a set of image sequences of lymphocytes obtained from the peripheral blood of mice undergoing skin transplantation using a phase contrast microscope. Experimental results coincide with the clinical observation and pathological analysis, demonstrating that the extracted cell morphology and motion features can provide new insights into the relationship between the dynamic behavior of lymphocytes and the occurrence of graft rejection.

High-resolution wide-field imaging of perfused capillaries without the use of contrast agent

Purpose:

Assessment of capillary abnormalities facilitates early diagnosis, treatment, and follow-up of common retinal pathologies. Injected contrast agents like fluorescein are widely used to image retinal capillaries, but this highly effective procedure has a few disadvantages, such as untoward side effects, inconvenience of injection, and brevity of the time window for clear visualization. The retinal function imager (RFI) is a tool for monitoring retinal functions, such as blood velocity and oximetry, based on intrinsic signals. Here we describe the clinical use of hemoglobin in red blood cells (RBCs) as an intrinsic motion-contrast agent in the generation of detailed noninvasive capillary-perfusion maps (nCPMs).

Patients and methods:

Multiple series of nCPM images were acquired from 130 patients with diabetic retinopathy, vein occlusion, central serous retinopathy, age-related macular degeneration, or metabolic syndrome, as well as from 37 healthy subjects. After registration, pixel value distribution parameters were analyzed to locate RBC motion.

Results:

The RFI yielded nCPMs demonstrating microvascular morphology including capillaries in exquisite detail. Maps from the same subject were highly reproducible in repeated measurements, in as much detail and often better than that revealed by the very best fluorescein angiography. In patients, neovascularization and capillary nonperfusion areas were clearly observed. Foveal avascular zones (FAZ) were sharply delineated and were larger in patients with diabetic retinopathy than in controls (FAZ diameter: 641.5 ± 82.3 versus 463.7 ± 105 μm; P < 0.001). Also visible were abnormal vascular patterns, such as shunts and vascular loops.

Conclusion:

Optical imaging of retinal capillaries in human patients based on motion contrast is noninvasive, comfortable, safe, and can be repeated as often as required for early diagnosis, treatment guidance, and follow up of retinal disease progression.

Speed quantification and tracking of moving objects in adaptive optics scanning laser ophthalmoscopy

Microscopic features of the human retina can be resolved noninvasively using an adaptive optics scanning laser ophthalmoscope (AOSLO). We describe an improved method to track and quantify the speed of moving objects in AOSLO videos, which is necessary for characterizing the hemodynamics of retinal capillaries. During video acquisition, the objects of interest are in constant motion relative to the background tissue (object motion). The background tissue is in constant motion relative to the AOSLO, due to continuous eye motion during video recordings (eye motion). The location at which AOSLO acquires data is also in continuous motion, since the imaging source is swept in a raster scan across the retina (raster scanning). We show that it is important to take into consideration the combination of object motion, eye motion, and raster scanning for accurate quantification of object speeds. The proposed methods performed well on both experimental AOSLO videos as well as synthetic videos generated by a virtual AOSLO. These methods improve the accuracy of methods to investigate hemodynamics using AOSLO imaging.

Microtubule dynamics analysis using kymographs and variable-rate particle filters

Studying intracellular dynamics is of fundamental importance for understanding healthy life at the molecular level and for developing drugs to target disease processes. One of the key technologies to enable this research is the automated tracking and motion analysis of these objects in microscopy image sequences. To make better use of the spatiotemporal information than common frame-by-frame tracking methods, two alternative approaches have recently been proposed, based upon either Bayesian estimation or space-time segmentation. In this paper, we propose to combine the power of both approaches, and develop a new probabilistic method to segment the traces of the moving objects in kymograph representations of the image data. It is based on variable-rate particle filtering and uses multiscale trend analysis of the extracted traces to estimate the relevant kinematic parameters. Experiments on realistic synthetically generated images as well as on real biological image data demonstrate the improved potential of the new method for the analysis of microtubule dynamics in vitro.

Live-cell tracking using SIFT features in DIC microscopic videos

In this paper, a novel motion-tracking scheme using scale-invariant features is proposed for automatic cell motility analysis in gray-scale microscopic videos, particularly for the live-cell tracking in low-contrast differential interference contrast (DIC) microscopy. In the proposed approach, scale-invariant feature transform (SIFT) points around live cells in the microscopic image are detected, and a structure locality preservation (SLP) scheme using Laplacian Eigenmap is proposed to track the SIFT feature points along successive frames of low-contrast DIC videos. Experiments on low-contrast DIC microscopic videos of various live-cell lines shows that in comparison with principal component analysis (PCA) based SIFT tracking, the proposed Laplacian-SIFT can significantly reduce the error rate of SIFT feature tracking. With this enhancement, further experimental results demonstrate that the proposed scheme is a robust and accurate approach to tackling the challenge of live-cell tracking in DIC microscopy.

Noninvasive visualization and analysis of parafoveal capillaries in humans

PURPOSE

To demonstrate a noninvasive method to visualize and analyze the parafoveal capillary network in humans.

METHODS

An adaptive optics scanning laser ophthalmoscope was used to acquire high resolution retinal videos on human subjects. Video processing tools that enhance motion contrast were developed and applied to the videos to generate montages of parafoveal retinal capillaries. The capillary network and foveal avascular zone (FAZ) were extracted using video and image analysis algorithms. The capillary densities in the zone immediately outside the FAZ were calculated and the variation in density as a function of direction was investigated. Extracted FAZ geometries were used to calculate area and effective diameters. The authors also compared their method against fluorescein angiography (FA) for one subject.

RESULTS

The parafoveal capillaries were clearly visible when the motion contrast in noninvasive videos was enhanced. There was a marked improvement in the contrast of the parafoveal capillaries when compared to the unprocessed videos. The average FAZ area was 0.323 mm(2), with an average effective diameter of 633 microm. There was no variation in capillary density near the FAZ in different directions.

CONCLUSIONS

Using motion cues to enhance vessel contrast is a powerful tool for visualizing the capillary network, in the absence of contrast agents. The authors demonstrate a tool to study the microcirculation of healthy subjects noninvasively.

An affine transformation invariance approach to cell tracking

Accurate and robust methods for automatically tracking rolling leukocytes facilitate inflammation research as leukocyte motion is a primary indicator of inflammatory response in the microvasculature. This paper reports on an affine transformation invariance approach we proposed to track rolling leukocyte in intravital microscopy image sequences. The method is based on the affine transformation invariance property, which enables the accommodation of linear affine transformations (translation, rotation, and/or scaling) of the target, and a particle filter that overcomes the effect of image clutter. In our data set of 50 sequences, we compared the new approach with an active contour tracking method and a Monte Carlo tracker. With the manual tracking result provided by an operator as the reference, the root mean square errors for the active contour tracking method, the Monte Carlo tracker and the affine transformation invariance approach were 0.95 microm, 0.79 microm and 0.74 microm, respectively, and the percentage of frames tracked were 72%, 75% and 89%, respectively. The affine transformation invariance approach demonstrated more robust (being able to successfully locate target leukocyte in more frames) and more accurate (lower root mean square error) tracking performance. We also separately studied the ability of the affine transformation invariance approach to track a dark target leukocyte and a bright target leukocyte by using the number of frames tracked as an evaluation measure. Dark target leukocyte possesses similar image intensity to the background, making it difficult to be located. In 20 sequences where the target leukocyte was dark, the affine transformation invariance approach tracked more frames in 18 sequences and fewer frames in 2 sequences when compared with the active contour tracking method. In comparison with the Monte Carlo tracker, the affine invariance method tracked more frames in 9 sequences, the same number of frames in 7 sequences and fewer frames in 4 sequences. In tracking a bright target leukocyte in 30 sequences, the affine transformation invariance approach demonstrated superior performance in 7 sequences and inferior performance in 1 sequence when compared with the active contour tracking method. It outperformed the Monte Carlo tracker in 15 sequences and underperformed in 1 sequence.

Spatio-temporal cell cycle phase analysis using level sets and fast marching methods

Enabled by novel molecular markers, fluorescence microscopy enables the monitoring of multiple cellular functions using live cell assays. Automated image analysis is necessary to monitor such model systems in a high-throughput and high-content environment. Here, we demonstrate the ability to simultaneously track cell cycle phase and cell motion at the single cell level. Using a recently introduced cell cycle marker, we present a set of image analysis tools for automated cell phase analysis of live cells over extended time periods. Our model-based approach enables the characterization of the four phases of the cell cycle G1, S, G2, and M, which enables the study of the effect of inhibitor compounds that are designed to block the replication of cancerous cells in any of the phases. We approach the tracking problem as a spatio-temporal volume segmentation task, where the 2D slices are stacked into a volume with time as the z dimension. The segmentation of the G2 and S phases is accomplished using level sets, and we designed a model-based shape/size constraint to control the evolution of the level set. Our main contribution is the design of a speed function coupled with a fast marching path planning approach for tracking cells across the G1 phase based on the appearance change of the nuclei. The viability of our approach is demonstrated by presenting quantitative results on both controls and cases in which cells are treated with a cell cycle inhibitor.

Liver microcirculation analysis by red blood cell motion modeling in intravital microscopy images

Intravital microscopy has been used to visualize the microcirculation by imaging fluorescent labeled red blood cells (RBCs). Traditionally, microcirculation has been modeled by computing the mean velocity of a few, randomly selected, manually tracked RBCs. However, this protocol is tedious, time consuming, and subjective with technician related bias. We present a new method for analyzing the microcirculation by modeling the RBC motion through automatic tracking. The tracking of RBCs is challenging as in each image, as many as 200 cells move through a complex network of vessels at a wide range of speeds while deforming in shape. To reliably detect RBCs traveling at a wide range of speeds, a window of temporal template matching is applied. Then, cells appearing in successive frames are corresponded based on the motion behavior constraints in terms of the direction, magnitude, and path. The performance evaluation against a ground truth indicates the detection accuracy up to 84% TP at 6% FP and a correspondence accuracy of 89%. We include an in-depth discussion on comparison of the microcirculation based on motion modeling from the proposed automated method against a mean velocity from manual analysis protocol in terms of precision, objectivity, and sensitivity.

Vessels as 4-D curves: global minimal 4-D paths to extract 3-D tubular surfaces and centerlines

In this paper, we propose an innovative approach to the segmentation of tubular structures. This approach combines all of the benefits of minimal path techniques such as global minimizers, fast computation, and powerful incorporation of user input, while also having the capability to represent and detect vessel surfaces directly which so far has been a feature restricted to active contour and surface techniques. The key is to represent the trajectory of a tubular structure not as a 3-D curve but to go up a dimension and represent the entire structure as a 4-D curve. Then we are able to fully exploit minimal path techniques to obtain global minimizing trajectories between two user supplied endpoints in order to reconstruct tubular structures from noisy or low contrast 3-D data without the sensitivity to local minima inherent in most active surface techniques. In contrast to standard purely spatial 3-D minimal path techniques, however, we are able to represent a full tubular surface rather than just a curve which runs through its interior. Our representation also yields a natural notion of a tube's "central curve." We demonstrate and validate the utility of this approach on magnetic resonance (MR) angiography and computed tomography (CT) images of coronary arteries.

A Monte Carlo approach to rolling leukocyte tracking in vivo

Tracking the movement of rolling leukocytes in vivo contributes to the understanding of the mechanism of the inflammatory process and to the development of anti-inflammatory drugs. Several roadblocks exist that hinder successful automated tracking including the moving background, the severe image noise and clutter, the occlusion of the target leukocyte by other leukocytes and structures, the jitter caused by the breathing movement of the living animal, and the weak image contrast. In this paper, a Monte Carlo tracker is developed for automatically tracking a single rolling leukocyte in vivo. Based on the leukocyte movement information and the image intensity features, a specialized sample-weighting criterion is tailored to the application. In comparison with a snake-based tracker, our experiments show that, as the noise intensity level increases, the performance of the snake tracker degrades more than that of the Monte Carlo tracker. In cases, where the leukocyte is observed in contact with the vessel wall, the Monte Carlo tracker is less affected by the image clutter. From tracking within 99 intravital microscopic video sequences, the Monte Carlo tracker exhibits superior performance in the reduced localization error and the increased number of frames tracked when compared with the centroid tracker, the correlation tracker and the GVF snake tracker.

Analysis of leukocyte rolling in vivo and in vitro

Leukocyte rolling is an important step for the successful recruitment of leukocytes from blood to tissues mediated by a specialized group of glycoproteins termed selectins. Because of the dynamic process of leukocyte rolling, binding of selectins to their respective counter-receptors (selectin ligands) needs to fulfill three major requirements: (1) rapid bond formation, (2) high tensile strength, and (3) fast dissociation rates. These criteria are perfectly met by selectins, which interact with specific carbohydrate determinants on selectin ligands. This chapter describes the theoretical background, technical requirements, and analytical tools needed to quantitatively assess leukocyte rolling in vivo and in vitro. For the in vivo setting, intravital microscopy allows the observation and recording of leukocyte rolling under different physiological and pathological conditions in almost every organ. Real-time and off-line analysis tools help to assess geometric, hemodynamic, and rolling parameters. Under in vitro conditions, flow chamber assays such as parallel plate flow chamber systems have been the mainstay to study interactions between leukocytes and adhesion molecules under flow. In this setting, adhesion molecules are immobilized on plastic, in a lipid monolayer, or presented on cultured endothelial cells on the chamber surface. Microflow chambers are available for studying leukocyte adhesion in the context of whole blood and without blood cell isolation. The microscopic observation of leukocyte rolling in different in vivo and in vitro settings has significantly contributed to our understanding of the molecular mechanisms responsible for the stepwise extravasation of leukocytes into inflamed tissues.

Data acceptance for automated leukocyte tracking through segmentation of spatiotemporal images

A crucial task in inflammation research and inflammatory drug validation is leukocyte velocity data collection from microscopic video imagery. Since manual methods are bias-prone and extremely time consuming, automated tracking methods are required to compute cell velocities. However, an automated tracking method is of little practical use unless it is accompanied by a mechanism to validate the tracker output. In this paper, we propose a validation technique that accepts or rejects the output of automated tracking methods. The proposed method first generates a spatiotemporal image from the cell locations given by a tracking method; then, it segments the spatiotemporal image to detect the presence or absence of a leukocyte. For segmenting the spatiotemporal images, we employ an edge-direction sensitive nonlinear filter followed by an active contour based technique. The proposed nonlinear filter, the maximum absolute average directional derivative (MAADD), first computes the magnitude of the mean directional derivative over an oriented line segment and then chooses the maximum of all such values within a range of orientations of the line segment. The proposed active contour segmentation is obtained via growing contours controlled by a two-dimensional force field, which is constructed by imposing a Dirichlet boundary condition on the gradient vector flow (GVF) field equations. The performance of the proposed validation method is reported here for the outputs of three different tracking techniques: the method was successful in 97% of the trials using manual tracking, in 94% using correlation tracking and in 93% using active contour tracking.

An automated method for analysis of flow characteristics of circulating particles from in vivo video microscopy

The behavior of white and red blood cells, platelets, and circulating injected particles is one of the most studied areas of physiology. Most methods used to analyze the circulatory patterns of cells are time consuming. We describe a system named CellTrack, designed for fully automated tracking of circulating cells and micro-particles and retrieval of their behavioral characteristics. The task of automated blood cell tracking in vessels from in vivo video is particularly challenging because of the blood cells' nonrigid shapes, the instability inherent in in vivo videos, the abundance of moving objects and their frequent superposition. To tackle this, the CellTrack system operates on two levels: first, a global processing module extracts vessel borders and center lines based on color and temporal patterns. This enables the computation of the approximate direction of the blood flow in each vessel. Second, a local processing module extracts the locations and velocities of circulating cells. This is performed by artificial neural network classifiers that are designed to detect specific types of blood cells and micro-particles. The motion correspondence problem is then resolved by a novel algorithm that incorporates both the local and the global information. The system has been tested on a series of in vivo color video recordings of rat mesentery. Our results show that the synergy between the global and local information enables CellTrack to overcome many of the difficulties inherent in tracking methods that rely solely on local information. A comparison was made between manual measurements and the automatically extracted measurements of leukocytes and fluorescent microspheres circulatory velocities. This comparison revealed an accuracy of 97%. CellTrack also enabled a much larger volume of sampling in a fraction of time compared to the manual measurements. All these results suggest that our method can in fact constitute a reliable replacement for manual extraction of blood flow characteristics from in vivo videos.

Image Stabilization and Registration for Tracking Cells in the Microvasculature

We propose a registration system to be used for tracking cells in intravital video microscopy that 1) stabilizes jitter-the undesired translational displacement of frames due to respiratory movement, etc., and 2) registers frames in a moving field of view (FOV) to allow for cell tracking over an extended range. For the first time, tracking of rolling leukocytes in vivo over a moving FOV is demonstrated. In a fixed FOV, stable background regions are located using a morphological approach. Template subregions are then selected from the stable regions and matched to corresponding locations in a reference frame. We show the effectiveness of the stabilization algorithm by using an active contour to track 15 leukocytes previously untrackable due to jitter. For 30 fixed FOV sequences containing rolling leukocytes, the resulting root-mean-square error (RMSE) is less than 0.5 microm. To align frames in a moving FOV, we present a modified correlation approach to estimate the common region between two consecutive fixed FOVs. We correlate the overlapping regions of the initial frame of the current fixed FOV and the final frame of the previous fixed FOV to register the images in the adjoining moving FOV. The RMSE of our moving FOV registration technique was less than 0.6 mmicrom. In 10 sequences from different venules, we were able to track 11 cells using an active contour approach over moving FOVs.

Motion gradient vector flow: an external force for tracking rolling leukocytes with shape and size constrained active contours

Recording rolling leukocyte velocities from intravital microscopic video imagery is a critical task in inflammation research and drug validation. Since manual tracking is excessively time consuming, an automated method is desired. This paper illustrates an active contour based automated tracking method, where we propose a novel external force to guide the active contour that takes the hemodynamic flow direction into account. The construction of the proposed force field, referred to as motion gradient vector flow (MGVF), is accomplished by minimizing an energy functional involving the motion direction, and the image gradient magnitude. The tracking experiments demonstrate that MGVF can be used to track both slow- and fast-rolling leukocytes, thus extending the capture range of previously designed cell tracking techniques.

A review on MR vascular image processing algorithms: acquisition and prefiltering: part I

Vascular segmentation has recently been given much attention. This review paper has two parts. Part I focuses on the physics of magnetic resonance angiography (MRA) generation and prefiltering techniques applied to MRA data sets. Part II of the review focuses on the vessel segmentation algorithms. The first section of this paper introduces the five different sets of receive coils used with the MRI system for magnetic resonance angiography data acquisition. This section then presents the five different types of the most popular data acquisition techniques: time-of-flight (TOF), phase-contrast, contrast-enhanced, black-blood, T2-weighted, and T2*-weighted, along with their pros and cons. Section II of this paper focuses on prefiltering algorithms for MRA data sets. This is necessary for removing the background nonvascular structures in the MRA data sets. Finally, the paper concludes with a clinical discussion on the challenges and the future of the data acquisition and the automated filtering algorithms.

Level set analysis for leukocyte detection and tracking

We propose a cell detection and tracking solution using image-level sets computed via threshold decomposition. In contrast to existing methods where manual initialization is required to track individual cells, the proposed approach can automatically identify and track multiple cells by exploiting the shape and intensity characteristics of the cells. The capture of the cell boundary is considered as an evolution of a closed curve that maximizes image gradient along the curve enclosing a homogeneous region. An energy functional dependent upon the gradient magnitude along the cell boundary, the region homogeneity within the cell boundary and the spatial overlap of the detected cells is minimized using a variational approach. For tracking between frames, this energy functional is modified considering the spatial and shape consistency of a cell as it moves in the video sequence. The integrated energy functional complements shape-based segmentation with a spatial consistency based tracking technique. We demonstrate that an acceptable, expedient solution of the energy functional is possible through a search of the image-level lines: boundaries of connected components within the level sets obtained by threshold decomposition. The level set analysis can also capture multiple cells in a single frame rather than iteratively computing a single active contour for each individual cell. Results of cell detection using the energy functional approach and the level set approach are presented along with the associated processing time. Results of successful tracking of rolling leukocytes from a number of digital video sequences are reported and compared with the results from a correlation tracking scheme.

Vessel Boundary Tracking for Intravital Microscopy Via Multiscale Gradient Vector Flow Snakes

Due to movement of the specimen, vasodilation, and intense clutter, the intravital location of a vessel boundary from video microscopy is a difficult but necessary task in analyzing the mechanics of inflammation and the structure of the microvasculature. This paper details an active contour model for vessel boundary detection and tracking. In developing the method, two innovations are introduced. First, the B-spline model is combined with the gradient vector flow (GVF) external force. Second, a multiscale gradient vector flow (MSGVF) is employed to elude clutter and to reliably localize the vessel boundaries. Using synthetic experiments and video microscopy obtained via transillumination of the mouse cremaster muscle, we demonstrate that the MSGVF approach is superior to the fixed-scale GVF approach in terms of boundary localization. In each experiment, the fixed scale approach yielded at least a 50% increase in root mean squared error over the multiscale approach. In addition to delineating the vessel boundary so that cells can be detected and tracked, we demonstrate the boundary location technique enables automatic blood flow velocity computation in vivo.

A review on MR vascular image processing: skeleton versus nonskeleton approaches: part II

Vascular segmentation has recently been given much attention. This review paper has two parts. Part I of this review focused on the physics of magnetic resonance angiography (MRA) and prefiltering techniques applied to MRA. Part II of this review presents the state-of-the-art overview, status, and new achievements in vessel segmentation algorithms from MRA. The first part of this review paper is focused on the nonskeleton or direct-based techniques. Here, we present eight different techniques along with their mathematical foundations, algorithms and their pros and cons. We will also focus on the skeleton or indirect-based techniques. We will discuss three different techniques along with their mathematical foundations, algorithms and their pros and cons. This paper also includes a clinical discussion on skeleton versus nonskeleton-based segmentation techniques. Finally, we shall conclude this paper with the possible challenges, the future, and a brief summary on vascular segmentation techniques.

Tracking leukocytes in vivo with shape and size constrained active contours

Inflammatory disease is initiated by leukocytes (white blood cells) rolling along the inner surface lining of small blood vessels called postcapillary venules. Studying the number and velocity of rolling leukocytes is essential to understanding and successfully treating inflammatory diseases. Potential inhibitors of leukocyte recruitment can be screened by leukocyte rolling assays and successful inhibitors validated by intravital microscopy. In this paper, we present an active contour or snake-based technique to automatically track the movement of the leukocytes. The novelty of the proposed method lies in the energy functional that constrains the shape and size of the active contour. This paper introduces a significant enhancement over existing gradient-based snakes in the form of a modified gradient vector flow. Using the gradient vector flow, we can track leukocytes rolling at high speeds that are not amenable to tracking with the existing edge-based techniques. We also propose a new energy-based implicit sampling method of the points on the active contour that replaces the computationally expensive explicit method. To enhance the performance of this shape and size constrained snake model, we have coupled it with Kalman filter so that during coasting (when the leukocytes are completely occluded or obscured), the tracker may infer the location of the center of the leukocyte. Finally, we have compared the performance of the proposed snake tracker with that of the correlation and centroid-based trackers. The proposed snake tracker results in superior performance measures, such as reduced error in locating the leukocyte under tracking and improvements in the percentage of frames successfully tracked. For screening and drug validation, the tracker shows promise as an automated data collection tool.

Detection of leukocytes in contact with the vessel wall from in vivo microscope recordings using a neural network

Leukocytes play an important role in the host defense as they may travel from the blood stream into the tissue in reacting to inflammatory stimuli. The leukocyte-vessel wall interactions are studied in post capillary vessels by intravital video microscopy during in vivo animal experiments. Sequences of video images are obtained and digitized with a frame grabber. A method for automatic detection and characterization of leukocytes in the video images is developed. Individual leukocytes are detected using a neural network that is trained with synthetic leukocyte images generated using a novel stochastic model. This model makes it feasible to generate images of leukocytes with different shapes and sizes under various lighting conditions. Experiments indicate that neural networks trained with the synthetic leukocyte images perform better than networks trained with images of manually detected leukocytes. The best performing neural network trained with synthetic leukocyte images resulted in an 18% larger area under the ROC curve than the best performing neural network trained with manually detected leukocytes.

Extraction of fluorescent dot traces from a scanning laser ophthalmoscope image sequence by spatio-temporal image analysis: Gabor filter and radon transform filtering

The scanning laser ophthalmoscope (SLO) allows the tracking of fluorescent dot motion, thereby enabling the flow velocities in perimacular capillaries to be directly measured. These can serve as an important index of local retinal soundness or reflect the whole body circulation status in disorders such as diabetes. Although it is possible to perceive moving fluorescent dots with the human eye, they are so faint and unstable that it is difficult to detect them by conventional digital still-image processing methods. To solve this problem, we generated spatio-temporal images of the fluorescent dots in a capillary and applied Gabor filters tuned to the direction of the traces in order to detect them. Finally, by discriminating and integrating the output using two levels of threshold, we were able to extract their traces. Because the medium-size Gabor filter requires a considerable amount of time for two-dimensional convolution calculation, we prove that there is a certain equivalence between the Gabor filter, the radon transform, and the Hough transform. In the light of this, we propose a form of radon transform filtering that includes a radon transform Gabor filter as a very long Gabor filter. This allows a whole trace to be detected in a single step with a one-dimensional convolution, thereby shortening the processing time. In an experiment, 60% of the traces could be detected without error, which is sufficient to allow the mean flow velocity in a capillary to be measured.

Three-dimensional multi-scale line filter for segmentation and visualization of curvilinear structures in medical images

This paper describes a method for the enhancement of curvilinear structures such as vessels and bronchi in three-dimensional (3-D) medical images. A 3-D line enhancement filter is developed with the aim of discriminating line structures from other structures and recovering line structures of various widths. The 3-D line filter is based on a combination of the eigenvalues of the 3-D Hessian matrix. Multi-scale integration is formulated by taking the maximum among single-scale filter responses, and its characteristics are examined to derive criteria for the selection of parameters in the formulation. The resultant multi-scale line-filtered images provide significantly improved segmentation and visualization of curvilinear structures. The usefulness of the method is demonstrated by the segmentation and visualization of brain vessels from magnetic resonance imaging (MRI) and magnetic resonance angiography (MRA), bronchi from a chest CT, and liver vessels (portal veins) from an abdominal CT.