Vessel Boundary Tracking for Intravital Microscopy Via Multiscale Gradient Vector Flow Snakes

CoVi-Net: A hybrid convolutional and vision transformer neural network for retinal vessel segmentation

Retinal vessel segmentation plays a crucial role in the diagnosis and treatment of ocular pathologies. Current methods have limitations in feature fusion and face challenges in simultaneously capturing global and local features from fundus images. To address these issues, this study introduces a hybrid network named CoVi-Net, which combines convolutional neural networks and vision transformer. In our proposed model, we have integrated a novel module for local and global feature aggregation (LGFA). This module facilitates remote information interaction while retaining the capability to effectively gather local information. In addition, we introduce a bidirectional weighted feature fusion module (BWF). Recognizing the variations in semantic information across layers, we allocate adjustable weights to different feature layers for adaptive feature fusion. BWF employs a bidirectional fusion strategy to mitigate the decay of effective information. We also incorporate horizontal and vertical connections to enhance feature fusion and utilization across various scales, thereby improving the segmentation of multiscale vessel images. Furthermore, we introduce an adaptive lateral feature fusion (ALFF) module that refines the final vessel segmentation map by enriching it with more semantic information from the network. In the evaluation of our model, we employed three well-established retinal image databases (DRIVE, CHASEDB1, and STARE). Our experimental results demonstrate that CoVi-Net outperforms other state-of-the-art techniques, achieving a global accuracy of 0.9698, 0.9756, and 0.9761 and an area under the curve of 0.9880, 0.9903, and 0.9915 on DRIVE, CHASEDB1, and STARE, respectively. We conducted ablation studies to assess the individual effectiveness of the three modules. In addition, we examined the adaptability of our CoVi-Net model for segmenting lesion images. Our experiments indicate that our proposed model holds promise in aiding the diagnosis of retinal vascular disorders.

Automated Layer Segmentation of Retinal Optical Coherence Tomography Images Using a Deep Feature Enhanced Structured Random Forests Classifier

Optical coherence tomography (OCT) is a high-resolution and noninvasive imaging modality that has become one of the most prevalent techniques for ophthalmic diagnosis. Retinal layer segmentation is very crucial for doctors to diagnose and study retinal diseases. However, manual segmentation is often a time-consuming and subjective process. In this work, we propose a new method for automatically segmenting retinal OCT images, which integrates deep features and hand-designed features to train a structured random forests classifier. The deep convolutional features are learned from deep residual network. With the trained classifier, we can get the contour probability graph of each layer; finally, the shortest path is employed to achieve the final layer segmentation. The experimental results show that our method achieves good results with the mean layer contour error of 1.215 pixels, whereas that of the state of the art was 1.464 pixels, and achieves an F1-score of 0.885, which is also better than 0.863 that is obtained by the state of the art method.

Multiresolution Subdivision Snakes

We present a new family of snakes that satisfy the property of multiresolution by exploiting subdivision schemes. We show in a generic way how to construct such snakes based on an admissible subdivision mask. We derive the necessary energy formulations and provide the formulas for their efficient computation. Depending on the choice of the mask, such models have the ability to reproduce trigonometric or polynomial curves. They can also be designed to be interpolating, a property that is useful in user-interactive applications. We provide explicit examples of subdivision snakes and illustrate their use for the segmentation of bioimages. We show that they are robust in the presence of noise and provide a multiresolution algorithm to enlarge their basin of attraction, which decreases their dependence on initialization compared to singleresolution snakes. We show the advantages of the proposed model in terms of computation and segmentation of structures with different sizes.

Accurate vessel segmentation with constrained B-snake

We describe an active contour framework with accurate shape and size constraints on the vessel cross-sectional planes to produce the vessel segmentation. It starts with a multiscale vessel axis tracing in a 3D computed tomography (CT) data, followed by vessel boundary delineation on the cross-sectional planes derived from the extracted axis. The vessel boundary surface is deformed under constrained movements on the cross sections and is voxelized to produce the final vascular segmentation. The novelty of this paper lies in the accurate contour point detection of thin vessels based on the CT scanning model, in the efficient implementation of missing contour points in the problematic regions and in the active contour model with accurate shape and size constraints. The main advantage of our framework is that it avoids disconnected and incomplete segmentation of the vessels in the problematic regions that contain touching vessels (vessels in close proximity to each other), diseased portions (pathologic structure attached to a vessel), and thin vessels. It is particularly suitable for accurate segmentation of thin and low contrast vessels. Our method is evaluated and demonstrated on CT data sets from our partner site, and its results are compared with three related methods. Our method is also tested on two publicly available databases and its results are compared with the recently published method. The applicability of the proposed method to some challenging clinical problems, the segmentation of the vessels in the problematic regions, is demonstrated with good results on both quantitative and qualitative experimentations; our segmentation algorithm can delineate vessel boundaries that have level of variability similar to those obtained manually.

Segmentation of the striatum from MR brain images to calculate the 99mTc-TRODAT-1 binding ratio in SPECT images

Quantification of regional ^99mTc-TRODAT-1 binding ratio in the striatum regions in SPECT images is essential for differential diagnosis between Alzheimer's and Parkinson's diseases. Defining the region of the striatum in the SPECT image is the first step toward success in the quantification of the TRODAT-1 binding ratio. However, because SPECT images reveal insufficient information regarding the anatomical structure of the brain, correct delineation of the striatum directly from the SPECT image is almost impossible. We present a method integrating the active contour model and the hybrid registration technique to extract regions from MR T1-weighted images and map them into the corresponding SPECT images. Results from three normal subjects suggest that the segmentation accuracy using the proposed method was compatible with the expert decision but has a higher efficiency and reproducibility than manual delineation. The binding ratio derived by this method correlated well (R² = 0.76) with those values calculated by commercial software, suggesting the feasibility of the proposed method.

Vessel Boundary Detection for its 3D Reconstruction by Using a Deformable Model (GVF Snake)

Vessel boundary detection and 3D modeling is a difficult but necessary task in analyzing the mechanics of inflammation and the structure of the microvasculature. We present in this paper a method of analyzing this structure by the means of the deformable model (using GVF Snake) for vessel boundary detection and three-dimensional reconstruction. For this purpose, we used a virtual vessel model and produced synthetic images. From these images, we obtained contours of vessels by the GVF Snake and then reconstructed a three-dimensional structure by using the coordinates of the Snakes.

Segmentation of brain tumors in MRI images using multi-scale gradient vector flow

The gradient vector flow (GVF) algorithm has been used extensively as an efficient method for medical image segmentation. This algorithm suffers from poor robustness against noise as well as lack of convergence in small scale details and concavities. As a cure to this problem, in this paper the idea of multi scale is applied to the traditional GVF algorithm for segmentation of brain tumors in MRI images. Using this idea, the active contour is evolved with respect to scaled edge maps in a multi scale manner. The edge detection performance of the modified GVF algorithm is further enhanced by applying a threshold-based edge detector to improve the edge map. The Bspline snake is selected for representation of the active contour, due to its ability to capture corners and its local control. The results showed an improvement of 30% in the accuracy of tumor segmentation against traditional GVF and 10 % as compared to Bspline GVF in the presence of noise, besides the repeatability of the algorithm in contrast to traditional GVF. The clinical evaluation also proved the accuracy and sensitivity of the proposed method as 92.8% and 95.4%, respectively.

Tracking monotonically advancing boundaries in image sequences using graph cuts and recursive kernel shape priors

We introduce a probabilistic computer vision technique to track monotonically advancing boundaries of objects within image sequences. Our method incorporates a novel technique for including statistical prior shape information into graph-cut based segmentation, with the aid of a majorization-minimization algorithm. Extension of segmentation from single images to image sequences then follows naturally using sequential Bayesian estimation. Our methodology is applied to two unrelated sets of real biomedical imaging data, and a set of synthetic images. Our results are shown to be superior to manual segmentation.

Image segmentation based on the Poincaré map method

Active contour models (ACMs) integrated with various kinds of external force fields to pull the contours to the exact boundaries have shown their powerful abilities in object segmentation. However, local minimum problems still exist within these models, particularly the vector field's "equilibrium issues." Different from traditional ACMs, within this paper, the task of object segmentation is achieved in a novel manner by the Poincaré map method in a defined vector field in view of dynamical systems. An interpolated swirling and attracting flow (ISAF) vector field is first generated for the observed image. Then, the states on the limit cycles of the ISAF are located by the convergence of Newton-Raphson sequences on the given Poincaré sections. Meanwhile, the periods of limit cycles are determined. Consequently, the objects' boundaries are represented by integral equations with the corresponding converged states and periods. Experiments and comparisons with some traditional external force field methods are done to exhibit the superiority of the proposed method in cases of complex concave boundary segmentation, multiple-object segmentation, and initialization flexibility. In addition, it is more computationally efficient than traditional ACMs by solving the problem in some lower dimensional subspace without using level-set methods.

Image-based characterization of thrombus formation in time-lapse DIC microscopy

The characterization of thrombus formation in time-lapse DIC microscopy is of increased interest for identifying genes which account for atherothrombosis and coronary artery diseases (CADs). In particular, we are interested in large-scale studies on zebrafish, which result in large amount of data, and require automatic processing. In this work, we present an image-based solution for the automatized extraction of parameters quantifying the temporal development of thrombotic plugs. Our system is based on the joint segmentation of thrombotic and aortic regions over time. This task is made difficult by the low contrast and the high dynamic conditions observed in vivo DIC microscopic scenes. Our key idea is to perform this segmentation by distinguishing the different motion patterns in image time series rather than by solving standard image segmentation tasks in each image frame. Thus, we are able to compensate for the poor imaging conditions. We model motion patterns by energies based on the idea of dynamic textures, and regularize the model by two prior energies on the shape of the aortic region and on the topological relationship between the thrombus and the aorta. We demonstrate the performance of our segmentation algorithm by qualitative and quantitative experiments on synthetic examples as well as on real in vivo microscopic sequences.

Automated localisation and boundary identification of superficial femoral artery on MRI sequences

In this paper, an automated method to localise the right superficial femoral artery (SFA) and identify its boundary on magnetic resonance imaging (MRI) sequences without contrast medium injection is proposed. Some anatomical knowledge combined with the mathematical morphology is used to distinguish SFA from other vessels. Afterwards, the directional gradient, continuity and the local contrast are applied as features to identify the artery's boundary using dynamic programming. The accuracy analysis shows that the system has average unsigned errors 3.1 ± 3.1% on five sequences compared to experts' manual tracings.

A statistical modeling approach to the analysis of spatial patterns of FDG-PET uptake in human sarcoma

Clinical experience with positron emission tomography (PET) scanning of sarcoma, using fluorodeoxyglucose (FDG), has established spatial heterogeneity in the standardized uptake values within the tumor mass as a key prognostic indicator of patient survival. But it may be that a more detailed quantitation of the tumor FDG uptake pattern could provide additional insights into risk. The present work develops a statistical model for this purpose. The approach is based on a tubular representation of the tumor mass with a simplified radial analysis of uptake, transverse to the tubular axis. The technique provides novel ways of characterizing the overall profile of the tumor, including the introduction of an approach for the measurement of its phase of development. The phase measure can distinguish between early phase tumors, in which the uptake is highest at the core, and later stage masses, in which there can often be central voids in FDG uptake. Biologically, these voids arise from necrosis and fluid, fat or cartilage accumulations. The tumor profiling technique is implemented using open-source software tools and illustrations are provided with clinically representative scans. A series of FDG-PET studies from 185 patients is used to formally evaluate the prognostic benefit. Significant improvements in the prediction of patient survival and progression are obtained from the tumor profiling analysis. After adjustment for other factors including heterogeneity, a typical one standard deviation increase in phase (as determined by the analysis) is associated with close to 20% more risk of progression or death. The work confirms that more detailed quantitative assessments of the spatial pattern of PET imaging data of tumor masses, beyond the maximum FDG uptake (SUV(max)) and previously considered measures of heterogeneity, provide improved prognostic information for potential input to treatment decisions for future patients.

Speckle reducing bilateral filter for cattle follicle segmentation

BACKGROUND

Ultrasound imaging technology has wide applications in cattle reproduction and has been used to monitor individual follicles and determine the patterns of follicular development. However, the speckles in ultrasound images affect the post-processing, such as follicle segmentation and finally affect the measurement of the follicles. In order to reduce the effect of speckles, a bilateral filter is developed in this paper.

RESULTS

We develop a new bilateral filter for speckle reduction in ultrasound images for follicle segmentation and measurement. Different from the previous bilateral filters, the proposed bilateral filter uses normalized difference in the computation of the Gaussian intensity difference. We also present the results of follicle segmentation after speckle reduction. Experimental results on both synthetic images and real ultrasound images demonstrate the effectiveness of the proposed filter.

CONCLUSIONS

Compared with the previous bilateral filters, the proposed bilateral filter can reduce speckles in both high-intensity regions and low intensity regions in ultrasound images. The segmentation of the follicles in the speckle reduced images by the proposed method has higher performance than the segmentation in the original ultrasound image, and the images filtered by Gaussian filter, the conventional bilateral filter respectively.

Multislice human organ extraction based on GVF

Segmentation techniques based on active contours algorithm are widely used in medical imaging. Unfortunately, they require a lot of parameters to be used and this can represent an issue for those physicians with not much informatics skills. This paper proposes a software tool which allows to segment multiple slice can differ organ extraction setting a small number of parameters. Moreover, the tool offers the functionality to perform a multiple segmentation the same time, so that an entire volume composed by hundreds slices can be segmented.

Arranging and interpolating sparse unorganized feature points with geodesic circular arc

A novel method to reconstruct object boundaries with geodesic circular arc is proposed in this paper. Within this framework, an energy of circular arc spline is utilized to simultaneously arrange and interpolate each member in the set of sparse unorganized feature points from the desired boundaries. A general form for a family of parametric circular arc spline is firstly derived and followed by a novel method of arranging these feature points by minimizing an energy term depending on the circular arc spline configuration defined on these feature points. With regard to the fact that the energy function is usually nonconvex and nondifferentiable at its critical points, an improved scheme of particle swarm optimizer is given to find the minimum for the energy in this paper. With this improved scheme, each pair of neighboring feature points along the boundaries of the desired objects are picked out from the set of sparse unorganized feature points, and the corresponding directional chord tangent angles are computed simultaneously to finish interpolation. We show experimentally and comparatively that the proposed method can perform effectively to restrict leakage on weak boundaries and premature convergence on long concave boundaries. Besides, it has good noise robustness and can as well extract multiple and open boundaries.

Segmentation of vessels cluttered with cells using a physics based model

Segmentation of vessels in biomedical images is important as it can provide insight into analysis of vascular morphology, topology and is required for kinetic analysis of flow velocity and vessel permeability. Intravital microscopy is a powerful tool as it enables in vivo imaging of both vasculature and circulating cells. However, the analysis of vasculature in those images is difficult due to the presence of cells and their image gradient. In this paper, we provide a novel method of segmenting vessels with a high level of cell related clutter. A set of virtual point pairs ("vessel probes") are moved reacting to forces including Vessel Vector Flow (VVF) and Vessel Boundary Vector Flow (VBVF) forces. Incorporating the cell detection, the VVF force attracts the probes toward the vessel, while the VBVF force attracts the virtual points of the probes to localize the vessel boundary without being distracted by the image features of the cells. The vessel probes are moved according to Newtonian Physics reacting to the net of forces applied on them. We demonstrate the results on a set of five real in vivo images of liver vasculature cluttered by white blood cells. When compared against the ground truth prepared by the technician, the Root Mean Squared Error (RMSE) of segmentation with VVF and VBVF was 55% lower than the method without VVF and VBVF.

Liver microcirculation analysis by red blood cell motion modeling in intravital microscopy images

Intravital microscopy has been used to visualize the microcirculation by imaging fluorescent labeled red blood cells (RBCs). Traditionally, microcirculation has been modeled by computing the mean velocity of a few, randomly selected, manually tracked RBCs. However, this protocol is tedious, time consuming, and subjective with technician related bias. We present a new method for analyzing the microcirculation by modeling the RBC motion through automatic tracking. The tracking of RBCs is challenging as in each image, as many as 200 cells move through a complex network of vessels at a wide range of speeds while deforming in shape. To reliably detect RBCs traveling at a wide range of speeds, a window of temporal template matching is applied. Then, cells appearing in successive frames are corresponded based on the motion behavior constraints in terms of the direction, magnitude, and path. The performance evaluation against a ground truth indicates the detection accuracy up to 84% TP at 6% FP and a correspondence accuracy of 89%. We include an in-depth discussion on comparison of the microcirculation based on motion modeling from the proposed automated method against a mean velocity from manual analysis protocol in terms of precision, objectivity, and sensitivity.

Analysis of leukocyte rolling in vivo and in vitro

Leukocyte rolling is an important step for the successful recruitment of leukocytes from blood to tissues mediated by a specialized group of glycoproteins termed selectins. Because of the dynamic process of leukocyte rolling, binding of selectins to their respective counter-receptors (selectin ligands) needs to fulfill three major requirements: (1) rapid bond formation, (2) high tensile strength, and (3) fast dissociation rates. These criteria are perfectly met by selectins, which interact with specific carbohydrate determinants on selectin ligands. This chapter describes the theoretical background, technical requirements, and analytical tools needed to quantitatively assess leukocyte rolling in vivo and in vitro. For the in vivo setting, intravital microscopy allows the observation and recording of leukocyte rolling under different physiological and pathological conditions in almost every organ. Real-time and off-line analysis tools help to assess geometric, hemodynamic, and rolling parameters. Under in vitro conditions, flow chamber assays such as parallel plate flow chamber systems have been the mainstay to study interactions between leukocytes and adhesion molecules under flow. In this setting, adhesion molecules are immobilized on plastic, in a lipid monolayer, or presented on cultured endothelial cells on the chamber surface. Microflow chambers are available for studying leukocyte adhesion in the context of whole blood and without blood cell isolation. The microscopic observation of leukocyte rolling in different in vivo and in vitro settings has significantly contributed to our understanding of the molecular mechanisms responsible for the stepwise extravasation of leukocytes into inflamed tissues.

Delineating fluid-filled region boundaries in optical coherence tomography images of the retina

We evaluate the ability of a deformable model to yield accurate shape descriptions of fluid-filled regions associated with age-related macular degeneration. Calculation of retinal thickness and volume by the current optical coherence tomography (OCT) system includes fluid-filled regions or lesions along with actual retinal tissue. In order to quantify these lesions independently from the retinal tissue, they must be outlined. A deformable model was applied to OCT images of retinas demonstrating cystoids and subretinal fluid spaces. Several implementation issues were addressed in order to choose appropriate parameters. The use of a nonlinear anisotropic diffusion filter to suppress speckle noise while at the same time preserving the edges of the original image was explored. Once the contours of the lesions were outlined, quantitative analysis of the surface area and volume of the lesions was performed. The deformable model could accurately outline fluid-filled regions within the retina. The detection method tested proved effective in capturing the complexity of fluid-filled regions in OCT images. Deformable models combined with nonlinear anisotropic diffusion filtering show promise in the detection of retinal features of interest for diagnosis in clinical OCT images. Thus, fluid-filled region detection may significantly aid in analysis of treatments and diagnosis.

An automated method for analysis of flow characteristics of circulating particles from in vivo video microscopy

The behavior of white and red blood cells, platelets, and circulating injected particles is one of the most studied areas of physiology. Most methods used to analyze the circulatory patterns of cells are time consuming. We describe a system named CellTrack, designed for fully automated tracking of circulating cells and micro-particles and retrieval of their behavioral characteristics. The task of automated blood cell tracking in vessels from in vivo video is particularly challenging because of the blood cells' nonrigid shapes, the instability inherent in in vivo videos, the abundance of moving objects and their frequent superposition. To tackle this, the CellTrack system operates on two levels: first, a global processing module extracts vessel borders and center lines based on color and temporal patterns. This enables the computation of the approximate direction of the blood flow in each vessel. Second, a local processing module extracts the locations and velocities of circulating cells. This is performed by artificial neural network classifiers that are designed to detect specific types of blood cells and micro-particles. The motion correspondence problem is then resolved by a novel algorithm that incorporates both the local and the global information. The system has been tested on a series of in vivo color video recordings of rat mesentery. Our results show that the synergy between the global and local information enables CellTrack to overcome many of the difficulties inherent in tracking methods that rely solely on local information. A comparison was made between manual measurements and the automatically extracted measurements of leukocytes and fluorescent microspheres circulatory velocities. This comparison revealed an accuracy of 97%. CellTrack also enabled a much larger volume of sampling in a fraction of time compared to the manual measurements. All these results suggest that our method can in fact constitute a reliable replacement for manual extraction of blood flow characteristics from in vivo videos.

Intravital leukocyte detection using the gradient inverse coefficient of variation

The problem of identifying and counting rolling leukocytes within intravital microscopy is of both theoretical and practical interest. Currently, methods exist for tracking rolling leukocytes in vivo, but these methods rely on manual detection of the cells. In this paper we propose a technique for accurately detecting rolling leukocytes based on Bayesian classification. The classification depends on a feature score, the gradient inverse coefficient of variation (GICOV), which serves to discriminate rolling leukocytes from a cluttered environment. The leukocyte detection process consists of three sequential steps: the first step utilizes an ellipse matching algorithm to coarsely identify the leukocytes by finding the ellipses with a locally maximal GICOV. In the second step, starting from each of the ellipses found in the first step, a B-spline snake is evolved to refine the leukocytes boundaries by maximizing the associated GICOV score. The third and final step retains only the extracted contours that have a GICOV score above the analytically determined threshold. Experimental results using 327 rolling leukocytes were compared to those of human experts and currently used methods. The proposed GICOV method achieves 78.6% leukocyte detection accuracy with 13.1% false alarm rate.

Motion gradient vector flow: an external force for tracking rolling leukocytes with shape and size constrained active contours

Recording rolling leukocyte velocities from intravital microscopic video imagery is a critical task in inflammation research and drug validation. Since manual tracking is excessively time consuming, an automated method is desired. This paper illustrates an active contour based automated tracking method, where we propose a novel external force to guide the active contour that takes the hemodynamic flow direction into account. The construction of the proposed force field, referred to as motion gradient vector flow (MGVF), is accomplished by minimizing an energy functional involving the motion direction, and the image gradient magnitude. The tracking experiments demonstrate that MGVF can be used to track both slow- and fast-rolling leukocytes, thus extending the capture range of previously designed cell tracking techniques.

A novel method to analyze leukocyte rolling behavior in vivo

Leukocyte endothelial cell interaction is a fundamentally important process in many disease states. Current methods to analyze such interactions include the parallel-plate flow chamber and intravital microscopy. Here, we present an improvement of the traditional intravital microscopy that allows leukocyte-endothelial cell interaction to be studied from the time the leukocyte makes its initial contact with the endothelium until it adheres to or detaches from the endothelium. The leukocyte is tracked throughout the venular tree with the aid of a motorized stage and the rolling and adhesive behavior is measured off-line. Because this method can involve human error, methods to automate the tracking procedure have been developed. This novel tracking method allows for a more detailed examination of leukocyte-endothelial cell interactions.