Constructing simple stable descriptions for image partitioning

Smoothed-NUV Priors for Imaging

Variations of L1 -regularization including, in particular, total variation regularization, have hugely improved computational imaging. However, sharper edges and fewer staircase artifacts can be achieved with convex-concave regularizers. We present a new class of such regularizers using normal priors with unknown variance (NUV), which include smoothed versions of the logarithm function and smoothed versions of Lp norms with p ≤ 1 . All NUV priors allow variational representations that lead to efficient algorithms for image reconstruction by iterative reweighted descent. A preferred such algorithm is iterative reweighted coordinate descent, which has no parameters (in particular, no step size to control) and is empirically robust and efficient. The proposed priors and algorithms are demonstrated with applications to tomography. We also note that the proposed priors come with built-in edge detection, which is demonstrated by an application to image segmentation.

A Robust and Automatic Method for the Best Symmetry Plane Detection of Craniofacial Skeletons

The accurate location of the mid-sagittal plane is fundamental for the assessment of craniofacial dysmorphisms and for a proper corrective surgery planning. To date, these elaborations are carried out by skilled operators within specific software environments. Since the whole procedure is based on the manual selection of specific landmarks, it is time-consuming, and the results depend on the operators’ professional experience. This work aims to propose a new automatic and landmark-independent technique which is able to extract a reliable mid-sagittal plane from 3D CT images. The algorithm has been designed to perform a robust evaluation, also in the case of large defect areas. The presented method is an upgraded version of a mirroring-and registration technique for the automatic symmetry plane detection of 3D asymmetrically scanned human faces, previously published by the authors. With respect to the published algorithm, the improvements here introduced concern both the objective function formulation and the method used to minimize it. The automatic method here proposed has been verified in the analysis of real craniofacial skeletons also with large defects, and the results have been compared with other recent technologies.

Predicting Illusory Contours Without Extracting Special Image Features

Boundary completion is one of the desired properties of a robust object boundary detection model, since in real-word images the object boundaries are commonly not fully and clearly seen. An extreme example of boundary completion occurs in images with illusory contours, where the visual system completes boundaries in locations without intensity gradient. Most illusory contour models extract special image features, such as L and T junctions, while the task is known to be a difficult issue in real-world images. The proposed model uses a functional optimization approach, in which a cost value is assigned to any boundary arrangement to find the arrangement with minimal cost. The functional accounts for basic object properties, such as alignment with the image, object boundary continuity, and boundary simplicity. The encoding of these properties in the functional does not require special features extraction, since the alignment with the image only requires extraction of the image edges. The boundary arrangement is represented by a border ownership map, holding object boundary segments in discrete locations and directions. The model finds multiple possible image interpretations, which are ranked according to the probability that they are supposed to be perceived. This is achieved by using a novel approach to represent the different image interpretations by multiple functional local minima. The model is successfully applied to objects with real and illusory contours. In the case of Kanizsa illusion the model predicts both illusory and real (pacman) image interpretations. The model is a proof of concept and is currently restricted to synthetic gray-scale images with solid regions.

Real-Time Nonlocal Means-Based Despeckling

In this paper, we propose a multiscale nonlocal means-based despeckling method for medical ultrasound. The multiscale approach leads to large computational savings and improves despeckling results over single-scale iterative approaches. We present two variants of the method. The first, denoted multiscale nonlocal means (MNLM), yields uniform robust filtering of speckle both in structured and homogeneous regions. The second, denoted unnormalized MNLM (UMNLM), is more conservative in regions of structure assuring minimal disruption of salient image details. Due to the popularity of anisotropic diffusion-based methods in the despeckling literature, we review the connection between anisotropic diffusion and iterative variants of NLM. These iterative variants in turn relate to our multiscale variant. As part of our evaluation, we conduct a simulation study making use of ground truth phantoms generated from clinical B-mode ultrasound images. We evaluate our method against a set of popular methods from the despeckling literature on both fine and coarse speckle noise. In terms of computational efficiency, our method outperforms the other considered methods. Quantitatively on simulations and on a tissue-mimicking phantom, our method is found to be competitive with the state-of-the-art. On clinical B-mode images, our method is found to effectively smooth speckle while preserving low-contrast and highly localized salient image detail.

Effects of specular highlights on perceived surface convexity

Shading is known to produce vivid perceptions of depth. However, the influence of specular highlights on perceived shape is unclear: some studies have shown that highlights improve quantitative shape perception while others have shown no effect. Here we ask how specular highlights combine with Lambertian shading cues to determine perceived surface curvature, and to what degree this is based upon a coherent model of the scene geometry. Observers viewed ambiguous convex/concave shaded surfaces, with or without highlights. We show that the presence/absence of specular highlights has an effect on qualitative shape, their presence biasing perception toward convex interpretations of ambiguous shaded objects. We also find that the alignment of a highlight with the Lambertian shading modulates its effect on perceived shape; misaligned highlights are less likely to be perceived as specularities, and thus have less effect on shape perception. Increasing the depth of the surface or the slant of the illuminant also modulated the effect of the highlight, increasing the bias toward convexity. The effect of highlights on perceived shape can be understood probabilistically in terms of scene geometry: for deeper objects and/or highly slanted illuminants, highlights will occur on convex but not concave surfaces, due to occlusion of the illuminant. Given uncertainty about the exact object depth and illuminant direction, the presence of a highlight increases the probability that the surface is convex.

A primary goal of the human visual system is to reconstruct the three-dimensional structure of the environment from two-dimensional retinal images. This process is under-determined: an infinite number of combinations of shape, material properties and illumination conditions could give rise to any single image. Rather than determining the true three-dimensional scene in a deductive manner, the visual system must make its ‘best guess’ based on the image, probabilistic models of image formation, and the stored probability of various scene configurations. For example, the visual system appears to assume that convex surfaces are more common than concave ones, biasing perception toward convex surfaces when the image is ambiguous. Here we identify a new probabilistic cue for surface shape: a shape with a visible specular highlight is more likely to be convex than one without. Highlights occur when light is reflected in a mirror-like way from glossy surfaces such as polished marble or metal. Due to the geometry of reflection, however, highlights are more likely to be occluded on concave objects. We show that the human visual system makes use of this constraint: shape perception is biased toward convex surfaces when highlights are apparent.

On detection of multiple object instances using Hough transforms

Hough transform-based methods for detecting multiple objects use nonmaxima suppression or mode seeking to locate and distinguish peaks in Hough images. Such postprocessing requires the tuning of many parameters and is often fragile, especially when objects are located spatially close to each other. In this paper, we develop a new probabilistic framework for object detection which is related to the Hough transform. It shares the simplicity and wide applicability of the Hough transform but, at the same time, bypasses the problem of multiple peak identification in Hough images and permits detection of multiple objects without invoking nonmaximum suppression heuristics. Our experiments demonstrate that this method results in a significant improvement in detection accuracy both for the classical task of straight line detection and for a more modern category-level (pedestrian) detection problem.

A Hierarchical Algorithm for Multiphase Texture Image Segmentation

Image segmentation is a fundamental task for many computer vision and image processing applications. There exist many useful and reliable models for two-phase segmentation. However, the multiphase segmentation is a more challenging problem than two phase segmentation, mainly due to strong dependence on initialization of solutions. In this paper we propose a reliable hierarchical algorithm for multiphase texture image segmentation by making full use of two-phase texture models in a fuzzy membership framework. Application of the new algorithm to the synthetic and real medical imaging data demonstrate more satisfactory results than existing algorithms.

The collaboration of grouping laws in vision

Gestalt theory gives a list of geometric grouping laws that could in principle give a complete account of human image perception. Based on an extensive thesaurus of clever graphical images, this theory discusses how grouping laws collaborate, and conflict toward a global image understanding. Unfortunately, as shown in the bibliographical analysis herewith, the attempts to formalize the grouping laws in computer vision and psychophysics have at best succeeded to compute individual partial structures (or partial gestalts), such as alignments or symmetries. Nevertheless, we show here that a never formalized clever Gestalt experimental procedure, the Nachzeichnung suggests a numerical set up to implement and test the collaboration of partial gestalts. The new computational procedure proposed here analyzes a digital image, and performs a numerical simulation that we call Nachtanz or Gestaltic dance. In this dance, the analyzed digital image is gradually deformed in a random way, but maintaining the detected partial gestalts. The resulting dancing images should be perceptually indistinguishable if and only if the grouping process was complete. Like the Nachzeichnung, the Nachtanz permits a visual exploration of the degrees of freedom still available to a figure after all partial groups (or gestalts) have been detected. In the new proposed procedure, instead of drawing themselves, subjects will be shown samples of the automatic Gestalt dances and required to evaluate if the figures are similar. Several numerical preliminary results with this new Gestaltic experimental setup are thoroughly discussed.

MINIMUM DESCRIPTION LENGTH METHOD FOR FACET MATCHING

The Minimum Description Length (MDL) criterion is used to fit a facet model of a car to an image. The best fit is achieved when the difference image between the car and the background has the greatest compression. MDL overcomes the overfitting and parameter precision problems which hamper the more usual maximum likelihood method of model fitting. Some preliminary results are shown.

Multiregion image segmentation by parametric kernel graph cuts

The purpose of this study is to investigate multiregion graph cut image partitioning via kernel mapping of the image data. The image data is transformed implicitly by a kernel function so that the piecewise constant model of the graph cut formulation becomes applicable. The objective function contains an original data term to evaluate the deviation of the transformed data, within each segmentation region, from the piecewise constant model, and a smoothness, boundary preserving regularization term. The method affords an effective alternative to complex modeling of the original image data while taking advantage of the computational benefits of graph cuts. Using a common kernel function, energy minimization typically consists of iterating image partitioning by graph cut iterations and evaluations of region parameters via fixed point computation. A quantitative and comparative performance assessment is carried out over a large number of experiments using synthetic grey level data as well as natural images from the Berkeley database. The effectiveness of the method is also demonstrated through a set of experiments with real images of a variety of types such as medical, synthetic aperture radar, and motion maps.

Mixed segmentation-detection-based technique for point target detection in nonhomogeneous sky

This paper deals with point target detection in infrared images of the sky for which there are local variations of the gray level mean value. We show that considering a simple image model with the gray level mean value varying as a linear or a quadratic function of the pixel coordinates can improve mixed segmentation-detection performance in comparison to homogeneous model-based approaches.

Effective level set image segmentation with a kernel induced data term

This study investigates level set multiphase image segmentation by kernel mapping and piecewise constant modeling of the image data thereof. A kernel function maps implicitly the original data into data of a higher dimension so that the piecewise constant model becomes applicable. This leads to a flexible and effective alternative to complex modeling of the image data. The method uses an active curve objective functional with two terms: an original term which evaluates the deviation of the mapped image data within each segmentation region from the piecewise constant model and a classic length regularization term for smooth region boundaries. Functional minimization is carried out by iterations of two consecutive steps: 1) minimization with respect to the segmentation by curve evolution via Euler-Lagrange descent equations and 2) minimization with respect to the regions parameters via fixed point iterations. Using a common kernel function, this step amounts to a mean shift parameter update. We verified the effectiveness of the method by a quantitative and comparative performance evaluation over a large number of experiments on synthetic images, as well as experiments with a variety of real images such as medical, satellite, and natural images, as well as motion maps.

MDL constrained 3-D grayscale skeletonization algorithm for automated extraction of dendrites and spines from fluorescence confocal images

This paper presents a method for improved automatic delineation of dendrites and spines from three-dimensional (3-D) images of neurons acquired by confocal or multi-photon fluorescence microscopy. The core advance presented here is a direct grayscale skeletonization algorithm that is constrained by a structural complexity penalty using the minimum description length (MDL) principle, and additional neuroanatomy-specific constraints. The 3-D skeleton is extracted directly from the grayscale image data, avoiding errors introduced by image binarization. The MDL method achieves a practical tradeoff between the complexity of the skeleton and its coverage of the fluorescence signal. Additional advances include the use of 3-D spline smoothing of dendrites to improve spine detection, and graph-theoretic algorithms to explore and extract the dendritic structure from the grayscale skeleton using an intensity-weighted minimum spanning tree (IW-MST) algorithm. This algorithm was evaluated on 30 datasets organized in 8 groups from multiple laboratories. Spines were detected with false negative rates less than 10% on most datasets (the average is 7.1%), and the average false positive rate was 11.8%. The software is available in open source form.

Automatic summarization of changes in biological image sequences using algorithmic information theory

An algorithmic information-theoretic method is presented for object-level summarization of meaningful changes in image sequences. Object extraction and tracking data are represented as an attributed tracking graph (ATG). Time courses of object states are compared using an adaptive information distance measure, aided by a closed-form multidimensional quantization. The notion of meaningful summarization is captured by using the gap statistic to estimate the randomness deficiency from algorithmic statistics. The summary is the clustering result and feature subset that maximize the gap statistic. This approach was validated on four bioimaging applications: 1) It was applied to a synthetic data set containing two populations of cells differing in the rate of growth, for which it correctly identified the two populations and the single feature out of 23 that separated them; 2) it was applied to 59 movies of three types of neuroprosthetic devices being inserted in the brain tissue at three speeds each, for which it correctly identified insertion speed as the primary factor affecting tissue strain; 3) when applied to movies of cultured neural progenitor cells, it correctly distinguished neurons from progenitors without requiring the use of a fixative stain; and 4) when analyzing intracellular molecular transport in cultured neurons undergoing axon specification, it automatically confirmed the role of kinesins in axon specification.

Highly undersampled magnetic resonance image reconstruction via homotopic l(0) -minimization

In clinical magnetic resonance imaging (MRI), any reduction in scan time offers a number of potential benefits ranging from high-temporal-rate observation of physiological processes to improvements in patient comfort. Following recent developments in compressive sensing (CS) theory, several authors have demonstrated that certain classes of MR images which possess sparse representations in some transform domain can be accurately reconstructed from very highly undersampled K-space data by solving a convex l(1) -minimization problem. Although l(1)-based techniques are extremely powerful, they inherently require a degree of over-sampling above the theoretical minimum sampling rate to guarantee that exact reconstruction can be achieved. In this paper, we propose a generalization of the CS paradigm based on homotopic approximation of the l(0) quasi-norm and show how MR image reconstruction can be pushed even further below the Nyquist limit and significantly closer to the theoretical bound. Following a brief review of standard CS methods and the developed theoretical extensions, several example MRI reconstructions from highly undersampled K-space data are presented.

Nonlinear scale space with spatially varying stopping time

A general scale space algorithm is presented for denoising signals and images with spatially varying dominant scales. The process is formulated as a partial differential equation with spatially varying time. The proposed adaptivity is semi-local and is in conjunction with the classical gradient-based diffusion coefficient, designed to preserve edges. The new algorithm aims at maximizing a local SNR measure of the denoised image. It is based on a generalization of a global stopping time criterion presented recently by the author and colleagues. Most notably, the method works well also for partially textured images and outperforms any selection of a global stopping time. Given an estimate of the noise variance, the procedure is automatic and can be applied well to most natural images.

Automatic microarray spot segmentation using a Snake-Fisher model

Inspired by Paragious and Deriche's work, which unifies boundary-based and region-based image partition approaches, we integrate the snake model and the Fisher criterion to capture, respectively, the boundary information and region information of microarray images. We then use the proposed algorithm to segment the spots in the microarray images, and compare our results with those obtained by commercial software. Our algorithm is automatic because the parameters are adaptively estimated from the data without human intervention.

Volume segmentation and reconstruction from freehand three-dimensional ultrasound data with application to ovarian follicle measurement

This article presents a semi-automatic method for segmentation and reconstruction of freehand three-dimensional (3D) ultrasound data. The method incorporates a number of interesting features within the level-set framework: First, segmentation is carried out using region competition, requiring multiple distinct and competing regions to be encoded within the framework. This region competition uses a simple dot-product based similarity measure to compare intensities within each region. In addition, segmentation and surface reconstruction is performed within the 3D domain to take advantage of the additional spatial information available. This means that the method must interpolate the surface where there are gaps in the data, a feature common to freehand 3D ultrasound reconstruction. Finally, although the level-set method is restricted to a voxel grid, no assumption is made that the data being segmented will conform to this grid and may be segmented in its world-reference position. The volume reconstruction method is demonstrated in vivo for the volume measurement of ovarian follicles. The 3D reconstructions produce a lower error variance than the current clinical measurement based on a mean diameter estimated from two-dimensional (2D) images. However, both the clinical measurement and the semi-automatic method appear to underestimate the true follicular volume.

A stochastic continuation approach to piecewise constant reconstruction

We address the problem of reconstructing a piecewise constant 3-D object from a few noisy 2-D line-integral projections. More generally, the theory developed here readily applies to the recovery of an ideal n-D signal (n > or =1) from indirect measurements corrupted by noise. Stabilization of this ill-conditioned inverse problem is achieved with the Potts prior model, which leads to a challenging optimization task. To overcome this difficulty, we introduce a new class of hybrid algorithms that combines simulated annealing with deterministic continuation. We call this class of algorithms stochastic continuation (SC). We first prove that, under mild assumptions, SC inherits the finite-time convergence properties of generalized simulated annealing. Then, we show that SC can be successfully applied to our reconstruction problem. In addition, we look into the concave distortion acceleration method introduced for standard simulated annealing and we derive an explicit formula for choosing the free parameter of the cost function. Numerical experiments using both synthetic data and real radiographic testing data show that SC outperforms standard simulated annealing.

Optical flow 3D segmentation and interpretation: a variational method with active curve evolution and level sets

This study investigates a variational, active curve evolution method for dense three-dimentional (3D) segmentation and interpretation of optical flow in an image sequence of a scene containing moving rigid objects viewed by a possibly moving camera. This method jointly performs 3D motion segmentation, 3D interpretation (recovery of 3D structure and motion), and optical flow estimation. The objective functional contains two data terms for each segmentation region, one based on the motion-only equation which relates the essential parameters of 3D rigid body motion to optical flow, and the other on the Horn and Schunck optical flow constraint. It also contains two regularization terms for each region, one for optical flow, the other for the region boundary. The necessary conditions for a minimum of the functional result in concurrent 3D-motion segmentation, by active curve evolution via level sets, and linear estimation of each region essential parameters and optical flow. Subsequently, the screw of 3D motion and regularized relative depth are recovered analytically for each region from the estimated essential parameters and optical flow. Examples are provided which verify the method and its implementation.

Minimal stochastic complexity image partitioning with unknown noise model

We present a generalization of a new statistical technique of image partitioning into homogeneous regions to cases where the family of the probability laws of the gray-level fluctuations is a priori unknown. For that purpose, the probability laws are described with step functions whose parameters are estimated. This approach is based on a polygonal grid which can have an arbitrary topology and whose number of regions and regularity of its boundaries are obtained by minimizing the stochastic complexity of the image. We demonstrate that efficient homogeneous image partitioning can be obtained when no parametric model of the probability laws of the gray levels is used and that this approach leads to a criterion without parameter to be tuned by the user. The efficiency of this technique is compared to a statistical parametric technique on a synthetic image and is compared to a standard unsupervised segmentation method on real optical images.

Level set segmentation with multiple regions

The popularity of level sets for segmentation is mainly based on the sound and convenient treatment of regions and their boundaries. Unfortunately, this convenience is so far not known from level set methods when applied to images with more than two regions. This communication introduces a comparatively simple way how to extend active contours to multiple regions keeping the familiar quality of the two-phase case. We further suggest a strategy to determine the optimum number of regions as well as initializations for the contours.

B-spline image model for energy minimization-based optical flow estimation

Robust estimation of the optical flow is addressed through a multiresolution energy minimization. It involves repeated evaluation of spatial and temporal gradients of image intensity which rely usually on bilinear interpolation and image filtering. We propose to base both computations on a single pyramidal cubic B-spline model of image intensity. We show empirically improvements in convergence speed and estimation error and validate the resulting algorithm on real test sequences.

Binocular shape constancy from novel views: The role of a priori constraints

We tested shape constancy from novel views in the case of binocular viewing, using a variety of stimuli, including polyhedra, polygonal lines, and points in 3-D. The results of the psychophysical experiments show that constraints such as planarity of surface contours and symmetry are critical for reliable shape constancy. These results are consistent with the results obtained in our previous psychophysical experiments on shape constancy from novel views in the presence of a kinetic depth effect (Pizlo & Stevenson, 1999). On the basis of these results, we developed a new model of binocular shape reconstruction. The model is based on the assumption that binocular reconstruction is a difficult inverse problem, whose solution requires imposing a priori constraints on the family of possible interpretations. In the model, binocular disparity is used to correct monocularly reconstructed shape. The new model was tested on the same shapes as those used in the psychophysical experiments. The reconstructions produced by this model are substantially more reliable than the reconstructions produced by models that do not use constraints. Interestingly, monocular (but not binocular) reconstructions produced by this model correlate well with both monocular and binocular performance of human subjects. This fact suggests that binocular and monocular reconstructions of shapes in the human visual system involve similar mechanisms based on monocular shape constraints.

DTI segmentation by statistical surface evolution

We address the problem of the segmentation of cerebral white matter structures from diffusion tensor images (DTI). A DTI produces, from a set of diffusion-weighted MR images, tensor-valued images where each voxel is assigned with a 3 x 3 symmetric, positive-definite matrix. This second order tensor is simply the covariance matrix of a local Gaussian process, with zero-mean, modeling the average motion of water molecules. As we will show in this paper, the definition of a dissimilarity measure and statistics between such quantities is a nontrivial task which must be tackled carefully. We claim and demonstrate that, by using the theoretically well-founded differential geometrical properties of the manifold of multivariate normal distributions, it is possible to improve the quality of the segmentation results obtained with other dissimilarity measures such as the Euclidean distance or the Kullback-Leibler divergence. The main goal of this paper is to prove that the choice of the probability metric, i.e., the dissimilarity measure, has a deep impact on the tensor statistics and, hence, on the achieved results. We introduce a variational formulation, in the level-set framework, to estimate the optimal segmentation of a DTI according to the following hypothesis: Diffusion tensors exhibit a Gaussian distribution in the different partitions. We must also respect the geometric constraints imposed by the interfaces existing among the cerebral structures and detected by the gradient of the DTI. We show how to express all the statistical quantities for the different probability metrics. We validate and compare the results obtained on various synthetic data-sets, a biological rat spinal cord phantom and human brain DTIs.

Bayesian image segmentation using local iso-intensity structural orientation

Image segmentation is a fundamental problem in early computer vision. In segmentation of flat shaded, nontextured objects in real-world images, objects are usually assumed to be piecewise homogeneous. This assumption, however, is not always valid with images such as medical images. As a result, any techniques based on this assumption may produce less-than-satisfactory image segmentation. In this work, we relax the piecewise homogeneous assumption. By assuming that the intensity nonuniformity is smooth in the imaged objects, a novel algorithm that exploits the coherence in the intensity profile to segment objects is proposed. The algorithm uses a novel smoothness prior to improve the quality of image segmentation. The formulation of the prior is based on the coherence of the local structural orientation in the image. The segmentation process is performed in a Bayesian framework. Local structural orientation estimation is obtained with an orientation tensor. Comparisons between the conventional Hessian matrix and the orientation tensor have been conducted. The experimental results on the synthetic images and the real-world images have indicated that our novel segmentation algorithm produces better segmentations than both the global thresholding with the maximum likelihood estimation and the algorithm with the multilevel logistic MRF model.

Automatic selection of parameters for vessel/neurite segmentation algorithms

An automated method is presented for selecting optimal parameter settings for vessel/neurite segmentation algorithms using the minimum description length principle and a recursive random search algorithm. It trades off a probabilistic measure of image-content coverage against its conciseness. It enables nonexpert users to select parameter settings objectively, without knowledge of underlying algorithms, broadening the applicability of the segmentation algorithm, and delivering higher morphometric accuracy. It enables adaptation of parameters across batches of images. It simplifies the user interface to just one optional parameter and reduces the cost of technical support. Finally, the method is modular, extensible, and amenable to parallel computation. The method is applied to 223 images of human retinas and cultured neurons, from four different sources, using a single segmentation algorithm with eight parameters. Improvements in segmentation quality compared to default settings using 1000 iterations ranged from 4.7%-21%. Paired t-tests showed that improvements are statistically significant (p < 0.0005). Most of the improvement occurred in the first 44 iterations. Improvements in description lengths and agreement with the ground truth were strongly correlated (p = 0.78).

Minimal stochastic complexity snake-based technique adapted to an unknown noise model

We propose a polygonal snake segmentation technique adapted to objects that can be composed of several regions with gray-level fluctuations described by a priori unknown probability laws. This approach is based on a histogram equalization and on the minimization of a criterion without parameter to be tuned by the user. We demonstrate the efficiency of this approach, which has low computational cost, on synthetic and real images perturbed by different types of optical noise.