Stochastic stereo matching over scale

Adaptive Superpixel-Based Disparity Estimation Algorithm Using Plane Information and Disparity Refining Mechanism in Stereo Matching

The motivation of this paper is to address the limitations of the conventional keypoint-based disparity estimation methods. Conventionally, disparity estimation is usually based on the local information of keypoints. However, keypoints may distribute sparsely in the smooth region, and keypoints with the same descriptors may appear in a symmetric pattern. Therefore, conventional keypoint-based disparity estimation methods may have limited performance in smooth and symmetric regions. The proposed algorithm is superpixel-based. Instead of performing keypoint matching, both keypoint and semiglobal information are applied to determine the disparity in the proposed algorithm. Since the local information of keypoints and the semi-global information of the superpixel are both applied, the accuracy of disparity estimation can be improved, especially for smooth and symmetric regions. Moreover, to address the non-uniform distribution problem of keypoints, a disparity refining mechanism based on the similarity and the distance of neighboring superpixels is applied to correct the disparity of the superpixel with no or few keypoints. The experiments show that the disparity map generated by the proposed algorithm has a lower matching error rate than that generated by other methods.

GA-Net-Pyramid: An Efficient End-to-End Network for Dense Matching

Dense matching plays a crucial role in computer vision and remote sensing, to rapidly provide stereo products using inexpensive hardware. Along with the development of deep learning, the Guided Aggregation Network (GA-Net) achieves state-of-the-art performance via the proposed Semi-Global Guided Aggregation layers and reduces the use of costly 3D convolutional layers. To solve the problem of GA-Net requiring large GPU memory consumption, we design a pyramid architecture to modify the model. Starting from a downsampled stereo input, the disparity is estimated and continuously refined through the pyramid levels. Thus, the disparity search is only applied for a small size of stereo pair and then confined within a short residual range for minor correction, leading to highly reduced memory usage and runtime. Tests on close-range, aerial, and satellite data demonstrate that the proposed algorithm achieves significantly higher efficiency (around eight times faster consuming only 20–40% GPU memory) and comparable results with GA-Net on remote sensing data. Thanks to this coarse-to-fine estimation, we successfully process remote sensing datasets with very large disparity ranges, which could not be processed with GA-Net due to GPU memory limitations.

Seamless images stitching for 3D human models

Realistic 3D human model reconstruction is an important component in computer graphics and computer vision. In particular, texturing on the surface of models is a key stage of reconstruction. In this paper, we dispose the texture mapping on the model’s surface as an optimization of image stitching, and present an effective method to generate a seamless, integrated and smooth texture on the surface of 3D human model. First, we build a corresponding Markov Random Field model with respect to color images and triangular meshes of the surface. On the basis of \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\alpha$$\end{document}α-expansion optimization for this Markov Random Field model, a 2D translation coordinate of color image, as an adaptive iterative factor, is introduced into the optimization to match the color content at the boundary of adjacent meshes. That compensates for the misalignment of adjacent color images, which caused by the inaccuracy of depth data and multi-view misregistration. Then we apply Poisson blending to a composite vector field in gradient domain, to resolve the small but noticeable illumination variations between different color images. To repair the blank regions, we parameterize the model’s surface and project it onto a 2D plane. Then the K-Nearest Neighbor algorithm is applied to fill up the blank regions with texture contents. Finally, we evaluate our method by comparison with another three advanced methods on some human models, and the results demonstrate that our method of images stitching creates a best texture on the surface of 3D human model both in visual effect and quantitative analysis.

Energy minimization in medical image analysis: Methodologies and applications

Energy minimization is of particular interest in medical image analysis. In the past two decades, a variety of optimization schemes have been developed. In this paper, we present a comprehensive survey of the state-of-the-art optimization approaches. These algorithms are mainly classified into two categories: continuous method and discrete method. The former includes Newton-Raphson method, gradient descent method, conjugate gradient method, proximal gradient method, coordinate descent method, and genetic algorithm-based method, while the latter covers graph cuts method, belief propagation method, tree-reweighted message passing method, linear programming method, maximum margin learning method, simulated annealing method, and iterated conditional modes method. We also discuss the minimal surface method, primal-dual method, and the multi-objective optimization method. In addition, we review several comparative studies that evaluate the performance of different minimization techniques in terms of accuracy, efficiency, or complexity. These optimization techniques are widely used in many medical applications, for example, image segmentation, registration, reconstruction, motion tracking, and compressed sensing. We thus give an overview on those applications as well.

Disparity Estimation on Stereo Mammograms

We consider the problem of depth estimation on digital stereo mammograms. Being able to elucidate 3D information from stereo mammograms is an important precursor to conducting 3D digital analysis of data from this promising new modality. The problem is generally much harder than the classic stereo matching problem on visible light images of the natural world, since nearly all of the 3D structural information of interest exists as complex network of multilayered, heavily occluded curvilinear structures. Toward addressing this difficult problem, we formulate a new stereo model that minimizes a global energy functional to densely estimate disparity on stereo mammogram images, by introducing a new singularity index as a constraint to obtain better estimates of disparity along critical curvilinear structures. Curvilinear structures, such as vasculature and spicules, are particularly salient structures in the breast, and being able to accurately position them in 3D is a valuable goal. Experiments on synthetic images with known ground truth and on real stereo mammograms highlight the advantages of the proposed stereo model over the canonical stereo model.

Joint reconstruction of multiview compressed images

Distributed representation of correlated multiview images is an important problem that arises in vision sensor networks. This paper concentrates on the joint reconstruction problem where the distributively compressed images are decoded together in order to take benefit from the image correlation. We consider a scenario where the images captured at different viewpoints are encoded independently using common coding solutions (e.g., JPEG) with a balanced rate distribution among different cameras. A central decoder first estimates the inter-view image correlation from the independently compressed data. The joint reconstruction is then cast as a constrained convex optimization problem that reconstructs total-variation (TV) smooth images, which comply with the estimated correlation model. At the same time, we add constraints that force the reconstructed images to be as close as possible to their compressed versions. We show through experiments that the proposed joint reconstruction scheme outperforms independent reconstruction in terms of image quality, for a given target bit rate. In addition, the decoding performance of our algorithm compares advantageously to state-of-the-art distributed coding schemes based on motion learning and on the DISCOVER algorithm.

Dynamic programming and graph algorithms in computer vision

Optimization is a powerful paradigm for expressing and solving problems in a wide range of areas, and has been successfully applied to many vision problems. Discrete optimization techniques are especially interesting since, by carefully exploiting problem structure, they often provide nontrivial guarantees concerning solution quality. In this paper, we review dynamic programming and graph algorithms, and discuss representative examples of how these discrete optimization techniques have been applied to some classical vision problems. We focus on the low-level vision problem of stereo, the mid-level problem of interactive object segmentation, and the high-level problem of model-based recognition.

Efficient stereoscopic ranging via stochastic sampling of match quality

We present an efficient method that computes dense stereo correspondences by stochastically sampling match quality values. Nonexhaustive sampling facilitates the use of quality metrics that take unique values at noninteger disparities. Depth estimates are iteratively refined with a stochastic cooperative search by perturbing the estimates, sampling match quality, and reweighting and aggregating the perturbations. The approach gains significant efficiencies when applied to video, where initial estimates are seeded using information from the previous pair in a novel application of the Z-buffering algorithm. This significantly reduces the number of search iterations required. We present a quantitative accuracy evaluation wherein the proposed method outperforms a microcanonical annealing approach by Barnard and a cooperative approach by Zitnick and Kanade , while using fewer match quality evaluations than either. The approach is shown to have more attractive memory usage and scaling than alternatives based on exhaustive sampling.

A comparative study of energy minimization methods for Markov random fields with smoothness-based priors

Among the most exciting advances in early vision has been the development of efficient energy minimization algorithms for pixel-labeling tasks such as depth or texture computation. It has been known for decades that such problems can be elegantly expressed as Markov random fields, yet the resulting energy minimization problems have been widely viewed as intractable. Recently, algorithms such as graph cuts and loopy belief propagation (LBP) have proven to be very powerful: for example, such methods form the basis for almost all the top-performing stereo methods. However, the tradeoffs among different energy minimization algorithms are still not well understood. In this paper we describe a set of energy minimization benchmarks and use them to compare the solution quality and running time of several common energy minimization algorithms. We investigate three promising recent methods graph cuts, LBP, and tree-reweighted message passing in addition to the well-known older iterated conditional modes (ICM) algorithm. Our benchmark problems are drawn from published energy functions used for stereo, image stitching, interactive segmentation, and denoising. We also provide a general-purpose software interface that allows vision researchers to easily switch between optimization methods. Benchmarks, code, images, and results are available at http://vision.middlebury.edu/MRF/.

What energy functions can be minimized via graph cuts?

In the last few years, several new algorithms based on graph cuts have been developed to solve energy minimization problems in computer vision. Each of these techniques constructs a graph such that the minimum cut on the graph also minimizes the energy. Yet, because these graph constructions are complex and highly specific to a particular energy function, graph cuts have seen limited application to date. In this paper, we give a characterization of the energy functions that can be minimized by graph cuts. Our results are restricted to functions of binary variables. However, our work generalizes many previous constructions and is easily applicable to vision problems that involve large numbers of labels, such as stereo, motion, image restoration, and scene reconstruction. We give a precise characterization of what energy functions can be minimized using graph cuts, among the energy functions that can be written as a sum of terms containing three or fewer binary variables. We also provide a general-purpose construction to minimize such an energy function. Finally, we give a necessary condition for any energy function of binary variables to be minimized by graph cuts. Researchers who are considering the use of graph cuts to optimize a particular energy function can use our results to determine if this is possible and then follow our construction to create the appropriate graph. A software implementation is freely available.

Stereoscopic ranging by matching image modulations

We apply an AM-FM surface albedo model to analyze the projection of surface patterns viewed through a binocular camera system. This is used to support the use of modulation-based stereo matching where local image phase is used to compute stereo disparities. The local image phase is an advantageous feature for image matching, since the problem of computing disparities reduces to identifying local phase shifts between the stereoscopic image data. Local phase shifts, however, are problematic at high frequencies due to phase wrapping when disparities exceed +/-pi. We meld powerful multichannel Gabor image demodulation techniques for multiscale (coarse-to-fine) computation of local image phase with a disparity channel model for depth computation. The resulting framework unifies phase-based matching approaches with AM-FM surface/image models. We demonstrate the concepts in a stereo algorithm that generates a dense, accurate disparity map without the problems associated with phase wrapping.

Multiresolution 3-D range segmentation using focus cues

This paper describes a novel system for computing a three-dimensional (3-D) range segmentation of an arbitrary visible scene using focus information. The process of range segmentation is divided into three steps: an initial range classification, a surface merging process, and a 3-D multiresolution range segmentation. First, range classification is performed to obtain quantized range estimates. The range classification is performed by analyzing focus cues within a Bayesian estimation framework. A combined energy functional measures the degree of focus and the Gibbs distribution of the class field. The range classification provides an initial range segmentation. Second, a statistical merging process is performed to merge the initial surface segments. This gives a range segmentation at a coarse resolution. Third, 3-D multiresolution range segmentation (3-D MRS) is performed to refine the range segmentation into finer resolutions. The proposed range segmentation method does not require initial depth estimates, it allows the analysis of scenes containing multiple objects, and it provides a rich description of the 3-D structure of a scene.