1
|
Hu Y, Duan M, Jin Y, Zhu C, Chen E, Xu C. Shading-based absolute phase unwrapping. OPTICS LETTERS 2021; 46:1955-1958. [PMID: 33857115 DOI: 10.1364/ol.419366] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Received: 01/14/2021] [Accepted: 03/19/2021] [Indexed: 06/12/2023]
Abstract
Absolute phase unwrapping in the phase-shifting profilometry (PSP) is significant for dynamic 3-D measurements over a large depth range. Among traditional phase unwrapping methods, spatial phase unwrapping can only retrieve a relative phase map, and temporal phase unwrapping requires auxiliary projection sequences. We propose a shading-based absolute phase unwrapping (SAPU) framework for in situ 3-D measurements without additional projection patterns. First, the wrapped phase map is calculated from three captured images. Then, the continuous relative phase map is obtained using the phase histogram check (PHC), from which the absolute phase map candidates are derived with different fringe orders. Finally, the correct absolute phase map candidate can be determined without additional patterns or spatial references by applying the shading matching check (SMC). The experimental results demonstrate the validity of the proposed method.
Collapse
|
2
|
Hao Y, Visentini-Scarzanella M, Li J, Zhang P, Ciuti G, Dario P, Huang Q. Light source position calibration method for photometric stereo in capsule endoscopy. Adv Robot 2020. [DOI: 10.1080/01691864.2020.1757203] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/24/2022]
Affiliation(s)
- Yang Hao
- Intelligent Robotics Institute, School of Mechatronical Engineering, Beijing Institute of Technology, Beijing, People's Republic of China
| | | | - Jing Li
- Beijing Advanced Innovation Center for Intelligent Robots and Systems, Beijing Institute of Technology, Beijing, People's Republic of China
| | - Peisen Zhang
- Intelligent Robotics Institute, School of Mechatronical Engineering, Beijing Institute of Technology, Beijing, People's Republic of China
| | - Gastone Ciuti
- Beijing Advanced Innovation Center for Intelligent Robots and Systems, Beijing Institute of Technology, Beijing, People's Republic of China
- The BioRobotics Institute, Scuola Superiore Sant'Anna, Pisa, Italy
- Department of Excellence in Robotics & AI, Scuola Superiore Sant'Anna, Pisa, Italy
| | - Paolo Dario
- Beijing Advanced Innovation Center for Intelligent Robots and Systems, Beijing Institute of Technology, Beijing, People's Republic of China
- The BioRobotics Institute, Scuola Superiore Sant'Anna, Pisa, Italy
- Department of Excellence in Robotics & AI, Scuola Superiore Sant'Anna, Pisa, Italy
| | - Qiang Huang
- Intelligent Robotics Institute, School of Mechatronical Engineering, Beijing Institute of Technology, Beijing, People's Republic of China
- Beijing Advanced Innovation Center for Intelligent Robots and Systems, Beijing Institute of Technology, Beijing, People's Republic of China
| |
Collapse
|
3
|
Xing G, Liu Y, Ling H, Granier X, Zhang Y. Automatic Spatially Varying Illumination Recovery of Indoor Scenes Based on a Single RGB-D Image. IEEE TRANSACTIONS ON VISUALIZATION AND COMPUTER GRAPHICS 2020; 26:1672-1685. [PMID: 30371374 DOI: 10.1109/tvcg.2018.2876541] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/08/2023]
Abstract
We propose an automatic framework to recover the illumination of indoor scenes based on a single RGB-D image. Unlike previous works, our method can recover spatially varying illumination without using any lighting capturing devices or HDR information. The recovered illumination can produce realistic rendering results. To model the geometry of the visible and invisible parts of scenes corresponding to the input RGB-D image, we assume that all objects shown in the image are located in a box with six faces and build a planar-based geometry model based on the input depth map. We then present a confidence-scoring based strategy to separate the light sources from the highlight areas. The positions of light sources both in and out of the camera's view are calculated based on the classification result and the recovered geometry model. Finally, an iterative procedure is proposed to calculate the colors of light sources and the materials in the scene. In addition, a data-driven method is used to set constraints on the light source intensities. Using the estimated light sources and geometry model, environment maps at different points in the scene are generated that can model the spatial variance of illumination. The experimental results demonstrate the validity and flexibility of our approach.
Collapse
|
4
|
Santo H, Waechter M, Lin WY, Sugano Y, Matsushita Y. Light Structure from Pin Motion: Geometric Point Light Source Calibration. Int J Comput Vis 2020. [DOI: 10.1007/s11263-020-01312-3] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/24/2022]
Abstract
AbstractWe present a method for geometric point light source calibration. Unlike prior works that use Lambertian spheres, mirror spheres, or mirror planes, we use a calibration target consisting of a plane and small shadow casters at unknown positions above the plane. We show that shadow observations from a moving calibration target under a fixed light follow the principles of pinhole camera geometry and epipolar geometry, allowing joint recovery of the light position and 3D shadow caster positions, equivalent to how conventional structure from motion jointly recovers camera parameters and 3D feature positions from observed 2D features. Moreover, we devised a unified light model that works with nearby point lights as well as distant light in one common framework. Our evaluation shows that our method yields light estimates that are stable and more accurate than existing techniques while having a much simpler setup and requiring less manual labor.
Collapse
|
5
|
Configurable 3D Scene Synthesis and 2D Image Rendering with Per-pixel Ground Truth Using Stochastic Grammars. Int J Comput Vis 2018. [DOI: 10.1007/s11263-018-1103-5] [Citation(s) in RCA: 17] [Impact Index Per Article: 2.8] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/28/2022]
|
6
|
Karaoglu S, Gevers T, Smeulders AWM. Point Light Source Position Estimation From RGB-D Images by Learning Surface Attributes. IEEE TRANSACTIONS ON IMAGE PROCESSING : A PUBLICATION OF THE IEEE SIGNAL PROCESSING SOCIETY 2017; 26:5149-5159. [PMID: 28749351 DOI: 10.1109/tip.2017.2731619] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.1] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/07/2023]
Abstract
Light source position (LSP) estimation is a difficult yet an important problem in computer vision. A common approach for estimating the LSP assumes Lambert's law. However, in real-world scenes, Lambert's law does not hold for all different types of surfaces. Instead of assuming all that surfaces follow Lambert's law, our approach classifies image surface segments based on their photometric and geometric surface attributes (i.e. glossy, matte, curved, and so on) and assigns weights to image surface segments based on their suitability for LSP estimation. In addition, we propose the use of the estimated camera pose to globally constrain LSP for RGB-D video sequences. Experiments on Boom and a newly collected RGB-D video data sets show that the state-of-the-art methods are outperformed by the proposed method. The results demonstrate that weighting image surface segments based on their attributes outperform the state-of-the-art methods in which the image surface segments are considered to equally contribute. In particular, by using the proposed surface weighting, the angular error for LSP estimation is reduced from 12.6° to 8.2° and 24.6° to 4.8° for Boom and RGB-D video data sets, respectively. Moreover, using the camera pose to globally constrain LSP provides higher accuracy (4.8°) compared with using single frames (8.5°).
Collapse
|
7
|
Elizondo DA, Zhou SM, Chrysostomou C. Light source detection for digital images in noisy scenes: a neural network approach. Neural Comput Appl 2017. [DOI: 10.1007/s00521-016-2281-0] [Citation(s) in RCA: 4] [Impact Index Per Article: 0.6] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/30/2022]
|
8
|
Nunes ALP, Maciel A, Cavazzola LT, Walter M. A laparoscopy-based method for BRDF estimation from in vivo human liver. Med Image Anal 2016; 35:620-632. [PMID: 27728873 DOI: 10.1016/j.media.2016.09.005] [Citation(s) in RCA: 3] [Impact Index Per Article: 0.4] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/04/2015] [Revised: 09/09/2016] [Accepted: 09/16/2016] [Indexed: 01/07/2023]
Abstract
While improved visual realism is known to enhance training effectiveness in virtual surgery simulators, the advances on realistic rendering for these simulators is slower than similar simulations for man-made scenes. One of the main reasons for this is that in vivo data is hard to gather and process. In this paper, we propose the analysis of videolaparoscopy data to compute the Bidirectional Reflectance Distribution Function (BRDF) of living organs as an input to physically based rendering algorithms. From the interplay between light and organic matter recorded in video images, we propose the definition of a process capable of establishing the BRDF for inside-the-body organic surfaces. We present a case study around the liver with patient-specific rendering under global illumination. Results show that despite the limited range of motion allowed within the body, the computed BRDF presents a high-coverage of the sampled regions and produces plausible renderings.
Collapse
Affiliation(s)
- A L P Nunes
- Institute of Informatics, Federal University of Rio Grande do Sul, Porto Alegre, Brazil; Federal Institute of Paraná, Londrina, Brazil.
| | - A Maciel
- Institute of Informatics, Federal University of Rio Grande do Sul, Porto Alegre, Brazil
| | - L T Cavazzola
- Institute of Health Sciences, Federal University of Rio Grande do Sul, Porto Alegre, Brazil
| | - M Walter
- Institute of Informatics, Federal University of Rio Grande do Sul, Porto Alegre, Brazil
| |
Collapse
|
9
|
|
10
|
Yi J, Mao X, Chen L, Xue Y, Compare A. Illuminant direction estimation for a single image based on local region complexity analysis and average gray value. APPLIED OPTICS 2014; 53:226-236. [PMID: 24514054 DOI: 10.1364/ao.53.000226] [Citation(s) in RCA: 3] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Received: 09/17/2013] [Accepted: 12/03/2013] [Indexed: 06/03/2023]
Abstract
Illuminant direction estimation is an important research issue in the field of image processing. Due to low cost for getting texture information from a single image, it is worthwhile to estimate illuminant direction by employing scenario texture information. This paper proposes a novel computation method to estimate illuminant direction on both color outdoor images and the extended Yale face database B. In our paper, the luminance component is separated from the resized YCbCr image and its edges are detected with the Canny edge detector. Then, we divide the binary edge image into 16 local regions and calculate the edge level percentage in each of them. Afterward, we use the edge level percentage to analyze the complexity of each local region included in the luminance component. Finally, according to the error function between the measured intensity and the calculated intensity, and the constraint function for an infinite light source model, we calculate the illuminant directions of the luminance component's three local regions, which meet the requirements of lower complexity and larger average gray value, and synthesize them as the final illuminant direction. Unlike previous works, the proposed method requires neither all of the information of the image nor the texture that is included in the training set. Experimental results show that the proposed method works better at the correct rate and execution time than the existing ones.
Collapse
|
11
|
Kawakami R, Zhao H, Tan RT, Ikeuchi K. Camera Spectral Sensitivity and White Balance Estimation from Sky Images. Int J Comput Vis 2013. [DOI: 10.1007/s11263-013-0632-1] [Citation(s) in RCA: 28] [Impact Index Per Article: 2.5] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/26/2022]
|
12
|
Panagopoulos A, Wang C, Samaras D, Paragios N. Simultaneous cast shadows, illumination and geometry inference using hypergraphs. IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE 2013; 35:437-449. [PMID: 22585101 DOI: 10.1109/tpami.2012.110] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 05/31/2023]
Abstract
The cast shadows in an image provide important information about illumination and geometry. In this paper, we utilize this information in a novel framework in order to jointly recover the illumination environment, a set of geometry parameters, and an estimate of the cast shadows in the scene given a single image and coarse initial 3D geometry. We model the interaction of illumination and geometry in the scene and associate it with image evidence for cast shadows using a higher order Markov Random Field (MRF) illumination model, while we also introduce a method to obtain approximate image evidence for cast shadows. Capturing the interaction between light sources and geometry in the proposed graphical model necessitates higher order cliques and continuous-valued variables, which make inference challenging. Taking advantage of domain knowledge, we provide a two-stage minimization technique for the MRF energy of our model. We evaluate our method in different datasets, both synthetic and real. Our model is robust to rough knowledge of geometry and inaccurate initial shadow estimates, allowing a generic coarse 3D model to represent a whole class of objects for the task of illumination estimation, or the estimation of geometry parameters to refine our initial knowledge of scene geometry, simultaneously with illumination estimation.
Collapse
|
13
|
Moreno-Noguer F, Fua P. Stochastic exploration of ambiguities for nonrigid shape recovery. IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE 2013; 35:463-475. [PMID: 22547426 DOI: 10.1109/tpami.2012.102] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.2] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 05/31/2023]
Abstract
Recovering the 3D shape of deformable surfaces from single images is known to be a highly ambiguous problem because many different shapes may have very similar projections. This is commonly addressed by restricting the set of possible shapes to linear combinations of deformation modes and by imposing additional geometric constraints. Unfortunately, because image measurements are noisy, such constraints do not always guarantee that the correct shape will be recovered. To overcome this limitation, we introduce a stochastic sampling approach to efficiently explore the set of solutions of an objective function based on point correspondences. This allows us to propose a small set of ambiguous candidate 3D shapes and then use additional image information to choose the best one. As a proof of concept, we use either motion or shading cues to this end and show that we can handle a complex objective function without having to solve a difficult nonlinear minimization problem. The advantages of our method are demonstrated on a variety of problems including both real and synthetic data.
Collapse
Affiliation(s)
- Francesc Moreno-Noguer
- Institut de Robòtica i Informàtica Industrial, CSIC-UPC, Llorens i Artigas 4-6, Barcelona 08028, Spain.
| | | |
Collapse
|
14
|
Liu Y, Granier X. Online tracking of outdoor lighting variations for augmented reality with moving cameras. IEEE TRANSACTIONS ON VISUALIZATION AND COMPUTER GRAPHICS 2012; 18:573-580. [PMID: 22402684 DOI: 10.1109/tvcg.2012.53] [Citation(s) in RCA: 3] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 05/31/2023]
Abstract
In augmented reality, one of key tasks to achieve a convincing visual appearance consistency between virtual objects and video scenes is to have a coherent illumination along the whole sequence. As outdoor illumination is largely dependent on the weather, the lighting condition may change from frame to frame. In this paper, we propose a full image-based approach for online tracking of outdoor illumination variations from videos captured with moving cameras. Our key idea is to estimate the relative intensities of sunlight and skylight via a sparse set of planar feature-points extracted from each frame. To address the inevitable feature misalignments, a set of constraints are introduced to select the most reliable ones. Exploiting the spatial and temporal coherence of illumination, the relative intensities of sunlight and skylight are finally estimated by using an optimization process. We validate our technique on a set of real-life videos and show that the results with our estimations are visually coherent along the video sequences.
Collapse
Affiliation(s)
- Yanli Liu
- College of Computer Science, Sichuan University, PR China.
| | | |
Collapse
|
15
|
Wavelet-Texture Method: Appearance Compression by Polarization, Parametric Reflection Model, and Daubechies Wavelet. Int J Comput Vis 2010. [DOI: 10.1007/s11263-009-0244-y] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/20/2022]
|
16
|
|
17
|
Xu S, Wallace A. Recovering surface reflectance and multiple light locations and intensities from image data. Pattern Recognit Lett 2008. [DOI: 10.1016/j.patrec.2008.04.007] [Citation(s) in RCA: 8] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/22/2022]
|
18
|
Marchand E. Control Camera and Light Source Positions using Image Gradient Information. ACTA ACUST UNITED AC 2007. [DOI: 10.1109/robot.2007.363822] [Citation(s) in RCA: 17] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/07/2022]
|
19
|
Boyaci H, Doerschner K, Snyder JL, Maloney LT. Surface color perception in three-dimensional scenes. Vis Neurosci 2006; 23:311-21. [PMID: 16961962 DOI: 10.1017/s0952523806233431] [Citation(s) in RCA: 28] [Impact Index Per Article: 1.6] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/10/2006] [Accepted: 03/10/2006] [Indexed: 11/06/2022]
Abstract
Researchers studying surface color perception have typically used stimuli that consist of a small number of matte patches (real or simulated) embedded in a plane perpendicular to the line of sight (a "Mondrian," Land & McCann, 1971). Reliable estimation of the color of a matte surface is a difficult if not impossible computational problem in such limited scenes (Maloney, 1999). In more realistic, three-dimensional scenes the difficulty of the problem increases, in part, because the effective illumination incident on the surface (the light field) now depends on surface orientation and location. We review recent work in multiple laboratories that examines (1) the degree to which the human visual system discounts the light field in judging matte surface lightness and color and (2) what illuminant cues the visual system uses in estimating the flow of light in a scene.
Collapse
Affiliation(s)
- Huseyin Boyaci
- Department of Psychology, University of Minnesota, Minneapolis, Minnesota 55455, USA.
| | | | | | | |
Collapse
|
20
|
Zickler T, Ramamoorthi R, Enrique S, Belhumeur PN. Reflectance sharing: predicting appearance from a sparse set of images of a known shape. IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE 2006; 28:1287-302. [PMID: 16886864 DOI: 10.1109/tpami.2006.170] [Citation(s) in RCA: 3] [Impact Index Per Article: 0.2] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 05/11/2023]
Abstract
Three-dimensional appearance models consisting of spatially varying reflectance functions defined on a known shape can be used in analysis-by-synthesis approaches to a number of visual tasks. The construction of these models requires the measurement of reflectance, and the problem of recovering spatially varying reflectance from images of known shape has drawn considerable interest. To date, existing methods rely on either: (1) low-dimensional (e.g., parametric) reflectance models, or (2) large data sets involving thousands of images (or more) per object. Appearance models based on the former have limited accuracy and generality since they require the selection of a specific reflectance model a priori, and while approaches based on the latter may be suitable for certain applications, they are generally too costly and cumbersome to be used for image analysis. We present an alternative approach that seeks to combine the benefits of existing methods by enabling the estimation of a nonparametric spatially varying reflectance function from a small number of images. We frame the problem as scattered-data interpolation in a mixed spatial and angular domain, and we present a theory demonstrating that the angular accuracy of a recovered reflectance function can be increased in exchange for a decrease in its spatial resolution. We also present a practical solution to this interpolation problem using a new representation of reflectance based on radial basis functions. This representation is evaluated experimentally by testing its ability to predict appearance under novel view and lighting conditions. Our results suggest that since reflectance typically varies slowly from point to point over much of an object's surface, we can often obtain a nonparametric reflectance function from a sparse set of images. In fact, in some cases, we can obtain reasonable results in the limiting case of only a single input image.
Collapse
Affiliation(s)
- Todd Zickler
- Division of Engineering and Applied Sciences, Harvard University, 33 Oxford St., Cambridge, MA 02138, USA.
| | | | | | | |
Collapse
|