Dynamic Refraction Stereo

Surface Normals and Shape From Water

In this paper, we introduce a novel method for reconstructing surface normals and depth of dynamic objects in water. Past shape recovery methods have leveraged various visual cues for estimating shape (e.g., depth) or surface normals. Methods that estimate both compute one from the other. We show that these two geometric surface properties can be simultaneously recovered for each pixel when the object is observed underwater. Our key idea is to leverage multi-wavelength near-infrared light absorption along different underwater light paths in conjunction with surface shading. Our method can handle both Lambertian and non-Lambertian surfaces. We derive a principled theory for this surface normals and shape from water method and a practical calibration method for determining its imaging parameters values. By construction, the method can be implemented as a one-shot imaging system. We prototype both an off-line and a video-rate imaging system and demonstrate the effectiveness of the method on a number of real-world static and dynamic objects. The results show that the method can recover intricate surface features that are otherwise inaccessible.

Use of structured light in 3D reconstruction of transparent objects

A simple non-interferometric incoherent light ray propagation model is introduced to perform three-dimensional profiling of transparent objects with typical thicknesses of the order of mm to cm by analyzing the distorted captured image behind the object. A two-dimensional cosine fringe is used as the incident reference image, whose periodicity is markedly altered by the shape of the object. By monitoring the local change in the period, the surface profile is simulated and optimized to achieve minimal error with experimental data and thus determine the final morphology. Our proposed method is simple, robust, straightforward, and single-shot, and can be used with coherent or incoherent illumination. Its feasibility for more complex applications is verified experimentally through rigorous error calculation. Moreover, the application of this technique for arbitrary transparent objects is theoretically attainable and promising.

Non-Rigid Shape From Water

We introduce a novel 3D sensing method for recovering a consistent, dense 3D shape of a dynamic, non-rigid object in water. The method reconstructs a complete (or fuller) 3D surface of the target object in a canonical frame (e.g., rest shape) as it freely deforms and moves between frames by estimating underwater 3D scene flow and using it to integrate per-frame depth estimates recovered from two near-infrared observations. The reconstructed shape is refined in the course of this global non-rigid shape recovery by leveraging both geometric and radiometric constraints. We implement our method with a single camera and a light source without the orthographic assumption on either by deriving a practical calibration method that estimates the point source position with respect to the camera. Our reconstruction method also accounts for scattering by water. We prototype a video-rate imaging system and show 3D shape reconstruction results on a number of real-world static, deformable, and dynamic objects and creatures in real-world water. The results demonstrate the effectiveness of the method in recovering complete shapes of complex, non-rigid objects in water, which opens new avenues of application for underwater 3D sensing in the sub-meter range.

Visual shape perception in the case of transparent objects

In order to estimate the shape of objects, the visual system must refer to shape-related regularities in the (retinal) image. For opaque objects, many such regularities have already been identified, but most of them cannot simply be transferred to transparent objects, because they are not available there at all or are available only in a substantially modified form. We here consider three potentially relevant regularities specific to transparent objects: optical background distortions due to refraction, changes in chromaticity and brightness due to absorption, and multiple mirror images due to specular reflection. Using computer simulations, we first analyze under which conditions these regularities may be used as shape cues. We further investigate experimentally how shape perception depends on the availability of these potential cues in realistic scenes under natural viewing conditions. Our results show that the shape of transparent objects was perceived both less accurately and less precisely than in the opaque case. Furthermore, the influence of individual image regularities varied considerably depending on the properties of both object and scene. This suggests that in the transparent case, what kind of information is usable as a shape cue depends on a complex interplay of properties of the transparent object and the surrounding scene.

Flexible method of refraction correction in vision measurement systems with multiple glass ports

Many vision measurement systems, especially in outdoor engineering, usually utilize glass ports to protect sensors against environmental influences. The refraction caused by glass ports lead to measurement errors in traditional single viewpoint model. Most existing methods only deal with the refraction that happens once, and the glass ports are required to be perpendicular to cameras or the orientations of glass ports are obtained by auxiliary equipment. This paper proposes a corrected 3D reconstruction model based on refraction geometry, which can be used for any number of glass ports with any orientations. The orientation of each glass port is obtained only using refracted and unrefracted images of the same scene, which doesn't need any auxiliary equipment. A series of validation experiments are performed. An existing image rectification method is used to make a comparison. The proposed method is also employed in a train wheelset profile measurement application, which proves that the method is effective in actual applications.

Sinusoidal Wave Estimation Using Photogrammetry and Short Video Sequences

The objective of the work is to model the shape of the sinusoidal shape of regular water waves generated in a laboratory flume. The waves are traveling in time and render a smooth surface, with no white caps or foam. Two methods are proposed, treating the water as a diffuse and specular surface, respectively. In either case, the water is presumed to take the shape of a traveling sine wave, reducing the task of the 3D reconstruction to resolve the wave parameters. The first conceived method performs the modeling part purely in 3D space. Having triangulated the points in a separate phase via bundle adjustment, a sine wave is fitted into the data in a least squares manner. The second method presents a more complete approach for the entire calculation workflow beginning in the image space. The water is perceived as a specular surface, and the traveling specularities are the only observations visible to the cameras, observations that are notably single image. The depth ambiguity is removed given additional constraints encoded within the law of reflection and the modeled parametric surface. The observation and constraint equations compose a single system of equations that is solved with the method of least squares adjustment. The devised approaches are validated against the data coming from a capacitive level sensor and on physical targets floating on the surface. The outcomes agree to a high degree.