Geometrical effects of positional errors in integral photography

Sub-pixel marking and depth-based correction methods for the elimination of voxel drifting in integral imaging display

Integral imaging is a kind of true three-dimensional (3D) display technology that uses a lens array to reconstruct vivid 3D images with full parallax and true color. In order to present a high-quality 3D image, it's vital to correct the axial position error caused by the misalignment and deformation of the lens array which makes the reconstructed lights deviate from the correct directions, resulting in severe voxel drifting and image blurring. We proposed a sub-pixel marking method to measure the axial position error of the lenses with great accuracy by addressing the sub-pixels under each lens and forming a homologous sub-pixel pair. The proposed measurement method relies on the geometric center alignment of image points, which is specifically expressed as the overlap between the test 3D voxel and the reference 3D voxel. Hence, measurement accuracy could be higher. Additionally, a depth-based sub-pixel correction method was proposed to eliminate the voxel drifting. The proposed correction method takes the voxel depth into consideration in the correction coefficient, and achieves accurate error correction for 3D images with different depths. The experimental results well confirmed that the proposed measuring and correction methods can greatly suppress the voxel drifting caused by the axial position error of the lenses, and greatly improve the 3D image quality.

Post-calibration compensation method for integral imaging system with macrolens array

Three-dimensional display with large-format is an inevitable and foreseeable trend for the future display technology, integral imaging is one of the most powerful and promising candidates to achieve this goal with full-parallax, true-color, acceptable viewing resolution and viewing angle. To obtain a 3D display with high quality, calibration is needed to correct optical misalignment and optical aberrations, which is often challenging and time consuming. We propose a post-calibration compensation method for the integral imaging system with macrolens array, the inter-lens position misalignment is corrected by forcing it to image in a regular ideal reference grid. Our method distinguishes itself from previous ones by finding the correct pixel-to-ray correspondence with a relatively simple setup and acceptable precision. A prototype is fabricated to evaluate the feasibility of the proposed method. Furthermore, the proposed method is evaluated in terms of the geometrical accuracy and quality of the reconstructed images.

Statistics-based reconstruction method with high random-error tolerance for integral imaging

A three-dimensional (3D) digital reconstruction method for integral imaging with high random-error tolerance based on statistics is proposed. By statistically analyzing the points reconstructed by triangulation from all corresponding image points in an elemental images array, 3D reconstruction with high random-error tolerance could be realized. To simulate the impacts of random errors, random offsets with different error levels are added to a different number of elemental images in simulation and optical experiments. The results of simulation and optical experiments showed that the proposed statistic-based reconstruction method has relatively stable and better reconstruction accuracy than the conventional reconstruction method. It can be verified that the proposed method can effectively reduce the impacts of random errors on 3D reconstruction of integral imaging. This method is simple and very helpful to the development of integral imaging technology.

Generic camera model and its calibration for computational integral imaging and 3D reconstruction

Integral imaging (II) is an important 3D imaging technology. To reconstruct 3D information of the viewed objects, modeling and calibrating the optical pickup process of II are necessary. This work focuses on the modeling and calibration of an II system consisting of a lenslet array, an imaging lens, and a charge-coupled device camera. Most existing work on such systems assumes a pinhole array model (PAM). In this work, we explore a generic camera model that accommodates more generality. This model is an empirical model based on measurements, and we constructed a setup for its calibration. Experimental results show a significant difference between the generic camera model and the PAM. Images of planar patterns and 3D objects were computationally reconstructed with the generic camera model. Compared with the images reconstructed using the PAM, the images present higher fidelity and preserve more high spatial frequency components. To the best of our knowledge, this is the first attempt in applying a generic camera model to an II system.

Rectification of elemental image set and extraction of lens lattice by projective image transformation in integral imaging

We propose a new method for rectifying a geometrical distortion in the elemental image set and extracting an accurate lens lattice lines by projective image transformation. The information of distortion in the acquired elemental image set is found by Hough transform algorithm. With this initial information of distortions, the acquired elemental image set is rectified automatically without the prior knowledge on the characteristics of pickup system by stratified image transformation procedure. Computer-generated elemental image sets with distortion on purpose are used for verifying the proposed rectification method. Experimentally-captured elemental image sets are optically reconstructed before and after the rectification by the proposed method. The experimental results support the validity of the proposed method with high accuracy of image rectification and lattice extraction.

Distortion-tolerant 3D recognition of underwater objects using neural networks

A method for distortion-tolerant recognition of objects in water using three-dimensional (3D) integral imaging with a neural network classification architecture is presented. Recognition algorithms are developed and experimental results are presented with rotation-variable 3D objects. To test the robustness of the system, objects are placed under a variety of water conditions, including variable Maalox-induced scattering levels and occlusion using pine needles. Neural networks have long been used for two-dimensional recognition and have recently been used for 3D digital holographic recognition. To the best of our knowledge, this is the first use of neural networks for passive 3D integral imaging and recognition of underwater objects.

Recent progress in three-dimensional information processing based on integral imaging

Recently developed integral imaging techniques are reviewed. Integral imaging captures and reproduces the light rays from the object space, enabling the acquisition and the display of the three-dimensional information of the object in an efficient way. Continuous effort on integral imaging has been improving the performance of the capture and display process in various aspects, including distortion, resolution, viewing angle, and depth range. Digital data processing of the captured light rays can now visualize the three-dimensional structure of the object with a high degree of freedom and enhanced quality. This recent progress is of high interest for both industrial applications and academic research.

Geometric analysis of spatial distortion in projection-type integral imaging

In projection-type integral imaging, positional errors in elemental images and elemental lenses affect three-dimensional (3D) image quality. We analyzed the relationships between the geometric distortion in elemental images caused by a projection lens and the spatial distortion in the reconstructed 3D image. As a result, we clarified that 3D images that were reconstructed far from the lens array were largely affected, and that the reconstructed images were significantly distorted in the depth direction at the corners of the displayed images.

Calculation of holograms from elemental images captured by integral photography

We describe a method in which holograms can be produced by calculation from images captured by integral photography (IP). We present a basic algorithm obtained by simulating IP reconstruction, in which conditions are set so as not to cause aliasing in the holograms after the calculations. To reduce the calculation load, we also propose a way to limit the range of calculation considering the distribution of light and a way to shift the optical field on the exit plane of microlenses in a lens array. Finally, by optical experiments, we confirm that three-dimensional images can be reconstructed from holograms calculated from an integral photograph of a real object captured with an IP camera.

Integral three-dimensional television using a 2000-scanning-line video system

We have developed an integral three-dimensional (3-D) television that uses a 2000-scanning-line video system that can shoot and display 3-D color moving images in real time. We had previously developed an integral 3-D television that used a high-definition television system. The new system uses -6 times as many elemental images [160 (horizontal) x 118 (vertical) elemental images] arranged at -1.5 times the density to improve further the picture quality of the reconstructed image. Through comparison an image near the lens array can be reconstructed at -1.9 times the spatial frequency, and the viewing angle is -1.5 times as wide.

Multifacet structure of observed reconstructed integral images

Three-dimensional images generated by an integral imaging system suffer from degradations in the form of grid of multiple facets. This multifacet structure breaks the continuity of the observed image and therefore reduces its visual quality. We perform an analysis of this effect and present the guidelines in the design of lenslet imaging parameters for optimization of viewing conditions with respect to the multifacet degradation. We consider the optimization of the system in terms of field of view, observer position and pupil function, lenslet parameters, and type of reconstruction. Numerical tests are presented to verify the theoretical analysis.

Enhanced depth of field integral imaging with sensor resolution constraints

One of the main challenges in integral imaging is to overcome the limited depth of field. Although it is widely assumed that such limitation is mainly imposed by diffraction due to lenslet imaging, we show that the most restricting factor is the pixelated structure of the sensor (CCD). In this context, we demonstrate that by proper reduction of the fill factor of pickup microlenses, the depth of field can be substantially improved with no deterioration of lateral resolution.