Computational integral imaging reconstruction of perspective and orthographic view images by common patches analysis

Computational Integral Imaging Reconstruction via Elemental Image Blending without Normalization

This paper presents a novel computational integral imaging reconstruction (CIIR) method using elemental image blending to eliminate the normalization process in CIIR. Normalization is commonly used in CIIR to address uneven overlapping artifacts. By incorporating elemental image blending, we remove the normalization step in CIIR, leading to decreased memory consumption and computational time compared to those of existing techniques. We conducted a theoretical analysis of the impact of elemental image blending on a CIIR method using windowing techniques, and the results showed that the proposed method is superior to the standard CIIR method in terms of image quality. We also performed computer simulations and optical experiments to evaluate the proposed method. The experimental results showed that the proposed method enhances the image quality over that of the standard CIIR method, while also reducing memory usage and processing time.

Real-time optical reconstruction for a three-dimensional light-field display based on path-tracing and CNN super-resolution

Three-Dimensional (3D) light-field display plays a vital role in realizing 3D display. However, the real-time high quality 3D light-field display is difficult, because super high-resolution 3D light field images are hard to be achieved in real-time. Although extensive research has been carried out on fast 3D light-field image generation, no single study exists to satisfy real-time 3D image generation and display with super high-resolution such as 7680×4320. To fulfill real-time 3D light-field display with super high-resolution, a two-stage 3D image generation method based on path tracing and image super-resolution (SR) is proposed, which takes less time to render 3D images than previous methods. In the first stage, path tracing is used to generate low-resolution 3D images with sparse views based on Monte-Carlo integration. In the second stage, a lite SR algorithm based on a generative adversarial network (GAN) is presented to up-sample the low-resolution 3D images to high-resolution 3D images of dense views with photo-realistic image quality. To implement the second stage efficiently and effectively, the elemental images (EIs) are super-resolved individually for better image quality and geometry accuracy, and a foreground selection scheme based on ray casting is developed to improve the rendering performance. Finally, the output EIs from CNN are used to recompose the high-resolution 3D images. Experimental results demonstrate that real-time 3D light-field display over 30fps at 8K resolution can be realized, while the structural similarity (SSIM) can be over 0.90. It is hoped that the proposed method will contribute to the field of real-time 3D light-field display.

Parallel multi-view polygon rasterization for 3D light field display

Three-dimensional (3D) light field displays require samples of image data captured from a large number of regularly spaced camera images to produce a 3D image. Generally, it is inefficient to generate these images sequentially because a large number of rendering operations are repeated in different viewpoints. The current 3D image generation algorithm with traditional single viewpoint computer graphics techniques is not sufficiently well suited to the task of generating images for the light field displays. A highly parallel multi-view polygon rasterization (PMR) algorithm for 3D multi-view image generation is presented. Based on the coherence of the triangular rasterization calculation among different viewpoints, the related rasterization algorithms including primitive setup, plane function, and barycentric coordinate interpolation in the screen space are derived. To verify the proposed algorithm, a hierarchical soft rendering pipeline with GPU is designed and implemented. Several groups of images of 3D objects are used to verify the performance of the PMR method, and the correct 3D light field image can be achieved in real time.

Image Enhancement for Computational Integral Imaging Reconstruction via Four-Dimensional Image Structure

This paper describes the image enhancement of a computational integral imaging reconstruction method via reconstructing a four-dimensional (4-D) image structure. A computational reconstruction method for high-resolution three-dimensional (3-D) images is highly required in 3-D applications such as 3-D visualization and 3-D object recognition. To improve the visual quality of reconstructed images, we introduce an adjustable parameter to produce a group of 3-D images from a single elemental image array. The adjustable parameter controls overlapping in back projection with a transformation of cropping and translating elemental images. It turns out that the new parameter is an independent parameter from the reconstruction position to reconstruct a 4-D image structure with four axes of x, y, z, and k. The 4-D image structure of the proposed method provides more visual information than existing methods. Computer simulations and optical experiments are carried out to show the feasibility of the proposed method. The results indicate that our method enhances the image quality of 3-D images by providing a 4-D image structure with the adjustable parameter.

Super-multiview integral imaging scheme based on sparse camera array and CNN super-resolution

In this paper, we propose a scheme based on sparse camera array and convolution neural network super-resolution for super-multiview integral imaging. In particular, the proposed scheme is adequate to not only the virtual-world three-dimensional scene with high performance and efficiency, but also the real-world 3D scene with higher availability than the traditional methods. In the proposed scheme, we first adopt the sparse camera array strategy to capture the sparse viewpoint images and use these images to synthesize the low-resolution elemental image array, then the convolution neural network super-resolution scheme is used to restore the high-resolution elemental image array from the low-resolution elemental image array for super-multiview integral image display. Experimental results verify the feasibility of the proposed scheme.