Real-time depth controllable integral imaging pickup and reconstruction method with a light field camera

Comprehensive High-Quality Three-Dimensional Display System Based on a Simplified Light-Field Image Acquisition Method and a Full-Connected Deep Neural Network

We propose a high-quality, three-dimensional display system based on a simplified light field image acquisition method, and a custom-trained full-connected deep neural network is proposed. The ultimate goal of the proposed system is to acquire and reconstruct the light field images with possibly the most elevated quality from the real-world objects in a general environment. A simplified light field image acquisition method acquires the three-dimensional information of natural objects in a simple way, with high-resolution/high-quality like multicamera-based methods. We trained a full-connected deep neural network model to output desired viewpoints of the object with the same quality. The custom-trained instant neural graphics primitives model with hash encoding output the overall desired viewpoints of the object within the acquired viewing angle in the same quality, based on the input perspectives, according to the pixel density of a display device and lens array specifications within the significantly short processing time. Finally, the elemental image array was rendered through the pixel re-arrangement from the entire viewpoints to visualize the entire field-of-view and re-constructed as a high-quality three-dimensional visualization on the integral imaging display. The system was implemented successfully, and the displayed visualizations and corresponding evaluated results confirmed that the proposed system offers a simple and effective way to acquire light field images from real objects with high-resolution and present high-quality three-dimensional visualization on the integral imaging display system.

Efficient evaluation of a three-dimensional eye-box in a near-eye display using light-field acquisition of luminance distribution

We propose, what we believe to be, a novel assessment methodology for evaluating three-dimensional (3D) characteristics of an eye-box volume in a near-eye display (NED) using a light-field (LF) data acquired at a single measuring distance. In contrast to conventional evaluation methods for the eye-box, where a light measuring device (LMD) changes its position in lateral and longitudinal directions, the proposed method requires an LF of the luminance distribution (LFLD) for the NEDs captured only at the single observation distance, and the 3D eye-box volume is evaluated via a simple post-analysis. We explore an LFLD-based representation for the efficient evaluation of the 3D eye-box, and the theoretical analysis is validated by simulation results using Zemax OpticStudio. As experimental verifications, we acquired an LFLD for an augmented reality NED at a single observation distance. The assessed LFLD constructed a 3D eye-box successfully over the distance range of 20 mm, which included assessment conditions where it was hard to measure the light rays' distributions directly in the conventional methodologies. The proposed method is further verified by comparing with actual observed images of the NED both inside and outside of the evaluated 3D eye-box.

Motion parallax enhanced 3-D integral imaging display from the commercial plenoptic camera

The direct pickup of integral imaging typically needs to overcome limitations especially the restricted depth of field (DoF) under a lenslet array. In order to solve the problem, we design a motion parallax enhancing approach for three-dimensional (3-D) integral optical display only relying on a commercial Lytro camera. First, the non-uniform axial compression from the zoom lens of the Lytro camera is analyzed and experimentally investigated. Next, using depth slicing, locating and retargeting, the parallax of the integral optical display is significantly enhanced. Additionally, the displayed depth information can be presented in a uniform compression with the same proportion as the real scene even without the prior knowledge of the actual object distance. The experimental results prove the feasibility of the proposed method, which provides an efficient way for the acquisition of the elemental image array. Additionally, it is also a new attempt to expand the application scope of the Lytro camera from 2-D refocusing to the content acquisition for the integral display.

Reciprocal 360-deg 3D light-field image acquisition and display system [Invited]

A reciprocal 360-deg three-dimensional light-field image acquisition and display system was designed using a common catadioptric optical configuration and a lens array. Proof-of-concept experimental setups were constructed with a full capturing part and a truncated display section to demonstrate that the proposed design works without loss of generality. Unlike conventional setups, which record and display rectangular volumes, the proposed configuration records 3D images from its surrounding spherical volume in the capture mode and projects 3D images to the same spherical volume in the display mode. This is particularly advantageous in comparison to other 360-deg multi-camera and multiple projector display systems that require extensive image and physical calibration. We analyzed the system and showed quality measures such as angular resolution and space bandwidth product based on design parameters. The issue due to pixel size difference between the available imaging sensor and the display was also addressed. A diffractive microlens array matching the sensor size was used in the acquisition part, whereas a vacuum cast lens array matching the display size was used in the display part with scaled optics. The experimental results demonstrate the proposed system design works well and is in good agreement with the simulation results.

Dual-dimensional microscopy: real-time in vivo three-dimensional observation method using high-resolution light-field microscopy and light-field display

Here, we present dual-dimensional microscopy that captures both two-dimensional (2-D) and light-field images of an in-vivo sample simultaneously, synthesizes an upsampled light-field image in real time, and visualizes it with a computational light-field display system in real time. Compared with conventional light-field microscopy, the additional 2-D image greatly enhances the lateral resolution at the native object plane up to the diffraction limit and compensates for the image degradation at the native object plane. The whole process from capturing to displaying is done in real time with the parallel computation algorithm, which enables the observation of the sample's three-dimensional (3-D) movement and direct interaction with the in-vivo sample. We demonstrate a real-time 3-D interactive experiment with Caenorhabditis elegans.

Focal stack camera in all-in-focus imaging via an electrically tunable liquid crystal lens doped with multi-walled carbon nanotubes

A focal stack camera, based on an electrically tunable-focusing liquid crystal (LC) lens doped with multi-walled carbon nanotubes, is proposed to generate a single all-in-focus image of a 3D scene without depth map in a relatively short time. Focal sweep strategy of the camera is devised. Both its depth of field (DOF) and focal sweep speed are analyzed and deduced. Nano doping method is adopted to improve electro-optical features of the LC lens. To efficiently produce all-in-focus image, a weighted average algorithm for all images in the focal stack is utilized. The experiments show that the result is a high contrast at sensor resolution. It is greatly potential in optical compact 3D imaging system.

F-number matching method in light field microscopy using an elastic micro lens array

In light field microscopy (LFM), the F-number of the micro lens array (MLA) should be matched with the image-side F-number of the objective lens to utilize full resolution of an image sensor. We propose a new F-number matching method that can be applied to multiple objective lenses by using an elastic MLA. We fabricate an elastic MLA with polydimethylsiloxane (PDMS) using a micro contact printing method and address the strain for the F-number variation. The strain response is analyzed, and the LFM system with the elastic MLA is demonstrated. Our proposed system can increase the F-number up to 27.3% and can be applied to multiple objective lenses.