Recent researches based on integral imaging display method

A Light Field Display Realization with a Nematic Liquid Crystal Microlens Array and a Polymer Dispersed Liquid Crystal Film

This study demonstrates a light field display system using a nematic liquid crystal (LC) microlens array (MLA) and a polymer dispersed liquid crystal (PDLC) film. LC-MLA without polarization effects presented high-resolution intermediate 3D images by adopting a depolarization algorithm. The adopted PDLC film modulated the reconstructed 3D images to deliver full-parallax images efficiently with a wide FOV. The experimental result shows that the peak signal to noise ratio (PSNR) value of photograph accurate display results improves compared to the pure LC-MLA method. The proposed method is an essential step toward high-quality light field display.

Viewing-angle and viewing-resolution enhanced integral imaging based on time-multiplexed lens stitching

A method for the viewing angle and viewing resolution enhancement of integral imaging (InIm) based on time-multiplexed lens stitching is demonstrated using the directional time-sequential backlight (DTS-BL) and the compound lens-array. In order to increase the lens-pitch of the compound lens-array for enlarging the viewing angle of InIm, DTS-BL is used to continuously stitch the adjacent elemental lenses in the time-multiplexed way. Through the compound lens-array with two pieces of lens in each lens unit, the parallel light beams from the DTS-BL converge and form a uniformly distributed dense point light source array (PLSA). Light rays emitting from the PLSA are modulated by the liquid crystal display (LCD) panel and then integrated as volumetric pixels of the reconstructed three-dimensional (3D) image. Meanwhile, time-multiplexed generation of the point light sources (PLSs) in the array is realized by time-multiplexed lens stitching implemented with the DTS-BL. As a result, the number of the PLSs, as the pixels of the perceived 3D image, is increased and then the viewing resolution of the 3D image is obviously enhanced. Additionally, joint optical optimization for the DTS-BL and the compound lens-array is used for suppressing the aberrations, and the imaging distortion can be decreased to 0.23% from 5.80%. In the experiment, a floating full-parallax 3D light-field image can be perceived with 4 times the viewing resolution enhancement in the viewing angle of 50°, where 7056 viewpoints are presented.

Super-multiview integral imaging scheme based on sparse camera array and CNN super-resolution

In this paper, we propose a scheme based on sparse camera array and convolution neural network super-resolution for super-multiview integral imaging. In particular, the proposed scheme is adequate to not only the virtual-world three-dimensional scene with high performance and efficiency, but also the real-world 3D scene with higher availability than the traditional methods. In the proposed scheme, we first adopt the sparse camera array strategy to capture the sparse viewpoint images and use these images to synthesize the low-resolution elemental image array, then the convolution neural network super-resolution scheme is used to restore the high-resolution elemental image array from the low-resolution elemental image array for super-multiview integral image display. Experimental results verify the feasibility of the proposed scheme.

Resolution priority holographic stereogram based on integral imaging with enhanced depth range

Conventional holographic stereogram (HS) can be generated through fast Fourier transforming parallax images into hogels. Conventional HS uses multiple plane waves to reconstruct 3D images with low resolution and is similar to the principle of depth priority integral imaging (II). We proposed the concept of resolution priority HS for the first time, which is based on the principle of resolution priority II, by adding a quadratic phase term on the conventional Fourier transform. In the proposed resolution priority HS, the resolution of reconstructed 3D images is much better than conventional HS, but the depth range is limited. To enhance the depth range, a multi-plane technique was used to present multiple central depth planes simultaneously. The proposed resolution priority HS with high resolution and enhanced depth range was verified by both simulation and optical experiment.

Computational integral imaging reconstruction of perspective and orthographic view images by common patches analysis

A novel method to computationally reconstruct perspective and orthographic view images with full resolution of a recording device from a single integral photograph is proposed. Firstly, a group of image slices that contain full yet redundant information to reconstruct the view image are generated, and the object surface is divided into pieces by the points that correspond to the centers of image slices. Secondly, the image slices that contribute to the pieces are extracted and redundant information embedded in them are figured out by common patches analysis. Finally, the view image is reconstructed by excluding the redundant information and resampling with maximum sampling rate. Each piece of the object surface is represented with 9 patches at most from 4 adjacent elemental images, and view images with high quality are reconstructed. Both simulations and experiments verify the validity of the method.

Optical full-depth refocusing of 3-D objects based on subdivided-elemental images and local periodic δ-functions in integral imaging

We propose a new approach for optical refocusing of three-dimensional (3-D) objects on their real depth without a pickup-range limitation based on subdivided-elemental image arrays (sub-EIAs) and local periodic δ-function arrays (L-PDFAs). The captured EIA from the 3-D objects locating out of the pickup-range, is divided into a number of sub-EIAs depending on the object distance from the lens array. Then, by convolving these sub-EIAs with each L-PDFA whose spatial period corresponds to the specific object's depth, as well as whose size is matched to that of the sub-EIA, arrays of spatially-filtered sub-EIAs (SF-sub-EIAs) for each object depth can be uniquely extracted. From these arrays of SF-sub-EIAs, 3-D objects can be optically reconstructed to be refocused on their real depth. Operational principle of the proposed method is analyzed based on ray-optics. In addition, to confirm the feasibility of the proposed method in the practical application, experiments with test objects are carried out and the results are comparatively discussed with those of the conventional method.

Resolution-enhanced integral imaging using two micro-lens arrays with different focal lengths for capturing and display

We proposed a resolution enhanced integral imaging display method using two micro-lens arrays (MLA) with different focal lengths for capturing and display respectively. An elemental image array (EIA) is captured with MLA of focal length of f(1) and a processed EIA is displayed with MLA of focal length of f(2) which is larger than f(1). We enlarge the "effective area" in processed EIA to increase the information obtained by viewer, in other words, enhance the viewing resolution. The two micro-lens arrays for capturing and display are g and mg distant from display device respectively, and we can get m(2) times resolution enhancement.

Active confocal imaging for visual prostheses

There are encouraging advances in prosthetic vision for the blind, including retinal and cortical implants, and other "sensory substitution devices" that use tactile or electrical stimulation. However, they all have low resolution, limited visual field, and can display only few gray levels (limited dynamic range), severely restricting their utility. To overcome these limitations, image processing or the imaging system could emphasize objects of interest and suppress the background clutter. We propose an active confocal imaging system based on light-field technology that will enable a blind user of any visual prosthesis to efficiently scan, focus on, and "see" only an object of interest while suppressing interference from background clutter. The system captures three-dimensional scene information using a light-field sensor and displays only an in-focused plane with objects in it. After capturing a confocal image, a de-cluttering process removes the clutter based on blur difference. In preliminary experiments we verified the positive impact of confocal-based background clutter removal on recognition of objects in low resolution and limited dynamic range simulated phosphene images. Using a custom-made multiple-camera system based on light-field imaging, we confirmed that the concept of a confocal de-cluttered image can be realized effectively.

Real-time 3D display system based on computer-generated integral imaging technique using enhanced ISPP for hexagonal lens array

This paper proposes an open computer language (OpenCL) parallel processing method to generate the elemental image arrays (EIAs) for hexagonal lens array from a three-dimensional (3D) object such as a volume data. Hexagonal lens array has a higher fill factor compared to the rectangular lens array case; however, each pixel of an elemental image should be determined to belong to the single hexagonal lens. Therefore, generation for the entire EIA requires very large computations. The proposed method reduces processing time for the EIAs for a given hexagonal lens array. By using the proposed image space parallel processing (ISPP) method, it can enhance the processing speed that generates the 3D display of real-time interactive integral imaging for hexagonal lens array. In our experiment, we implemented the EIAs for hexagonal lens array in real-time and obtained a good processing time for a large of volume data for multiple cases of lens arrays.

Three-dimensional imaging and visualization using off-axially distributed image sensing

This Letter presents an off-axially distributed image sensing (ODIS) system for three-dimensional (3D) imaging and visualization. The off-axially distributed sensing method provides both lateral and longitudinal perspectives for 3D scenes even though the sensor moves along a slanted, one-dimensional path. A 3D volume is generated from a set of recorded images by use of a computational algorithm based on ray backprojection. Preliminary experimental results are presented to illustrate the feasibility of the proposed system. To the best of our knowledge, this is the first report on 3D imaging and visualization using ODIS.

Effect of fundamental depth resolution and cardboard effect to perceived depth resolution on multi-view display

In three-dimensional television (3D TV) broadcasting, we find the effect of fundamental depth resolution and the cardboard effect to the perceived depth resolution on multi-view display is important. The observer distance and the specification of multi-view display quantize the expressible depth range, which affect the perception of depth resolution of the observer. In addition, the multi-view 3D TV needs the view synthesis process using depth image-based rendering which induces the cardboard effect from the relation among the stereo pickup, the multi-view synthesis and the multi-view display. In this paper, we analyze the fundamental depth resolution and the cardboard effect from the synthesis process in the multi-view 3D TV broadcasting. After the analysis, the numerical comparison and subjective tests with 20 participants are performed to find the effect of fundamental depth resolution and the cardboard effect to the perceived depth resolution.

Depth extraction of three-dimensional objects using block matching for slice images in synthetic aperture integral imaging

We describe a computational method for depth extraction of three-dimensional (3D) objects using block matching for slice images in synthetic aperture integral imaging (SAII). SAII is capable of providing high-resolution 3D slice images for 3D objects because the picked-up elemental images are high-resolution ones. In the proposed method, the high-resolution elemental images are recorded by moving a camera; a computational reconstruction algorithm based on ray backprojection generates a set of 3D slice images from the recorded elemental images. To extract depth information of the 3D objects, we propose a new block-matching algorithm between a reference elemental image and a set of 3D slice images. The property of the slices images is that the focused areas are the right location for an object, whereas the blurred areas are considered to be empty space; thus, this can extract robust and accurate depth information of the 3D objects. To demonstrate our method, we carry out the preliminary experiments of 3D objects; the results indicate that our method is superior to a conventional method in terms of depth-map quality.

Visibility-enhanced reconstruction of three-dimensional objects under a heavily scattering medium through combined use of intermediate view reconstruction, multipixel extraction, and histogram equalization methods in the conventional integral imaging system

In this paper, we propose an effective approach for reconstructing visibility-enhanced three-dimensional (3D) objects under the heavily scattering medium of dense fog in the conventional integral imaging system through the combined use of the intermediate view reconstruction (IVR), multipixel extraction (MPE), and histogram equalization (HE) methods. In the proposed system, the limited number of elemental images (EIs) picked up from the 3D objects under the dense fog is increased by as many as required by using the IVR technique. The increased number of EIs is transformed into the subimages (SIs) in which the resolution of the transformed SIs has been also improved as much as possible with the MPE method. Subsequently, by using the HE algorithm, the histogram of the resolution-enhanced SIs is uniformly redistributed over the entire range of discrete pixel levels of the image in a way that the subimage contrast can be much enhanced. Then, these equalized SIs are converted back into the newly modified EIs, and consequently a visibility-enhanced 3D object image can be reconstructed. Successful experimental results with the test object confirmed the feasibility of the proposed method.

Effective reduction of the novel look-up table memory size based on a relationship between the pixel pitch and reconstruction distance of a computer-generated hologram

In this paper, we propose an approach, new to our knowledge, to effectively generate and reconstruct the resolution-enhanced computer-generated hologram (CGH) of three-dimensional (3-D) objects with a significantly reduced in memory size novel look-up table (N-LUT) by taking into account a relationship between the pixel pitch and reconstruction distance of the hologram pattern. In the proposed method, a CGH pattern composed of shifted versions of the principal fringe patterns (PFPs) with a short pixel pitch can be reconstructed just by using the CGH generated with a much longer pixel pitch by controlling the hologram reconstruction distance. Accordingly, the corresponding N-LUT memory size required for resolution-enhanced hologram patterns can be significantly reduced in the proposed method. To confirm the feasibility of the proposed method, experiments are carried out and the results are discussed.