Multi-viewer tracking integral imaging system and its viewing zone analysis

Integral imaging reconstruction system based on the human eye viewing mechanism

For integral stereo imaging systems based on lens arrays, the cross-mixing of erroneous light rays between adjacent lenses seriously affects the quality of the reconstructed light field. In this paper, we proposed a light field reconstruction method based on the human eye viewing mechanism, which incorporates simplified human eye imaging into the integral imaging system. First, the light field model for specified viewpoint is established, and the distribution of the light source for each viewpoint is accurately calculated for the EIA generation algorithm of fixed viewpoint. Second, according to the ray tracing algorithm in this paper, non-overlapping EIA based on the human eye viewing mechanism is designed to suppress the amount of crosstalk rays fundamentally. The actual viewing clarity is improved with the same reconstructed resolution. Experimental results verify the effectiveness of the proposed method. The SSIM value is higher than 0.93, which verifies that the viewing angle range is increased to 62°.

Wide-Viewing-Angle Integral Imaging System with Full-Effective-Pixels Elemental Image Array

There exists a defect of the narrow viewing angle in the conventional integral imaging system. One reason for this is that only partial pixels of each elemental image contribute to the viewing angle and the others cause image flips. In this paper, a wide-viewing-angle integral imaging system with a full-effective-pixels elemental image array (FEP-EIA) was proposed. The correspondence between viewpoints and pixel coordinates within the elemental image array was built up, and effective pixel blocks and pixels leading to flipping images were deduced. Then, a pixel replacement method was proposed to generate the FEP-EIAs, which adapt to different viewing distances. As a result, the viewing angle of the proposed integral imaging system was effectively extended through the replacement of the pixels, which caused the image flips. Experiment results demonstrated that wide viewing angles are available for the proposed integral imaging system regardless of the viewing distances.

Multi-Image Encryption Method via Computational Integral Imaging Algorithm

Under the framework of computational integral imaging, a multi-image encryption scheme based on the DNA-chaos algorithm is proposed. In this scheme, multiple images are merged to one image by a computational integral imaging algorithm, which significantly improves the efficiency of image encryption. Meanwhile, the computational integral imaging algorithm can merge images at different depth distances, thereby the different depth distances of multiple images can also be used as keys to increase the security of the encryption method. In addition, the high randomness of the chaos algorithm is combined to address the outline effect caused by the DNA encryption algorithm. We have experimentally verified the proposed multi-image encryption scheme. The entropy value of the encrypted image is 7.6227, whereas the entropy value of the merge image with two input images is 3.2886, which greatly reduces the relevance of the image. The simulation results also confirm that the proposed encryption scheme has high key security and can protect against various attacks.

Real-mode depth-fused display with viewer tracking

A real-mode depth-fused display is proposed by employing an integral imaging method in the depth-fused display system with viewer tracking. By giving depth-fusing effect between a transparent display and a floated planar two-dimensional image generated by the real-mode integral imaging method, a three-dimensional image is generated in front of the display plane unlike conventional depth-fused displays. The viewing angle of the system is expanded with a viewer tracking method. In addition, dynamic vertical and horizontal motion parallax can be given according to the tracked position of the viewer. As the depth-fusing effect is not dependent on the viewing distance, accommodation cue and motion parallax are provided for a wide range of viewing position. We demonstrate the feasibility of our proposed method by experimental system.

Large viewing angle three-dimensional display with smooth motion parallax and accurate depth cues

A three-dimensional (3D) display with smooth motion parallax and large viewing angle is demonstrated, which is based on a microlens array and a coded two-dimensional (2D) image on a 50 inch liquid crystal device (LCD) panel with the resolution of 3840 × 2160. Combining with accurate depth cues expressing, the flipping images of the traditional integral imaging (II) are eliminated, and smooth motion parallax can be achieved. The image on the LCD panel is coded as an elemental image packed repeatedly, and the depth cue is determined by the repeated period of elemental image. To construct the 3D image with complex depth structure, the varying period of elemental image is required. Here, the detailed principle and coding method are presented. The shape and the texture of a target 3D image are designed by a structure image and an elemental image, respectively. In the experiment, two groups of structure images and their corresponding elemental images are utilized to construct a 3D scene with a football in a green net. The constructed 3D image exhibits obviously enhanced 3D perception and smooth motion parallax. The viewing angle is 60°, which is much larger than that of the traditional II.

Wide-viewing integral imaging using fiber-coupled monocentric lens array

We propose a novel three dimensional integral imaging display system with improved viewing angle using a monocentric lens array (MoLA) coupled with fiber bundle. In conventional integral imaging, the off-axis aberrations of the conventional lens array limit the viewing angle in the display stage. The key to our design is a MoLA that eliminates most of the off-axis aberrations and generates a wide-angle image on a spherical surface. The fiber bundle acts as relay optics from the flat-panel display to spherical focal plane of the MoLA. The viewing angle enhancement of the proposed method is analyzed, and the achromatic condition is deduced for the MoLA to correct the chromatic aberration. The experimental result illustrates the capabilities of the proposed method.

Instability of the perceived world while watching 3D stereoscopic imagery: A likely source of motion sickness symptoms

Watching 3D content using a stereoscopic display may cause various discomforting symptoms, including eye strain, blurred vision, double vision, and motion sickness. Numerous studies have reported motion-sickness-like symptoms during stereoscopic viewing, but no causal linkage between specific aspects of the presentation and the induced discomfort has been explicitly proposed. Here, we describe several causes, in which stereoscopic capture, display, and viewing differ from natural viewing resulting in static and, importantly, dynamic distortions that conflict with the expected stability and rigidity of the real world. This analysis provides a basis for suggested changes to display systems that may alleviate the symptoms, and suggestions for future studies to determine the relative contribution of the various effects to the unpleasant symptoms.

Partially-overlapped viewing zone based integral imaging system with super wide viewing angle

In this paper, we analyze the relationship between viewer and viewing zones of integral imaging (II) system and present a partially-overlapped viewing zone (POVZ) based integral imaging system with a super wide viewing angle. In the proposed system, the viewing angle can be wider than the viewing angle of the conventional tracking based II system. In addition, the POVZ can eliminate the flipping and time delay of the 3D scene as well. The proposed II system has a super wide viewing angle of 120° without flipping effect about twice as wide as the conventional one.

Improvement method of integral imaging quality based on an aperture-tunable lens array

A method for improving the viewing quality of integral imaging (II) is proposed, based on using an aperture-tunable lens array (LA). The proposed method uses a liquid crystal (LC) panel without a backlighting unit to tune the aperture of an LA dynamically. The shape and transmittance of the aperture can be controlled arbitrarily by programming the state of the pixels on the LC panel. Adding the temporal multiplexing technique, the viewing quality can be improved by the after-image effect of the human eye. Moreover, the relationships between the lateral resolution and the aperture tuning pattern and the depth of field and the aperture tuning pattern are derived, respectively. The product of the depth of field, the lateral resolution squared, and the lateral viewing range is proposed as a new figure of merit for an II system. Experimental results show the validity of the proposed method.

Compensation of elemental image using multiple view vectors for off-axis integral floating system

A novel method is proposed to compensate the distortion caused by the off-axis structure in integral floating with a concave mirror. The proposed method of this paper can generate the elemental image that is integrated into the multiple predistortion images for the corresponding multiview directions, which cannot be performed by the predistortion method using a single-view vector. The fundamental concept of the proposed method is that the predistortion images can be generated for more than two view vectors, which are defined by each set of corresponding elemental lens and subimage, and captured onto the subimage. In addition, the subimage can be divided into the several segments to increase the number of view vectors, so that the predistortion images generated by more view vectors can enhance the degree of compensation. The proper number of the view vectors per subimage is discussed and analyzed based on the small-angle approximation. Experiments to verify the feasibility of the proposed method are performed.

Analysis of the depth of field of integral imaging displays based on wave optics

In this paper, we analyze the depth of field (DOF) of integral imaging displays based on wave optics. With considering the diffraction effect, we analyze the intensity distribution of light with multiple micro-lenses and derive a DOF calculation formula for integral imaging display system. We study the variations of DOF values with different system parameters. Experimental results are provided to verify the accuracy of the theoretical analysis. The analyses and experimental results presented in this paper could be beneficial for better understanding and designing of integral imaging displays.

Viewing-zone control of integral imaging display using a directional projection and elemental image resizing method

Viewing-zone control of integral imaging (II) displays using a directional projection and elemental image (EI) resizing method is proposed. Directional projection of EIs with the same size of microlens pitch causes an EI mismatch at the EI plane. In this method, EIs are generated computationally using a newly introduced algorithm: the directional elemental image generation and resizing algorithm considering the directional projection geometry of each pixel as well as an EI resizing method to prevent the EI mismatch. Generated EIs are projected as a collimated projection beam with a predefined directional angle, either horizontally or vertically. The proposed II display system allows reconstruction of a 3D image within a predefined viewing zone that is determined by the directional projection angle.

Online tracking of deformable objects under occlusion using dominant points

This paper deals with tracking of deformable objects in the presence of occlusion using dominant point representation of the boundary contour. A novel nonintegral time propagation model for propagating the dominant points is proposed. It uses an initial guess generated from a linear operation and an analytical conjugate gradient approach for online robust learning of the shape deformation and motion model. A scheme is presented to automatically detect and correct the region of large local deformation. In order to deal with occlusion, admissible restrictions on deformation and motion of the object are automatically determined. The proposed method overcomes the need of offline learning and learns the deformation and motion model of the object using very few initial frames of the input video. The performance of the method is demonstrated using varieties of videos of different objects.

Multiple ray cluster rendering for interactive integral imaging system

In this paper, we present an efficient Computer Generated Integral Imaging (CGII) method, called multiple ray cluster rendering (MRCR). Based on the MRCR, an interactive integral imaging system is realized, which provides accurate 3D image satisfying the changeable observers' positions in real time. The MRCR method can generate all the elemental image pixels within only one rendering pass by ray reorganization of multiple ray clusters and 3D content duplication. It is compatible with various graphic contents including mesh, point cloud, and medical data. Moreover, multi-sampling method is embedded in MRCR method for acquiring anti-aliased 3D image result. To our best knowledge, the MRCR method outperforms the existing CGII methods in both the speed performance and the display quality. Experimental results show that the proposed CGII method can achieve real-time computational speed for large-scale 3D data with about 50,000 points.

Integral floating display systems for augmented reality

Novel integral floating three-dimensional (3D) display methods are proposed for implementing an augmented reality (AR) system. The 3D display for AR requires a long-range focus depth and a see-though property. A system that adopts a concave lens instead of a convex lens is proposed for realizing the integral floating system with a long working distance using a reduced pixel pitch of the elemental image. An investigation that reveals that the location of the central depth plane is restricted by the pixel pitch of the display device is presented. An optical see-through system using a convex half mirror is also proposed for providing 3D images with a proper accommodation response. The concepts of the proposed methods are explained and the validity of system is proved by the experimental results.

Three-dimensional display technologies of recent interest: principles, status, and issues [Invited]

Recent trends in three-dimensional (3D) display technologies are very interesting in that both old-fashioned and up-to-date technologies are being actively investigated together. The release of the first commercially successful 3D display product raised new research topics in stereoscopic display. Autostereoscopic display renders a ray field of a 3D image, whereas holography replicates a wave field of it. Many investigations have been conducted on the next candidates for commercial products to resolve existing limitations. Up-to-date see-through 3D display is a concept close to the ultimate goal of presenting seamless virtual images. Although it is still far from practical use, many efforts have been made to resolve issues such as occlusion problems.

Viewing angle enhanced integral imaging display by using a high refractive index medium

In the integral imaging system, the viewing angle is limited by the size and focal length of the elemental lens. In this regard, we propose a new method for the viewing angle enhancement in the InIm. The proposed method employs a refractive index medium between the elemental image plane and the lens array. The viewing angle enhanced InIm display is analyzed based on the imaging terms. The experimental result shows that the viewing angle is doubled.

Super multi-view display with a lower resolution flat-panel display

A lenticular-type super multi-view (SMV) display normally requires an ultra high-resolution flat-panel display. To reduce this resolution requirement, two or more views are generated around each eye with an interval smaller than the pupil diameter. Cylindrical lenses constituting a lenticular lens project a group of pixels of the flat-panel display to generate a group of viewing zones. Pixel groups generating left and right viewing zones through the same cylindrical lens are partitioned to separate the two zones. The left and right pixel groups for different cylindrical lenses are interlaced horizontally. A prototype SMV display is demonstrated.

Elimination of image discontinuity in integral floating display by using adaptive image mapping

We propose an adaptive image mapping method that removes image discontinuity, which is the main degradation factor of image quality in an integral floating display. The proposed method is based on a newly modified image mapping algorithm that removes image discontinuity for an observer at a specific location. By taking advantage of an eye tracking scheme, the modified algorithm can produce a floating 3D image in real time for moving observers as well. The observable area of the proposed system is analyzed, and we show that multiple observers in the area can view floating 3D images without interfering with one another's views. The effect of tracking error on the floating 3D image is analyzed, and a strategy to deal with it is also given.