3D touchable holographic light-field display

Distortion-Corrected Integral Imaging 3D Display System Based on Lens Array Holographic Optical Element

We propose a distortion-corrected integral imaging (II) 3D display system based on lens array holographic optical element (LAHOE). The LAHOE is used as a projection screen. The projection beam of the LAHOE is parallel light. Hence, the projection system consists of a spatial light modulator, a reverse projection lens, a relay optical element, and a telecentric lens. The acquired 3D data and the reconstructed 3D image of II are symmetrically related to each other. Therefore, there is lens distortion in the projection system. To avoid affecting the viewing experience of the viewers, the elemental image array (EIA) is projected obliquely on the LAHOE, causing the lateral distortion of the EIA. There is a position deviation in the projection system, so the projected EIA has geometric deformation. Due to the distortion of the EIA, it is difficult to precisely align the projected EIA and LAHOE, which results in serious flip of the reconstructed 3D images. The distortion of the EIA affects the asymmetry of the 3D image reconstruction. Lens distortion can be solved by the distortion compensation method. Lateral and the geometric deformation can be solved by the perspective transformations in computer graphics. After correction, the undistorted EIA is projected, and the projected EIA on the LAHOE has little distortion. In the process of 3D image reconstruction, the causes of asymmetry affecting 3D image reconstruction are analyzed, and the issues that generate these asymmetric factors are addressed. Experimental results indicate that a better 3D display effect is achieved.

Interactive optical 3D-touch user interface using a holographic light-field display and color information

We propose an interactive optical 3D-touch user interface (UI) by using a holographic light-field (LF) 3D display and a color detection system of the scattered light from the touched 3D image. In the proposed system, color information embedded in the LF is used to realize the 3D position identification and movement detection of the interaction point in 3D space only with a single RGB camera. We demonstrate the real-time interactive implementation of the interface working at 12 frames per second, which verifies the feasibility of our proposed concept.

Roadmap on 3D integral imaging: sensing, processing, and display

This Roadmap article on three-dimensional integral imaging provides an overview of some of the research activities in the field of integral imaging. The article discusses various aspects of the field including sensing of 3D scenes, processing of captured information, and 3D display and visualization of information. The paper consists of a series of 15 sections from the experts presenting various aspects of the field on sensing, processing, displays, augmented reality, microscopy, object recognition, and other applications. Each section represents the vision of its author to describe the progress, potential, vision, and challenging issues in this field.

Occlusion-robust sensing method by using the light-field of a 3D display system toward interaction with a 3D image

The recent developments in 3D display technology are remarkable. Owing to such developments, the importance of methods for interacting with 3D images has been increasing. In particular, the ability to perform input operations by directly touching displayed images is important. Therefore, a sensing method is required for recognizing the 3D positions of fingers and determining whether the fingers are touching the 3D image displayed. Conventionally, such a sensing method generally involves position sensing of fingers through image-based active sensing. However, this does not solve the following two problems: (1) the problem of positional registration, that is, the accurate matching between the displayed image and input location, and (2) the problem of occlusion robustness, that is, the achievement of successful sensing even with the presence of an occluding object between the sensor and hands to allow free movement of the hands. Our proposed method solves these problems through the following two ideas. First, we used a method called aerial imaging by retroreflection, which focuses light rays from a wider range than other 3D display systems. Second, for capturing the reflected light as well as projecting to the sensing target in active sensing, we used the light-field formed by the 3D display system. We also propose a method for obtaining rotation information, considering that the light-field formed by the reflected light changes based on the angle of the sensing target. Simulations and experiments were performed to evaluate the proposed system, and the system setup was optimized through simulations. In the first two experiments conducted on position and rotation sensing, we evaluated the sensing errors due to occlusion and found them to be less than 1.74 mm and less than 15.4 deg, respectively. In the third experiment, we constructed an interaction system with 3D images by using the proposed method and evaluated this system.

Off-axis virtual-image display and camera by holographic mirror and blur compensation

We propose off-axis virtual-image display and camera systems, which integrate a vertically-standing holographic off-axis mirror, blur-compensation optical systems, and digital imaging devices. In the system, the holographic mirror is used for an off-axis reflector, which realizes an upright and thin screen for virtual-image formation. By combining it with a display unit, an off-axis virtual-image display is realized, where the virtual image can be seen behind the upright holographic mirror. Simultaneously, by combining it with a camera unit, an off-axis camera is implemented, which realizes frontal shooting of objects by a camera placed at an off-axis position. Since both the off-axis display and the camera can be implemented by a single holographic mirror, it can be applied to a two-way visual-telecommunication system with a thin screen, which implements eye contact and the observer--image distance. A problem with the proposed system is image blur, which is caused by the chromatic dispersion of the holographic mirror. To solve this, we designed optical blur-compensation systems using a diffractive optical element and a diffuser or a lens. Experimental results verify the concept of the proposed systems with clarifying the effect of designed blur-compensation methods.

Digitally designed holographic optical element for light field displays

Concave micro-mirror arrays fabricated as holographic optical elements are used in projector-based light field displays due to their see-through characteristics. The optical axes of each micro-mirror in the array are usually made parallel to each other, which simplifies the fabrication, integral image rendering, and calibration process. However, this demands that the beam from the projector be collimated and made parallel to the optical axis of each elemental micro-mirror. This requires additional collimation optics, which puts serious limitations on the size of the display. In this Letter, we propose a solution to the above issue by introducing a new method to fabricate holographic concave micro-mirror array sheets and explain how they work in detail. 3D light field reconstructions of the size 20  cm×10  cm and 6 cm in depth are achieved using a conventional projector without any collimation optics.

Rapid calibration of a projection-type holographic light-field display using hierarchically upconverted binary sinusoidal patterns

A projection-type holographic light-field (LF) display is a full-parallax, full-color, and glass-free three-dimensional (3D) display with a holographic optical element and a projector. The display has unique characteristics, including transparency; however, a rapid calibration method has not yet been established. In this paper, we propose a rapid calibration method for a holographic LF display without sacrificing its accuracy. The proposed method performs calibration via the projection of binary sinusoidal patterns whose frequencies are iteratively and hierarchically upconverted. Compared to the conventional method, in the proposed method, the required number of projections is reduced from linear to logarithmic with the projector's resolution. We confirm the successful reconstruction of the 3D image using the proposed method.