Wavelength-multiplexed multi-focal-plane seethrough near-eye displays

High dimensional optical data - varifocal multiview imaging, compression and evaluation

Varifocal multiview (VFMV) is an emerging high-dimensional optical data in computational imaging and displays. It describes scenes in angular, spatial, and focal dimensions, whose complex imaging conditions involve dense viewpoints, high spatial resolutions, and variable focal planes, resulting in difficulties in data compression. In this paper, we propose an efficient VFMV compression scheme based on view mountain-shape rearrangement (VMSR) and all-directional prediction structure (ADPS). The VMSR rearranges the irregular VFMV to form a new regular VFMV with mountain-shape focusing distributions. This special rearrangement features prominently in enhancing inter-view correlations by smoothing focusing status changes and moderating view displacements. Then, the ADPS efficiently compresses the rearranged VFMV by exploiting the enhanced correlations. It conducts row-wise hierarchy divisions and creates prediction dependencies among views. The closest adjacent views from all directions serve as reference frames to improve the prediction efficiency. Extensive experiments demonstrate the proposed scheme outperforms comparison schemes by quantitative, qualitative, complexity, and forgery protection evaluations. As high as 3.17 dB gains of peak signal-to-noise ratio (PSNR) and 61.1% bitrate savings can be obtained, achieving the state-of-the-art compression performance. VFMV is also validated could serve as a novel secure imaging format protecting optical data against the forgery of large models.

Super multi-view near-eye virtual reality with directional backlights from wave-guides

Directional backlights have often been employed for generating multiple view-zones in three-dimensional (3D) display, with each backlight converging into a corresponding view-zone. By designing the view-zone interval for each pupil smaller than the pupil's diameter, super multi-view (SMV) can get implemented for a VAC-free 3D display. However, expanding the backlight from a light-source to cover the corresponding display panel often needs an extra thickness, which results in a thicker structure and is unwanted by a near-eye display. In this paper, two wave-guides are introduced into a near-eye virtual reality (NEVR) system, for sequentially guiding more than one directional backlight to each display panel for SMV display without bringing obvious extra thickness. A prototype SMV NEVR gets demonstrated, with two backlights from each wave-guide converging into two view-zones for a corresponding pupil. Although the additional configured light-sources are positioned far from the corresponding wave-guide in our proof-of-concept prototype, multiple light-sources can be attached to the corresponding wave-guide compactly if necessary. As proof, a 3D scene with defocus-blur effects gets displayed. The design range of the backlights' total reflection angles in the wave-guide is also discussed.

Flicker-free dual-volume augmented reality display using a pixelated interwoven integral floating technique with a geometric phase lens

A geometric phase (GP) integral floating display can provide multifocal three-dimensional (3D) augmented reality (AR) images with enhanced depth expression by switching the focal modes of the GP lens via polarization control. However, using temporal multiplexing to switch between the focal modes of GP optics causes flickering as each 3D AR image is fully presented in different frames and their temporal luminance profile becomes easily recognizable, particularly as the number of available focal modes increases. Here, we propose a novel integral floating technique to generate pixelated interwoven 3D AR images; a half of each image is spatially mixed with another and presented in both focal modes simultaneously to resolve the flickering issue. The principle was verified via experimental demonstration and optically measured data.

Dual-depth augmented reality display with reflective polarization-dependent lenses

Vergence-accommodation conflict (VAC) is a common annoying issue in near-eye displays using stereoscopy technology to provide the perception of three-dimensional (3D) depth. By generating multiple image planes, the depth cues can be corrected to accommodate a comfortable 3D viewing experience. In this study, we propose a multi-plane optical see-through augmented reality (AR) display with customized reflective polarization-dependent lenses (PDLs). Leveraging the different optical powers of two PDLs, a proof-of-concept dual-plane AR device is realized. The proposed design paves the way to a compact, lightweight, and fatigue-free AR display.

Distortion corrected tomographic near-eye displays using light field optimization

Several multifocal displays have been proposed to provide accurate accommodation cues. However, multifocal displays have an undesirable feature, which is especially emphasized in near-eye displays configuration, that the field of views (FOVs) of the virtual planes change over depth. We demonstrate that this change in FOV causes image distortions, which reduces overall image quality, and depth perception error due to the variation of image sizes according to depths. Here, we introduce a light field optimization technique to compensate for magnification variations among the focal planes. Our approach alleviates image distortions, especially noticeable in the contents with large depth discontinuity, and reconstructs the image size to precise depths, while maintaining a specific tolerance length for the target eye relief. To verify the feasibility of the algorithm, we employ this optimization method for the tomographic near-eye display system to acquire the optimal image and backlight sequences for a volumetric scene. In general, we confirm that the structural similarity index measure of reconstructed images against ground truth increases by 20% when the eye relief is 15 mm, and the accommodation cue is appropriately stimulated at the target depth with our proposed method.

Maxwellian near-eye display with an expanded eyebox

Maxwellian view systems can be employed to circumvent the vergence-accommodation conflict in near-eye displays (NEDs), which directly project images onto the retina regardless of the human eye's depth of focus. However, Maxwellian view optics typically have a limited eyebox, which prevents broader applications of this architecture in NEDs. Here, we demonstrate a thin-film two-dimensional beam deflector composed of multi-twist broad-band Pancharatnam-Berry deflectors to mitigate this limitation via eyebox replication. Based on experimental validation, our proposed design can display always-focused full-color images within a 9 mm × 9 mm eyebox and thus mitigate the limitation of conventional Maxwellian displays while adding negligible weight and volume.

1000-volume/s high-speed volumetric display for high-speed HMD

In this paper, we propose a high-speed volumetric display principle that can solve two problems faced by three-dimensional displays using the parallax stereo principle (namely, the vergence-accommodation conflict and display latency) and we report evaluation results. The proposed display method can update a set of images at different depths at 1000 Hz and is consistent with accommodation. The method selects the depth position in microseconds by combining a high-speed variable-focus lens that vibrates at about 69 kHz and sub-microsecond control of illumination light using an LED. By turning on the LED for only a few hundred nanoseconds when the refractive power of the lens is at a certain value, an image can be presented with this specific refractive power. The optical system is combined with a DMD to form an image at each depth. 3D information consisting of multiple planes in the depth direction can be presented at a high refresh rate by switching the images and changing the refractive power at high speed. A proof-of-concept system was developed to show the validity of the proposed display principle. The system successfully displayed 3D information consisting of six binary images at an update rate of 1000 volume/s.

Gigapixel and 1440-perspective extended-angle display by megapixel MEMS-SLM

Orders-of-magnitude increases are desired in the pixel count and density of spatial light modulators (SLMs) for next-gen displays. We present in-plane and simultaneous angular-spatial light modulation by a micro electro mechanical system (MEMS)-based SLM, a digital micromirror device (DMD), to generate gigapixel output by time and angular multiplexing. Pulsed illumination synchronized to the micromirror actuation achieves pixel-implemented and diffraction-based angular modulation, and source multiplexing increases angular selectivity. We demonstrate 1440-perspective image output across a 43.9^∘×1.8^∘ FOV, 8-bit multi-perspective videos at 30 FPS, and multi-focal-plane image generation. We discuss scalability to terapixels and implications for near-to-eye displays.

Augmented Reality and Virtual Reality Displays: Perspectives and Challenges

As one of the most promising candidates for next-generation mobile platform, augmented reality (AR) and virtual reality (VR) have potential to revolutionize the ways we perceive and interact with various digital information. In the meantime, recent advances in display and optical technologies, together with the rapidly developing digital processers, offer new development directions to advancing the near-eye display systems further. In this perspective paper, we start by analyzing the optical requirements in near-eye displays poised by the human visual system and then compare it against the specifications of state-of-the-art devices, which reasonably shows the main challenges in near-eye displays at the present stage. Afterward, potential solutions to address these challenges in both AR and VR displays are presented case by case, including the most recent optical research and development, which are already or have the potential to be industrialized for extended reality displays.

Polarization Volume Gratings for Near-Eye Displays and Novel Photonic Devices

Liquid crystal-based reflective polarization volume grating (PVG), also known as a linear Bragg–Berry phase optical element or a member of volume Bragg gratings (VBGs), is a functional planar structure with patterned orientation of optical axis. Due to the strong polarization selectivity, nearly 100% diffraction efficiency, large diffraction angle, and simple fabrication process, PVGs have found potential applications in novel photonic devices and emerging near-eye displays. In this review paper, we describe the operation principles, discuss the optical properties, present the fabrication methods, and provide promising applications of PVGs for near-eye displays and novel photonic devices.

Passive polymer-dispersed liquid crystal enabled multi-focal plane displays

A multi-focal plane see-through near-eye display using a transparent projection display is demonstrated. The key component of the transparent projection display is a passive polymer-dispersed liquid crystal (PDLC), which is highly transparent for a large range of incident angles in air but strongly scattering at large oblique angles in high refractive index medium (e.g. glass). The use of a passive device can avoid temporal multiplexing. Such a display is highly transparent in air and can easily deliver full-color images. The proposed method is an important step toward transparent display-enabled multi-focal plane displays.

Eccentricity effects on blur and depth perception

Foveation and (de)focus are two important visual factors in designing near eye displays. Foveation can reduce computational load by lowering display details towards the visual periphery, while focal cues can reduce vergence-accommodation conflict thereby lessening visual discomfort in using near eye displays. We performed two psychophysical experiments to investigate the relationship between foveation and focus cues. The first study measured blur discrimination sensitivity as a function of visual eccentricity, where we found discrimination thresholds significantly lower than previously reported. The second study measured depth discrimination threshold where we found a clear dependency on visual eccentricity. We discuss the study results and suggest further investigation.