LensIet VR: Thin, Flat and Wide-FOV Virtual Reality Display Using Fresnel Lens and LensIet Array

Wearable Augmented Reality: Research Trends and Future Directions from Three Major Venues

Wearable Augmented Reality (AR) has attracted considerable attention in recent years, as evidenced by the growing number of research publications and industry investments. With swift advancements and a multitude of interdisciplinary research areas within wearable AR, a comprehensive review is crucial for integrating the current state of the field. In this paper, we present a review of 389 research papers on wearable AR, published between 2018 and 2022 in three major venues: ISMAR, TVCG, and CHI. Drawing inspiration from previous works by Zhou et al. and Kim et al., which summarized AR research at ISMAR over the past two decades (1998-2017), we categorize the papers into different topics and identify prevailing trends. One notable finding is that wearable AR research is increasingly geared towards enabling broader consumer adoption. From our analysis, we highlight key observations related to potential future research areas essential for capitalizing on this trend and achieving widespread adoption. These include addressing challenges in Display, Tracking, Interaction, and Applications, and exploring emerging frontiers in Ethics, Accessibility, Avatar and Embodiment, and Intelligent Virtual Agents.

Achromatic diffractive liquid-crystal optics for virtual reality displays

Diffractive liquid-crystal optics is a promising optical element for virtual reality (VR) and mixed reality as it provides an ultrathin formfactor and lightweight for human factors and ergonomics. However, its severe chromatic aberrations impose a big challenge for full-color display applications. In this study, we demonstrate an achromatic diffractive liquid-crystal device to overcome this longstanding chromatic aberration issue. The proposed device consists of three stacked diffractive liquid crystal optical elements with specifically designed spectral response and polarization selectivity. The concept is validated by both simulations and experiments. Our experimental results show a significant improvement in imaging performance with two types of light engines: a laser projector and an organic light-emitting diode display panel. In addition, our simulation results indicate that the lateral color shift is reduced by ~100 times in comparison with conventional broadband diffractive liquid-crystal lens. Potential applications for VR-enabled metaverse, spatial computing, and digital twins that have found widespread applications in smart tourism, smart education, smart healthcare, smart manufacturing, and smart construction are foreseeable.

Integrated metasurfaces for re-envisioning a near-future disruptive optical platform

Metasurfaces have been continuously garnering attention in both scientific and industrial fields, owing to their unprecedented wavefront manipulation capabilities using arranged subwavelength artificial structures. To date, research has mainly focused on the full control of electromagnetic characteristics, including polarization, phase, amplitude, and even frequencies. Consequently, versatile possibilities of electromagnetic wave control have been achieved, yielding practical optical components such as metalenses, beam-steerers, metaholograms, and sensors. Current research is now focused on integrating the aforementioned metasurfaces with other standard optical components (e.g., light-emitting diodes, charged-coupled devices, micro-electro-mechanical systems, liquid crystals, heaters, refractive optical elements, planar waveguides, optical fibers, etc.) for commercialization with miniaturization trends of optical devices. Herein, this review describes and classifies metasurface-integrated optical components, and subsequently discusses their promising applications with metasurface-integrated optical platforms including those of augmented/virtual reality, light detection and ranging, and sensors. In conclusion, this review presents several challenges and prospects that are prevalent in the field in order to accelerate the commercialization of metasurfaces-integrated optical platforms.

Machine learning enables the design of a bidirectional focusing diffractive lens

Machine learning can efficiently empower the inverse design of cascaded diffractive optical elements. In this work, we explore the inverse design of a bidirectional focusing diffractive lens in a cascaded configuration through the diffractive optical neural network (DONN) machine learning method. The bidirectional focusing diffractive lens consists of two on-axially cascaded multi-level diffractive lenses. Each lens consists of concentric rings with equal widths and varying heights. The height of each concentric ring is optimized as part of the design algorithm. The diffractive lens has a focal length f₊ as light propagates in the forward (Z+) direction. As light propagates in the backward (Z-) direction, the focal length changes to f_-. The designed lens is fabricated through a two-photon polymerization 3D printing technique. The proposed design is polarization insensitive and miniature and can be readily applied in future functional optical imaging systems.

Effect of spatial distortions in head-mounted displays on visually induced motion sickness

Incomplete optical distortion correction in VR HMDs leads to spatial dynamic distortion, which is a potential cause of VIMS. A perception experiment is designed for the investigation with three spatial distortion levels, with the subjective SSQ, five-scale VIMS level rating, and objective postural instability adopted as the evaluation metrics. The results show that the factor of spatial distortion level has a significant effect on all metrics increments (p<0.05). As the spatial distortion level drops off, the increments of VIMS symptoms decrease. The study highlights the importance of perfect spatial distortion correction in VR HMDs for eliminating the potential VIMS aggravation effect.

Stray light analysis and suppression method of a pancake virtual reality head-mounted display

Pancake virtual reality head-mounted displays (VR-HMDs) have attracted the attention of researchers in both academia and industry because of the compact size and light weight. However, owing to the use of optical path folding, there exist various stray lights in the optical system, which seriously degrades user experience. In this study, we analyze the causes and effects of multiple types of stray light systematically and design a VR-HMD with low stray light, large exit pupil diameter (EPD), compact form and light weight. Subsequently, several effective stray light suppression solutions are proposed and implemented. Finally, a prototype of a compact pancake VR-HMD system is successfully demonstrated. The prototype has stray light of less than 2.3%, a diagonal field of view (FOV) of 96° and an EPD of 10 mm at an 11 mm eye relief (ERF).

Ultracompact virtual reality system with a Pancharatnam-Berry phase deflector

We propose an ultracompact virtual reality (VR) system with three optical components: a lenslet array, a Pancharatnam-Berry phase deflector (PBD), and a deflector array. The lenslet array aims to collect and collimate the input light from the display panel. The PBD steers the deviated beams after the lenslet array toward the optical axis so that the image uniformity and angular resolution can be enhanced, which plays a key role to enable this ultracompact design. Finally, the deflector array deflects the collimated beam from each lenslet to the exit pupil to widen the field of view. Such an ultracompact design is particularly attractive for next-generation glasses-like, lightweight VR headsets.

Accelerating a spatially varying aberration correction of holographic displays with low-rank approximation

Correction of spatially varying aberrations in holographic displays often requires intractable computational loads. In this Letter, we introduce a low-rank approximation method that decomposes sub-holograms into a small number of modes, thereby reformulating the computer-generated hologram calculation into a summation of a few convolutions. The low-rank approximation is carried out with two different algorithms: the Karhunen-Loeve transform as the optimum solution with respect to the mean-squared error criterion and a novel, to the best of our knowledge, optimization method to provide uniform image quality over the entire field of view. The proposed method is two orders of magnitude faster than the conventional point-wise integration method in our experimental setup, with comparable image quality.

Cross-talk elimination for lenslet array near eye display based on eye-gaze tracking

Lenslet array (LA) near-eye displays (NEDs) are a recent technical development that creates a virtual image in the field of view of one or both eyes. A problem occurs when the user's pupil moves out of the LA-NED eye box (i.e., cross-talk) making the image look doubled or ghosted. It negatively impacts the user experience. Although eye-gaze tracking can mitigate this problem, the effect of the solution has not been studied to understand the impact of pupil size and human perception. In this paper, we redefine the cross-talk region as the practical pupil movable region (PPMR₅₀), which differs from eye box size because it considers pupil size and human visual perception. To evaluate the effect of eye-gaze tracking on subjective image quality, three user studies were conducted. From the results, PPMR₅₀ was found to be consistent with human perception, and cross-talk elimination via eye-gaze tracking was better understood in a static gaze scenario. Although the system latency prevented the complete elimination of cross-talk for fast movements or large pupil changes, the problem was greatly alleviated. We also analyzed system delays based on PPMR₅₀, which we newly defined in this paper and provided an optimization scheme to meet the maximum eyeball rotation speed.

Optical design and pupil swim analysis of a compact, large EPD and immersive VR head mounted display

Virtual reality head-mounted displays (VR-HMDs) are crucial to Metaverse which appears to be one of the most popular terms to have been adopted over the internet recently. It provides basic infrastructure and entrance to cater for the next evolution of social interaction, and it has already been widely used in many fields. The VR-HMDs with traditional aspherical or Fresnel optics are not suitable for long-term usage because of the image quality, system size, and weight. In this study, we designed and developed a large exit pupil diameter (EPD), compact, and lightweight VR-HMD with catadioptric optics. The mathematical formula for designing the catadioptric VR optics is derived. The reason why this kind of immersive VR optics could achieve a compact size and large EPD simultaneously is answered. Various catadioptric forms are systematically proposed and compared. The design can achieve a diagonal field of view (FOV) of 96° at -1 diopter, with an EPD of 10 mm at 11 mm eye relief (ERF). The overall length (OAL) of the system was less than 20 mm. A prototype of a compact catadioptric VR-HMD system was successfully developed.

Unfiltered holography: optimizing high diffraction orders without optical filtering for compact holographic displays

Computer-generated holography suffers from high diffraction orders (HDOs) created from pixelated spatial light modulators, which must be optically filtered using bulky optics. Here, we develop an algorithmic framework for optimizing HDOs without optical filtering to enable compact holographic displays. We devise a wave propagation model of HDOs and use it to optimize phase patterns, which allows HDOs to contribute to forming the image instead of creating artifacts. The proposed method significantly outperforms previous algorithms in an unfiltered holographic display prototype.

Augmented reality and virtual reality displays: emerging technologies and future perspectives

With rapid advances in high-speed communication and computation, augmented reality (AR) and virtual reality (VR) are emerging as next-generation display platforms for deeper human-digital interactions. Nonetheless, to simultaneously match the exceptional performance of human vision and keep the near-eye display module compact and lightweight imposes unprecedented challenges on optical engineering. Fortunately, recent progress in holographic optical elements (HOEs) and lithography-enabled devices provide innovative ways to tackle these obstacles in AR and VR that are otherwise difficult with traditional optics. In this review, we begin with introducing the basic structures of AR and VR headsets, and then describing the operation principles of various HOEs and lithography-enabled devices. Their properties are analyzed in detail, including strong selectivity on wavelength and incident angle, and multiplexing ability of volume HOEs, polarization dependency and active switching of liquid crystal HOEs, device fabrication, and properties of micro-LEDs (light-emitting diodes), and large design freedoms of metasurfaces. Afterwards, we discuss how these devices help enhance the AR and VR performance, with detailed description and analysis of some state-of-the-art architectures. Finally, we cast a perspective on potential developments and research directions of these photonic devices for future AR and VR displays.

Doubling the optical efficiency of VR systems with a directional backlight and a diffractive deflection film

We demonstrate an approach to double the optical efficiency of virtual reality (VR) systems based on a directional backlight and a diffractive deflection film (DDF). The directional backlight consists of a commercial collimated light-emitting diode (LED) array and a two-layer privacy film, while the DDF is a three-domain Pancharatnam-Berry (PB) phase lens. Such a PB phase lens was fabricated by the zone exposure and spin-coating method. The focal length of each domain is designed according to the imaging optics of the VR system. Our approach works well in both Fresnel and "pancake" VR systems. We also build the corresponding models in LightTools, and the simulation results are in good agreement with experiment. In experiment, we achieved a 2.25x optical efficiency enhancement for both systems, which agrees with the simulation results (2.48x for Fresnel and 2.44x for "pancake" systems) well. Potential application for high efficiency VR displays is foreseeable.