Retinal image quality in near-eye pupil-steered systems

The Perceptual Science of Augmented Reality

Augmented reality (AR) systems aim to alter our view of the world and enable us to see things that are not actually there. The resulting discrepancy between perception and reality can create compelling entertainment and can support innovative approaches to education, guidance, and assistive tools. However, building an AR system that effectively integrates with our natural visual experience is hard. AR systems often suffer from visual limitations and artifacts, and addressing these flaws requires basic knowledge of perception. At the same time, AR system development can serve as a catalyst that drives innovative new research in perceptual science. This review describes recent perceptual research pertinent to and driven by modern AR systems, with the goal of highlighting thought-provoking areas of inquiry and open questions.

Cross talk-free retinal projection display based on a holographic complementary viewpoint array

In near-eye displays (NEDs), retinal projection display (RPD) is one kind of promising technology to alleviate the vergence-accommodation conflict (VAC) issue due to its always-in-focus feature. Viewpoint replication is widely used to enlarge the limited eyebox. However, the mismatch between viewpoint interval and eye pupil diameter will cause the inter-viewpoint cross talk when multiple viewpoints enter the pupil simultaneously. In this Letter, a holographic complementary viewpoint method is proposed to solve this cross talk problem. Instead of avoiding observing multiple viewpoint images simultaneously, it is designed that multiple complementary viewpoints jointly project the complete image on the retina without cross talk. To do this, the target image is segmented into multiple sub-images, each multiplied with a corresponding partial spherical phase to converge to a specific complementary viewpoint. A group of complementary viewpoint enter the eye pupil simultaneously, and each viewpoint project a corresponding sub-image on a specific area of the retina and splice to a complete image. All of the complementary viewpoints are duplicated to an interlaced two-dimensional array to extend the eyebox in both horizontal and vertical directions. Optical experiment verifies that the proposed method could present smooth transition between viewpoints to avoid both inter-viewpoint cross talk and blank image issues.

In-Depth Review of Augmented Reality: Tracking Technologies, Development Tools, AR Displays, Collaborative AR, and Security Concerns

Augmented reality (AR) has gained enormous popularity and acceptance in the past few years. AR is indeed a combination of different immersive experiences and solutions that serve as integrated components to assemble and accelerate the augmented reality phenomena as a workable and marvelous adaptive solution for many realms. These solutions of AR include tracking as a means for keeping track of the point of reference to make virtual objects visible in a real scene. Similarly, display technologies combine the virtual and real world with the user's eye. Authoring tools provide platforms to develop AR applications by providing access to low-level libraries. The libraries can thereafter interact with the hardware of tracking sensors, cameras, and other technologies. In addition to this, advances in distributed computing and collaborative augmented reality also need stable solutions. The various participants can collaborate in an AR setting. The authors of this research have explored many solutions in this regard and present a comprehensive review to aid in doing research and improving different business transformations. However, during the course of this study, we identified that there is a lack of security solutions in various areas of collaborative AR (CAR), specifically in the area of distributed trust management in CAR. This research study also proposed a trusted CAR architecture with a use-case of tourism that can be used as a model for researchers with an interest in making secure AR-based remote communication sessions.

Augmented reality and virtual reality displays: emerging technologies and future perspectives

With rapid advances in high-speed communication and computation, augmented reality (AR) and virtual reality (VR) are emerging as next-generation display platforms for deeper human-digital interactions. Nonetheless, to simultaneously match the exceptional performance of human vision and keep the near-eye display module compact and lightweight imposes unprecedented challenges on optical engineering. Fortunately, recent progress in holographic optical elements (HOEs) and lithography-enabled devices provide innovative ways to tackle these obstacles in AR and VR that are otherwise difficult with traditional optics. In this review, we begin with introducing the basic structures of AR and VR headsets, and then describing the operation principles of various HOEs and lithography-enabled devices. Their properties are analyzed in detail, including strong selectivity on wavelength and incident angle, and multiplexing ability of volume HOEs, polarization dependency and active switching of liquid crystal HOEs, device fabrication, and properties of micro-LEDs (light-emitting diodes), and large design freedoms of metasurfaces. Afterwards, we discuss how these devices help enhance the AR and VR performance, with detailed description and analysis of some state-of-the-art architectures. Finally, we cast a perspective on potential developments and research directions of these photonic devices for future AR and VR displays.

LensIet VR: Thin, Flat and Wide-FOV Virtual Reality Display Using Fresnel Lens and LensIet Array

We propose a new thin and flat virtual reality (VR) display design using a Fresnel lenslet array, a Fresnel lens, and a polarization-based optical folding technique. The proposed optical system has a wide field of view (FOV) of 102°x102°, a wide eye-box of 8.8 mm, and an ergonomic eye-relief of 20 mm. Simultaneously, only 3.3 mm of physical distance is required between the display panel and the lens, so that the integrated VR display can have a compact form factor like sunglasses. Moreover, since all lenslet of the lenslet array is designed to operate under on-axis condition with low aberration, the discontinuous pupil swim distortion between the lenslets is hardly observed. In addition, all on-axis lenslets can be designed identically, reducing production cost, and even off-the-shelf Fresnel optics can be used. In this paper, we introduce how we design system parameters and analyze system performance. Finally, we demonstrate two prototypes and experimentally verify that the proposed VR display system has the expected performance while having a glasses-like form factor.

Extending eyebox with tunable viewpoints for see-through near-eye display

The Maxwellian display presents always-focused images to the viewer, alleviating the vergence-accommodation conflict (VAC) in near-eye displays (NEDs). However, the limited eyebox of the typical Maxwellian display prevents it from wider applications. We propose a Maxwellian see-through NED based on a multiplexed holographic optical element (HOE) and polarization gratings (PGs) to extend the eyebox by viewpoint multiplication. The multiplexed HOE functions as multiple convex lenses to form multiple viewpoints, which are copied to different locations by PGs. To mitigate the imaging problem that multiple viewpoints or no viewpoints enter the eye pupil, the viewpoints can be tuned by mechanically moving a PG. We implement our method in a proof-of-concept system. The optical experiments confirm that the proposed display system provides always in-focus images within a 12 mm eyebox in the horizontal direction with a 32.7° diagonal field of view (FOV) and a 16.5 mm eye relief (ERF), and its viewpoints are tunable to match the actual eye pupil size. Compared with other techniques to extend the eyebox of Maxwellian displays, the proposed method shows competitive performances of a large eyebox, adaptability to the eye pupil size, and focus cues within a large depth range.

Eye-box extended retinal projection type near-eye display with multiple independent viewpoints [Invited]

We introduce an approach to expand the eye-box in a retinal-projection-based near-eye display. The retinal projection display has the advantage of providing clear images in a wide depth range; however, it has difficulty in practical use with a narrow eye-box. Here, we propose a method to enhance the eye-box of the retinal projection display by generating multiple independent viewpoints, maintaining a wide depth of field. The method prevents images projected from multiple viewpoints from overlapping one other in the retina. As a result, our proposed system can provide a continuous image over a wide viewing angle without an eye tracker or image update. We discuss the optical design for the proposed method and verify its feasibility through simulation and experiment.

A perceptual eyebox for near-eye displays

In near-eye display systems that support three-dimensional (3D) augmented and virtual reality, a central factor in determining the user experience is the size of the eyebox. The eyebox refers to a volume where the eye receives an acceptable view of the image with respect to a set of criteria and thresholds. The size and location of this volume are primarily driven by optical architecture choices in which designers trade-off a number of constraints, such as field of view, image quality, and product design. It is thus important to clearly quantify how design decisions affect the properties of the eyebox. Recent work has started evaluating the eyebox in 3D based purely on optical criteria. However, such analyses do not incorporate perceptual criteria that determine visual quality, which are particularly important for binocular 3D systems. To address this limitation, we introduce the framework of a perceptual eyebox. The perceptual eyebox is the volume where the eye(s) must be located for the user to experience a visual percept falling within a perceptually-defined criterion. We combine optical and perceptual data to characterize an example perceptual eyebox for display visibility in augmented reality. The key contributions in this paper include: comparing the perceptual eyebox for monocular and binocular display designs, modeling the effects of user eye separation, and examining the effects of eye rotation on the eyebox volume.

Retinal projection type lightguide-based near-eye display with switchable viewpoints

We present a retinal-projection-based near-eye display with switchable multiple viewpoints by polarization-multiplexing. Active switching of viewpoints is provided by the polarization grating, multiplexed holographic optical elements and polarization-dependent eyepiece lens that can generate one of the dual-divided focus groups according to the pupil position. The lightguide-combined optical devices have a potential to enable a wide field of view (FOV) and short eye relief with compact form factor. Our proposed system can support a pupil movement with an extended eyebox and mitigate image problem caused by duplicated viewpoints. We discuss the optical design for guiding system and demonstrate that proof-of-concept system provides all-in-focus images with 37 degrees FOV and 16 mm eyebox in horizontal direction.