Optical see-through Maxwellian near-to-eye display with an enlarged eyebox

Enlarged Eye-Box Accommodation-Capable Augmented Reality with Hologram Replicas

Augmented reality (AR) technology has been widely applied across a variety of fields, with head-up displays (HUDs) being one of its prominent uses, offering immersive three-dimensional (3D) experiences and interaction with digital content and the real world. AR-HUDs face challenges such as limited field of view (FOV), small eye-box, bulky form factor, and absence of accommodation cue, often compromising trade-offs between these factors. Recently, optical waveguide based on pupil replication process has attracted increasing attention as an optical element for its compact form factor and exit-pupil expansion. Despite these advantages, current waveguide displays struggle to integrate visual information with real scenes because they do not produce accommodation-capable virtual content. In this paper, we introduce a lensless accommodation-capable holographic system based on a waveguide. Our system aims to expand the eye-box at the optimal viewing distance that provides the maximum FOV. We devised a formalized CGH algorithm based on bold assumption and two constraints and successfully performed numerical observation simulation. In optical experiments, accommodation-capable images with a maximum horizontal FOV of 7.0 degrees were successfully observed within an expanded eye-box of 9.18 mm at an optimal observation distance of 112 mm.

Theoretical efficiency limit of diffractive input couplers in augmented reality waveguides

Considerable efforts have been devoted to augmented reality (AR) displays to enable the immersive user experience in the wearable glasses form factor. Transparent waveguide combiners offer a compact solution to guide light from the microdisplay to the front of eyes while maintaining the see-through optical path to view the real world simultaneously. To deliver a realistic virtual image with low power consumption, the waveguide combiners need to have high efficiency and good image quality. One important limiting factor for the efficiency of diffractive waveguide combiners is the out-coupling problem in the input couplers, where the guided light interacts with the input gratings again and get partially out-coupled. In this study, we introduce a theoretical model to deterministically find the upper bound of the input efficiency of a uniform input grating, constrained only by Lorentz reciprocity and energy conservation. Our model considers the polarization management at the input coupler and can work for arbitrary input polarization state ensemble. Our model also provides the corresponding characteristics of the input coupler, such as the grating diffraction efficiencies and the Jones matrix of the polarization management components, to achieve the optimal input efficiency. Equipped with this theoretical model, we investigate how the upper bound of input efficiency varies with geometric parameters including the waveguide thickness, the projector pupil size, and the projector pupil relief distance. Our study shines light on the fundamental efficiency limit of input couplers in diffractive waveguide combiners and highlights the benefits of polarization control in improving the input efficiency.

Framework for optimizing AR waveguide in-coupler architectures

Waveguide displays have been shown to exhibit multiple interactions of light at the in-coupler diffractive surface, leading to light loss. Any losses at the in-coupler set a fundamental upper limit on the full-system efficiency. Furthermore, these losses vary spatially across the beam for each field, significantly decreasing the displayed image quality. We present a framework for alleviating the losses based on irradiance, efficiency, and MTF maps. We then derive and quantify the innate tradeoff between the in-coupling efficiency and the achievable modulation transfer function (MTF) characterizing image quality. Applying the framework, we show a new in-coupler architecture that mitigates the efficiency vs image quality tradeoff. In the example architecture, we demonstrate a computation speed that is 2,000 times faster than that of a commercial non-sequential ray tracer, enabling faster optimization and more thorough exploration of the parameter space. Results show that with this architecture, the in-coupling efficiency still meets the fundamental limit, while the MTF achieves the diffraction limit up to and including 30 cycles/deg, equivalent to 20/20 vision.

Effect of shrinkage in photopolymer film on the information transmitted through the holographic waveguide for near eye displays

Due to shrinkage in photopolymer materials, the angle of the reconstruction beam in holographic optical elements (HOEs) does not match with the Bragg condition, resulting in a decreased amount of light in the desired direction or loss of transmitted information to rematch the Bragg condition. Thus, to ensure final display features it is imperative to precompensate the shrinkage effect. We derived simplified expressions for precompensation in recording geometries of required HOEs in holographic waveguide-based Maxwellian near eye displays. An acceptable range of detuning from the Bragg angle is also analyzed. The experimentally measured 4.95% shrinkage in photopolymer film for 0° and 45° recording angles of beams was precompensated using -0.86∘ and 43.7° recording angles. Theoretical results are validated through simulation and experiments.

Lens array-based holographic 3D display with an expanded field of view and eyebox

Conventional spatial light modulator (SLM)-based holographic 3D display faces limited field of view (FOV) and eyebox, due to its limited pixel number. In this paper, a lens array is used to expand the FOV and eyebox of an SLM-based holographic display. The hologram is calculated to reconstruct a 3D sub-image array, each sub-image corresponding to a specific perspective of the 3D object. Then, the 3D sub-image array is imaged and magnified by the lens array to integrate to the original 3D image. The FOV is expanded due to the large numerical aperture of the lens, and the eyebox is expanded because the lens array generates multiple viewpoints with a large pitch. The optical experiment realizes a 17.6° FOV and 50 mm eyebox, which contains 4 × 4 viewpoints. Apparent motion parallax is observed through the viewpoint array, which is usually hard to observe in a conventional SLM-based holographic display. The proposed method provides a novel, to the best of our knowledge, way to expand the FOV and eyebox of holographic 3D display without increasing the total pixel number of the SLM.

Holographic display using layered computer-generated volume hologram

The spatial frequency of the reconstructed image of planar computer-generated hologram(CGH) is limited by the sampling interval and the lack of thickness. To break through this limitation of planar CGH, we propose a new computer-generated volume hologram(CGVH) for full-color dynamic holographic three-dimensional(3D) display, and an iteration-free layered CGVH generation method. The proposed CGVH is equivalent to a volume hologram sampled discretely in three directions. The generation method employs the layered angular spectral diffraction to calculate the light field in the layered CGVH, and then encodes it into a CGVH. Numerical simulation results show that the CGVH can accurately reconstruct full-color 3D objects, where better imaging quality, more concentrated diffraction energy, denser reconstructed spatial frequency information, and larger viewing angle are achieved. The proposed CGVH is expected to be applied to realize dynamic modulation, wavelength multiplexing, and angle multiplexing in various optical fields in the future.

Extended depth of field in augmented reality

The 3D display device shows an image with depth information. Conventional 3D display devices based on binocular parallax can focus accurately only on the depth of a specific screen. Because the human eye has a narrow depth of field (DOF) under normal circumstances, 3D displays that provide a relatively wide range of virtual depth areas have limitations on the DOF where clear 3D images are seen. To resolve this problem, it is necessary to find the optical conditions to extend the DOF and analyze the phenomena related to it. For this, by using the Rayleigh criterion and the Strehl ratio, a criterion for this extension of the DOF is suggested. A practical optical structure that can effectively extend the DOF is devised using a flat panel display. This optical structure could be applied to AR, VR, and MR in the field of near-eye displays. From the results of this research, the fundamental optical conditions and standards are proposed for 3D displays that will provide 3D images with extended DOF in the future. Furthermore, it is also expected that these conditions and criteria can be applied to optical designs for the required performance in the development of 3D displays in various fields.

Cross talk-free retinal projection display based on a holographic complementary viewpoint array

In near-eye displays (NEDs), retinal projection display (RPD) is one kind of promising technology to alleviate the vergence-accommodation conflict (VAC) issue due to its always-in-focus feature. Viewpoint replication is widely used to enlarge the limited eyebox. However, the mismatch between viewpoint interval and eye pupil diameter will cause the inter-viewpoint cross talk when multiple viewpoints enter the pupil simultaneously. In this Letter, a holographic complementary viewpoint method is proposed to solve this cross talk problem. Instead of avoiding observing multiple viewpoint images simultaneously, it is designed that multiple complementary viewpoints jointly project the complete image on the retina without cross talk. To do this, the target image is segmented into multiple sub-images, each multiplied with a corresponding partial spherical phase to converge to a specific complementary viewpoint. A group of complementary viewpoint enter the eye pupil simultaneously, and each viewpoint project a corresponding sub-image on a specific area of the retina and splice to a complete image. All of the complementary viewpoints are duplicated to an interlaced two-dimensional array to extend the eyebox in both horizontal and vertical directions. Optical experiment verifies that the proposed method could present smooth transition between viewpoints to avoid both inter-viewpoint cross talk and blank image issues.

Design of a high-speed circular polarization converter with a large field of view and wavelength range

A high-speed circular polarization converter (CPC) with a wide field of view (FOV) and wavelength range is designed and fabricated in this paper. The multi-waveplate combined structure is applied to constitute the basic configuration of the CPC for broadening the wavelength range. An electrically suppressed helix ferroelectric liquid crystal (ESHFLC) material with fast response is used as a medium for dynamic polarization operation. The compensation films are used to expand the FOV by attaching to the configuration. The simulation results demonstrate that the optimized CPC structure can achieve over 97% orthogonal circular polarization conversion efficiency in 300 nm bandwidth at a 90° viewing cone for both working states. Finally, we have experiments and the results show well consistency with the theoretical results.

Metagrating meets the geometry-based efficiency limit for AR waveguide in-couplers

Recently, augmented reality (AR) displays have attracted considerable attention due to the highly immersive and realistic viewer experience they can provide. One key challenge of AR displays is the fundamental trade-off between the extent of the field-of-view (FOV) and the size of the eyebox, set by the conservation of etendue sets this trade-off. Exit-pupil expansion (EPE) is one possible solution to this problem. However, it comes at the cost of distributing light over a larger area, decreasing the overall system's brightness. In this work, we show that the geometry of the waveguide and the in-coupler sets a fundamental limit on how efficient the combiner can be for a given FOV. This limit can be used as a tool for waveguide designers to benchmark the in-coupling efficiency of their in-coupler gratings. We design a metasurface-based grating (metagrating) and a commonly used SRG as in-couplers using the derived limit to guide optimization. We then compare the diffractive efficiencies of the two types of in-couplers to the theoretical efficiency limit. For our chosen waveguide geometry, the metagrating's 28% efficiency surpasses the SRG's 20% efficiency and nearly matches the geometry-based limit of 29% due to the superior angular response control of metasurfaces compared to SRGs. This work provides new insight into the efficiency limit of waveguide-based combiners and paves a novel path toward implementing metasurfaces in efficient waveguide AR displays.

In-Depth Review of Augmented Reality: Tracking Technologies, Development Tools, AR Displays, Collaborative AR, and Security Concerns

Augmented reality (AR) has gained enormous popularity and acceptance in the past few years. AR is indeed a combination of different immersive experiences and solutions that serve as integrated components to assemble and accelerate the augmented reality phenomena as a workable and marvelous adaptive solution for many realms. These solutions of AR include tracking as a means for keeping track of the point of reference to make virtual objects visible in a real scene. Similarly, display technologies combine the virtual and real world with the user's eye. Authoring tools provide platforms to develop AR applications by providing access to low-level libraries. The libraries can thereafter interact with the hardware of tracking sensors, cameras, and other technologies. In addition to this, advances in distributed computing and collaborative augmented reality also need stable solutions. The various participants can collaborate in an AR setting. The authors of this research have explored many solutions in this regard and present a comprehensive review to aid in doing research and improving different business transformations. However, during the course of this study, we identified that there is a lack of security solutions in various areas of collaborative AR (CAR), specifically in the area of distributed trust management in CAR. This research study also proposed a trusted CAR architecture with a use-case of tourism that can be used as a model for researchers with an interest in making secure AR-based remote communication sessions.

Lensless phase-only holographic retinal projection display based on the error diffusion algorithm

Holographic retinal projection display (RPD) can project images directly onto the retina without any lens by encoding a convergent spherical wave phase with the target images. Conventional amplitude-type holographic RPD suffers from strong zero-order light and conjugate. In this paper, a lensless phase-only holographic RPD based on error diffusion algorithm is demonstrated. It is found that direct error diffusion of the complex Fresnel hologram leads to low image quality. Thus, a post-addition phase method is proposed based on angular spectrum diffraction. The spherical wave phase is multiplied after error diffusion process, and acts as an imaging lens. In this way, the error diffusion functions better due to reduced phase difference between adjacent pixels, and a virtual image with improved quality is produced. The viewpoint is easily deflected just by changing the post-added spherical phase. A full-color holographic RPD with adjustable eyebox is demonstrated experimentally with time-multiplexing technique.

Design method of imaging optical systems using confocal flat phase elements

Imaging systems consisting of flat phase element such as diffractive optical element, holographic optical element, and metasurface have important applications in many fields. However, there is still a lack of generalized and efficient design methods of these systems, especially for systems with nonsymmetric configurations. We proposed a design method of imaging system consisting of flat phase elements based on confocal properties. The description of the generalized phase function for realizing point-to-point stigmatic imaging is derived. Given the focal length or magnification as well as the locations of the elements based on the design requirements, the phase functions can be calculated very fast and stigmatic imaging of the central field is realized. The systems can be taken as good starting points for further optimization, during which the rotationally symmetric or freeform phase terms can be added. Several design examples are demonstrated to validate the feasibility of the method. The proposed method increases design efficiency while decreasing the dependence on existing systems and skills significantly, and can be easily integrated into optical design software.

Multiplane holographic augmented reality head-up display with a real-virtual dual mode and large eyebox

We propose a multiplane augmented reality (AR) head-up display (HUD) with a real-virtual dual mode based on holographic optical elements (HOEs). The picture generation unit (PGU) is only a single free-focus projector, and the optical combiner includes a HOE lens (HOEL) for long-distance virtual image display and a HOE diffuser (HOED) for in-plane real image display. A HOED with directional scattering characteristics in the real image mode can significantly increase the size of the eyebox (EB) without increasing the size of the HOE, and a HOEL with a flexible design for the optical focal length in the virtual image mode can be used to achieve a different depth of the AR display. The proposed AR HUD system, which has a compact structure and offers high light transmittance, high energy usage, a multiplane display, and a large EB, is expected to be widely used in the future.

Compact near-eye display with a wide FOV and a wide eye box using 24 multiple-exposure holograms

A compact near-eye display with a 60° horizontal field of view, wide eye box of 5 mm, and high resolution of 720 p is proposed and developed by combining a transmission hologram that duplicates the beam of a scanning display and a reflection hologram that reflects duplicated beams toward the user's eye. The feasibility of the proposed near-eye display is demonstrated by examining the specifications and exposure of 24 multiple holograms. A compact NED that can display images with a horizontal FOV of 60° and that has a 6.2 mm × 4.8 mm eye box and 720 pixels vertical resolution is achieved.

Analysis on image quality of a holographic lens with a non-converging signal wave for compact near-eye displays

We analyze an image quality of a holographic lens (HL) in order to implement compact near-eye displays using a flat-panel-type micro-display panel. The proposed method utilizes a non-converging signal wave in a fabrication process of the HL, so that it provides affordable eye-box size with minimizing the aberration due to rays in the off-Bragg condition. For analyzing and optimizing the HL based on the non-converging signal wave, we introduce a comprehensive analysis model for an assessment of the image quality in the HL. The analysis model, inspired from the conventional lens design strategy for near-eye displays, evaluates the focal spot quality for incident rays forming each pixel with considering the on- and off-Bragg diffraction. The theoretical analysis is validated by simulation results using a volume hologram model in Zemax OpticStudio. As experimental verifications, we realize a prototype system using photopolymer-based HLs in a green color with the high transmittance of 89.3%. The image quality of the HLs is analyzed, which coincides well with the proposed analysis and assessment metric. By building a compact experimental setup employing the HL and a micro-organic light emitting diode display, we present see-through images with 8.0 mm of eye-box with reduced aberrations.

Tunable focal waveguide-based see-through display with negative liquid crystal lens

A see-through display based on a planar holographic waveguide with a tunable focal plane is presented. A negative liquid crystal lens is attached on the outcoupling location of the waveguide to manipulate the image distance. The continuous tunable range for the focal length is from negative infinity to -65 cm. The demonstrated prototype system provides 10.5° field-of-view (FOV) for the images not locating at infinity. The FOV for the images not locating at infinity is limited by the diameter of the liquid crystal lens. The lens function of the liquid crystal lens is polarization dependent. By controlling the polarization states of the real scene and the input information image, the liquid crystal lens keeps the see-through function for a real scene and simultaneously plays the role of a negative lens for the input information image. Compared to the see-through display system with a single focal plane, the presented system offers a more comfortable augmented reality (AR) experience.

Multiplexed Holographic Combiner with Extended Eye Box Fabricated by Wave Front Printing

We present an array-based volume holographic optical element (vHOE) recorded as an optical combiner for novel display applications such as smart glasses. The vHOE performs multiple, complex optical functions in the form of large off-axis to on-axis wave front transformations and an extended eye box implemented in the form of two distinct vertex points with red and green chromatic functions. The holographic combiner is fabricated by our extended immersion-based wave front printing setup, which provides extensive prototyping capabilities due to independent wave front modulation and large possible off-axis recording angles, enabling vHOEs in reflection with a wide range of different recording configurations. The presented vHOE is build up as an array of sub-holograms, where each element is recorded with individual optical functions. We introduce a design and fabrication method to combine two angular and two spectral functions in the volume grating of individual sub-holograms, demonstrating complex holographic elements with four multiplexed optical functions comprised in a single layer of photopolymer film. The introduced design and fabrication process allows the precise tuning of the vHOE’s diffractive properties to achieve well-balanced diffraction efficiencies and angular distributions between individual multiplexed functions.

Advanced liquid crystal devices for augmented reality and virtual reality displays: principles and applications

Liquid crystal displays (LCDs) and photonic devices play a pivotal role to augmented reality (AR) and virtual reality (VR). The recently emerging high-dynamic-range (HDR) mini-LED backlit LCDs significantly boost the image quality and brightness and reduce the power consumption for VR displays. Such a light engine is particularly attractive for compensating the optical loss of pancake structure to achieve compact and lightweight VR headsets. On the other hand, high-resolution-density, and high-brightness liquid-crystal-on-silicon (LCoS) is a promising image source for the see-through AR displays, especially under high ambient lighting conditions. Meanwhile, the high-speed LCoS spatial light modulators open a new door for holographic displays and focal surface displays. Finally, the ultrathin planar diffractive LC optical elements, such as geometric phase LC grating and lens, have found useful applications in AR and VR for enhancing resolution, widening field-of-view, suppressing chromatic aberrations, creating multiplanes to overcome the vergence-accommodation conflict, and dynamic pupil steering to achieve gaze-matched Maxwellian displays, just to name a few. The operation principles, potential applications, and future challenges of these advanced LC devices will be discussed.

Design of off-axis reflective imaging systems based on freeform holographic elements

Holographic optical element (HOE) can be used in many areas in optics due to its characteristics of thin structure, flexible wavefront reconstruction/control ability and angular/wavelength selectivity. In this paper, we propose a design method of off-axis reflective imaging systems based on freeform HOEs, which are fabricated by freeform wavefronts. The freeform HOEs offer many degrees of design freedom and can correct the aberrations in nonsymmetric imaging systems. The initial imaging system with freeform HOEs is generated using a point-by-point design approach, and is used for the preliminary design of the imaging system and the freeform recording system of each HOE. Then a joint optimization is conducted for all the systems, simultaneously considering the imaging performance, the diffraction efficiency, the system constraints and fabrication to get the final design results. To validate the feasibility and effectiveness of the proposed method, an off-axis reflective head-up display system with good performance based on freeform HOEs is designed and fabricated. Detailed procedures of the design and development process of the prototype are demonstrated.

Holographic super multi-view Maxwellian near-eye display with eyebox expansion

A holographic super multi-view (SMV) Maxwellian display based on flexible wavefront modulation is proposed for the first time, to the best of our knowledge. It solves the issue that the previous holographic Maxwellian displays could not provide depth cues for monocular vision. Different from the previous methods, two or more parallax images are multiplied by quadric phase distributions and converged to the viewpoints existing in the pupil to provide 3-D vision. A time division method is proposed to eliminate the cross talk caused by the coherence of different spherical waves. Experiments demonstrate that the proposed method can accurately reconstruct images at different depth without cross talk. The proposed method inherits the previous holographic Maxwellian display's advantages of flexible viewpoint position adjustment and large depth of field (DOF). Superior to geometric optics based SMV displays, the proposed system is compact without lens aberration since only a single spatial light modulator (SLM) is needed without any additional optical elements.

Pixelated volume holographic optical element for augmented reality 3D display

Augmented reality (AR) three-dimensional (3D) display is the hardware entrance of metaverse and attracts great interest. The fusion of physical world with 3D virtual images is non-trivial. In this paper, we proposed an AR 3D display based on a pixelated volume holographic optical element (P-VHOE). The see-through combiner is prepared by spatial multiplexing. A prototype of AR 3D display with high diffraction efficiency (78.59%), high transmission (>80%) and non-repeating views is realized. Virtual 3D objects with high fidelity in depth is reconstructed by P-VHOE, with a complex wavelet structural similarity (CW-SSIM) value of 0.9882. The proposed prototype provides an efficient solution for a compact glasses-free AR 3D display. Potential applications include window display, exhibition, education, teleconference.

Adjustable and continuous eyebox replication for a holographic Maxwellian near-eye display

A Maxwellian display presents always-focused images to the viewer, alleviating the vergence-accommodation conflict (VAC) in near-eye displays (NEDs). Recently, many methods of improving its limited eyebox have been proposed, among which viewpoint replication has attracted a lot of attention. However, double-image, blind-area, and image-shift effects always happen in typical eyebox-replication Maxwellian NEDs when the eye moves between the replicated viewpoints, which prevents these NEDs from being applied more widely. In this Letter, we propose a method for designing a holographic Maxwellian NED system with continuous eyebox replication as well as flexible interval adjustment by changing the projection angles of the reconstructed images. Thus, holograms corresponding to the positions of different viewpoints are calculated to match the interval of the replicated viewpoints with the human pupil diameter, making it possible to eliminate or alleviate double-image or blind-area effects. Also, seamless viewpoint conversion in the eyebox is achieved by aligning the images of adjacent viewpoints on the retina via hologram pre-processing independently. These effects are verified successfully in optical experiments and have the potential to be applied in near-eye three-dimensional displays without VAC.

Color-selective geometric-phase lenses for focusing and imaging based on liquid crystal polymer films

The geometric-phase lens (GPLs) with small form factor compared to traditional refractive lenses has been identified as a compelling solution in augmented-/virtual-/mixed-reality (AR/VR/MR) headsets. Formed either with liquid crystals (LCs) or metasurfaces, the GPL is a type of emerging leading technology that implements the arbitrary aspheric phase to realize low loss and minimal ghosting. However, the inherent chromatic abberation (CA) of GPLs can significantly degrade the image quality. A possible solution is the independent spectral phase implementation for RGB. In this work, we propose the design of three types of multi-twist LC based color-selective GPLs (CS-GPLs), exhibiting highly chromatic efficiency spectra with diameter 30 mm, focal length around 41.2~mm, and F -number 1.37. Through theoretical and experimental validation, each type of CS-GPL manifests high diffraction efficiency (>91%) on respective primary color of orthogonal polarization and high transmission on the complementary color of input polarization. The triplet composed by RGB CS-GPLs demonstrates relative contrast ratio and minimal ghosting. The strong color and polarization dependency of CS-GPLs not only provide a novel technique to mitigate CA but also offer more design freedom in the AR/VR/MR polarization and imaging system.

Waveguide-type Maxwellian near-eye display using a pin-mirror holographic optical element array

We propose a novel, to the best of our knowledge, waveguide-type optical see-through Maxwellian near-eye display for augmented reality. A pin-mirror holographic optical element (HOE) array enables the Maxwellian view and eye-box replication. Virtual images with deep depth of field are presented by each pin-mirror HOE, alleviating the discrepancy between vergence and accommodation distance. The compact form factor is achieved by the thin waveguide and HOE couplers.

Pupil steering holographic display for pre-operative vision screening of cataracts

Cataract is the most common cause of preventable blindness and vision loss where the only treatment is surgical replacement of the natural lens with an intraocular lens. Computer-generated holography (CGH) enables to control phase, size, and shape of the light beam entering through the eye-pupil. We developed a holographic vision simulator to assess visual acuity for patients to experience the postoperative corrected vision before going through surgery. A holographically shaped light beam is directed onto the retina using small non-cataractous regions of the lens with the help of a pupil tracker. A Snellen chart hologram is shown to subjects at desired depth with myopia and hyperopia correction. Tests with 13 patients demonstrated substantial improvements in visual acuity and the simulator results are consistent with the post-operative vision tests. Holographic simulator overperforms the existing vision simulators, which are limited to static pinhole exit pupils and incapable of correcting aberrations.

Waveguide-type see-through dual focus near-eye display with a polarization grating

Waveguide-type near-eye displays have useful properties such as compact form factor, lightweight and see-through capability. Conventional systems, however, support only a single image plane fixed at a certain distance, which may induce eye fatigue due to the vergence-accommodation conflict. In this paper, we propose a waveguide-type near-eye display with two image planes using a polarization grating. Two images with orthogonal polarizations propagate within the waveguide with different total internal reflection angles and form virtual images at different distances. The use of the polarization grating and two pairs of holographic optical elements enables dual image plane formation by a single waveguide with high transparency for the real scene. Optical experiments confirm the principle of the proposed optical system.

Conjugate wavefront encoding: an efficient eyebox extension approach for holographic Maxwellian near-eye display

Conventional holographic display suffers from the conjugate light issue. In this Letter, we propose to efficiently extend the eyebox of holographic Maxwellian near-eye display by encoding the conjugate wavefront as the multiplication of plane wave phase with the target image. It is interesting that after being focused by the lens, the generated conjugate viewpoints also present erect virtual images with the same image quality as the signal viewpoints. Multiple plane wave encoding is used for eyebox extension, and, because of the utilization of conjugate light, the effect of eyebox extension is doubled. That is, the space bandwidth of the amplitude-type hologram is fully used. A speckless holographic image is produced in mid-air with high quality within a large depth range. The proposed display is compact and promising for the augmented reality near-eye display. Furthermore, it may inspire better solutions for the conjugate light issue of amplitude-type holography.

Augmented reality and virtual reality displays: emerging technologies and future perspectives

With rapid advances in high-speed communication and computation, augmented reality (AR) and virtual reality (VR) are emerging as next-generation display platforms for deeper human-digital interactions. Nonetheless, to simultaneously match the exceptional performance of human vision and keep the near-eye display module compact and lightweight imposes unprecedented challenges on optical engineering. Fortunately, recent progress in holographic optical elements (HOEs) and lithography-enabled devices provide innovative ways to tackle these obstacles in AR and VR that are otherwise difficult with traditional optics. In this review, we begin with introducing the basic structures of AR and VR headsets, and then describing the operation principles of various HOEs and lithography-enabled devices. Their properties are analyzed in detail, including strong selectivity on wavelength and incident angle, and multiplexing ability of volume HOEs, polarization dependency and active switching of liquid crystal HOEs, device fabrication, and properties of micro-LEDs (light-emitting diodes), and large design freedoms of metasurfaces. Afterwards, we discuss how these devices help enhance the AR and VR performance, with detailed description and analysis of some state-of-the-art architectures. Finally, we cast a perspective on potential developments and research directions of these photonic devices for future AR and VR displays.

Lensless full-color holographic Maxwellian near-eye display with a horizontal eyebox expansion

A lensless full-color holographic Maxwellian near-eye display using a single amplitude-type spatial light modulator is proposed in this Letter. The color holographic image is directly projected onto the retina without any eyepiece. The color crosstalk is clearly separated from the signal in the space owing to the encoded spherical wave and carrier wave. An aperture numerical filter and a real polarized filter are used at the pupil plane to accurately stop the crosstalk light. A high-quality dynamic speckless color holographic image was produced in the mid-air within a specific depth range. The horizontal eyebox expansion is achieved simply through multiple spherical wave encoding and verified through an optical experiment. The proposed display is compact and promising as the augmented reality near-eye display.

Optical See-through 2D/3D Compatible Display Using Variable-Focus Lens and Multiplexed Holographic Optical Elements

An optical see-through two-dimensional (2D)/three-dimensional (3D) compatible display using variable-focus lens and multiplexed holographic optical elements (MHOE) is presented. It mainly consists of a MHOE, a variable-focus lens and a projection display device. The customized MHOE, by using the angular multiplexing technology of volumetric holographic grating, records the scattering wavefront and spherical wavefront array required for 2D/3D compatible display. In particular, we proposed a feasible method to switch the 2D and 3D display modes by using a variable-focus lens in the reconstruction process. The proposed system solves the problem of bulky volume, and makes the MHOE more efficient to use. Based on the requirements of 2D and 3D displays, we calculated the liquid pumping volume of the variable-focus lens under two kinds of diopters.

Holographic near-eye display based on complex amplitude modulation with band-limited zone plates

A holographic near-eye display (NED) system based on complex amplitude modulation (CAM) with band-limited zone plates is proposed. The whole system mainly consists of a phase-only spatial light modulator (SLM), an Abbe-Porter filter system, an eyepiece, and an image combiner. The point source method based on band limited zone plates is used to accurately control the bandwidth of the target complex amplitude. The effects of intensity modulation coefficient γ in the frequency-filtering method on the intensity and the quality of reconstructed images are analyzed, which provide a judgment basis for selecting the appropriate value of γ. We also derive the expressions of the field of view (FOV) and exit pupil of the NED system. Since the holographic image is magnified in two steps in this system, the large FOV can be obtained. The optical experimental results show that the proposed system can provide a dynamic holographic three-dimensional (3D) augmented reality (AR) display with a 23.5° horizontal FOV.

Extending eyebox with tunable viewpoints for see-through near-eye display

The Maxwellian display presents always-focused images to the viewer, alleviating the vergence-accommodation conflict (VAC) in near-eye displays (NEDs). However, the limited eyebox of the typical Maxwellian display prevents it from wider applications. We propose a Maxwellian see-through NED based on a multiplexed holographic optical element (HOE) and polarization gratings (PGs) to extend the eyebox by viewpoint multiplication. The multiplexed HOE functions as multiple convex lenses to form multiple viewpoints, which are copied to different locations by PGs. To mitigate the imaging problem that multiple viewpoints or no viewpoints enter the eye pupil, the viewpoints can be tuned by mechanically moving a PG. We implement our method in a proof-of-concept system. The optical experiments confirm that the proposed display system provides always in-focus images within a 12 mm eyebox in the horizontal direction with a 32.7° diagonal field of view (FOV) and a 16.5 mm eye relief (ERF), and its viewpoints are tunable to match the actual eye pupil size. Compared with other techniques to extend the eyebox of Maxwellian displays, the proposed method shows competitive performances of a large eyebox, adaptability to the eye pupil size, and focus cues within a large depth range.

Aberration-free pupil steerable Maxwellian display for augmented reality with cholesteric liquid crystal holographic lenses

Maxwellian displays offer unique features like always-in-focus quality, high efficiency, and large field-of-view, but its small eyebox remains a major challenge for augmented reality. To enlarge the eyebox, pupil steering is a promising approach. However, previous pupil steering methods generally rely on changing the incident light angle on the lens coupler, which results in serious aberrations. In this Letter, we demonstrate a pupil steerable see-through Maxwellian display incorporating novel cholesteric liquid crystal (CLC) holographic lenses. By actively modulating the polarization state of the incident light, we can schematically choose which holographic lens to function, which fundamentally eliminates the aberrations.

Full-color retinal-projection near-eye display using a multiplexing-encoding holographic method

We propose a novel method to construct an optical see-through retinal-projection near-eye display using the Maxwellian view and a holographic method. To provide a dynamic full-color virtual image, a single phase-only spatial light modulator (SLM) was employed in conjunction with a multiplexing-encoding holographic method. Holographic virtual images can be directly projected onto the retina using an optical see-through eyepiece. The virtual image is sufficiently clear when the crystal lens can focus at different depths; the presented method can resolve convergence and accommodation conflict during the use of near-eye displays. To verify the proposed method, a proof-of-concept prototype was developed to provide vivid virtual images alongside real-world ones.

Enlarging the Eyebox of Maxwellian Displays with a Customized Liquid Crystal Dammann Grating

The Maxwellian view offers a promising approach to overcome the vergence-accommodation conflict in near-eye displays, however, its pinhole-like imaging naturally limits the eyebox size. Here, a liquid crystal polymer-based Dammann grating with evenly distributed energy among different diffraction orders is developed to enlarge the eyebox of Maxwellian view displays via pupil replication. In the experiment, a 3-by-3 Dammann grating is designed and fabricated, which exhibits good efficiency and high brightness uniformity. We further construct a proof-of-concept Maxwellian view display breadboard by inserting the Dammann grating into the optical system. The prototype successfully demonstrates the enlarged eyebox and full-color operation. Our work provides a promising route of eyebox expansion in Maxwellian view displays while maintaining full-color operation, simple system configuration, compactness, and lightweight.

Hybrid holographic Maxwellian near-eye display based on spherical wave and plane wave reconstruction for augmented reality display

The holographic Maxwellian display is a promising technique for augmented reality (AR) display because it solves the vergence-accommodation conflict while presenting a high-resolution display. However, conventional holographic Maxwellian display has the inherent trade-off between depth of field (DOF) and image quality. In this paper, two types of holographic Maxwellian displays, the spherical wave type and the plane wave type, are proposed and analyzed. The spherical wavefront and the plane wavefront are produced by a spatial light modulator (SLM) for Maxwellian display. Due to the focusing properties of different wavefronts, the two types of display have complementary DOF ranges. A hybrid approach combining the spherical wavefront and plane wavefront is proposed for a large DOF with high image quality. An optical experiment with AR display is demonstrated to verify the proposed method.

Eye-box extended retinal projection type near-eye display with multiple independent viewpoints [Invited]

We introduce an approach to expand the eye-box in a retinal-projection-based near-eye display. The retinal projection display has the advantage of providing clear images in a wide depth range; however, it has difficulty in practical use with a narrow eye-box. Here, we propose a method to enhance the eye-box of the retinal projection display by generating multiple independent viewpoints, maintaining a wide depth of field. The method prevents images projected from multiple viewpoints from overlapping one other in the retina. As a result, our proposed system can provide a continuous image over a wide viewing angle without an eye tracker or image update. We discuss the optical design for the proposed method and verify its feasibility through simulation and experiment.

Design of achromatic augmented reality visors based on composite metasurfaces

A compact near-eye visor (NEV) system that can guide light from a display to the eye could transform augmented reality (AR) technology. Unfortunately, existing implementations of such an NEV either suffer from small field of view or chromatic aberrations. See-through quality and bulkiness further make the overall performance of the visors unsuitable for a seamless user experience. Metasurfaces are an emerging class of nanophotonic elements that can dramatically reduce the size of optical elements while enhancing functionality. In this paper, we present a design of composite metasurfaces for an ultracompact NEV. We simulate the performance of a proof-of-principle visor corrected for chromatic aberrations while providing a large display field of view (>77^∘ both horizontally and vertically) and good see-through quality [>70% transmission and less than a wavelength root mean-square (RMS) wavefront error over the whole visible wavelength range] as needed for an immersive AR experience.

Maxwellian near-eye display with an expanded eyebox

Maxwellian view systems can be employed to circumvent the vergence-accommodation conflict in near-eye displays (NEDs), which directly project images onto the retina regardless of the human eye's depth of focus. However, Maxwellian view optics typically have a limited eyebox, which prevents broader applications of this architecture in NEDs. Here, we demonstrate a thin-film two-dimensional beam deflector composed of multi-twist broad-band Pancharatnam-Berry deflectors to mitigate this limitation via eyebox replication. Based on experimental validation, our proposed design can display always-focused full-color images within a 9 mm × 9 mm eyebox and thus mitigate the limitation of conventional Maxwellian displays while adding negligible weight and volume.

Submillisecond-Response Polymer Network Liquid Crystal Phase Modulators

A submillisecond-response and light scattering-free polymer-network liquid crystal (PNLC) for infrared spatial light modulators is demonstrated. Our new liquid crystal host exhibits a higher birefringence, comparable dielectric anisotropy, and slightly lower visco-elastic constant than a commonly employed commercial material, HTG-135200. Moreover, the electro-optical performance of our PNLCs with different monomer concentrations, cell gaps, and liquid crystal (LC) hosts is compared and discussed from four aspects: operating voltage, hysteresis, relaxation time, and light scattering loss. The temperature effect on hysteresis is also analyzed. Potential applications of PNLCs for laser beam steering and spatial light modulators especially in the infrared region are foreseeable.

Toward the next-generation VR/AR optics: a review of holographic near-eye displays from a human-centric perspective

Wearable near-eye displays for virtual and augmented reality (VR/AR) have seen enormous growth in recent years. While researchers are exploiting a plethora of techniques to create life-like three-dimensional (3D) objects, there is a lack of awareness of the role of human perception in guiding the hardware development. An ultimate VR/AR headset must integrate the display, sensors, and processors in a compact enclosure that people can comfortably wear for a long time while allowing a superior immersion experience and user-friendly human-computer interaction. Compared with other 3D displays, the holographic display has unique advantages in providing natural depth cues and correcting eye aberrations. Therefore, it holds great promise to be the enabling technology for next-generation VR/AR devices. In this review, we survey the recent progress in holographic near-eye displays from the human-centric perspective.

Full-color see-through near-eye holographic display with 80° field of view and an expanded eye-box

A full-color see-through near-eye holographic display is proposed with 80° field of view (FOV) and an expanded eye-box. The system is based on a holographic optical element (HOE) to achieve a large FOV while the image light is focused at the entrance to human pupil and the image of entire field enters human eye. As we know, one of the major limitations of the large FOV holographic display system is the small eye-box that needs to be expanded. We design a double layer diffraction structure for HOE to realize eye-box expansion. The HOE consists of two non-uniform volume holographic gratings and a transparent substrate. Two fabricated holographic gratings are attached to front and back surfaces of the substrate to multiplex image light and achieve eye-box expansion. Simultaneously, the HOE is also manufactured for RGB colors to realize full-color display. The experiment results show that our proposed display system develops 80° round FOV and an enlarged eye-box of 7.5 mm (H) ×5 mm (V) at the same time. The dynamic display ability is also tested in the experiments. The proposed system provides a new solution for the practical application of augmented reality display.

1000-volume/s high-speed volumetric display for high-speed HMD

In this paper, we propose a high-speed volumetric display principle that can solve two problems faced by three-dimensional displays using the parallax stereo principle (namely, the vergence-accommodation conflict and display latency) and we report evaluation results. The proposed display method can update a set of images at different depths at 1000 Hz and is consistent with accommodation. The method selects the depth position in microseconds by combining a high-speed variable-focus lens that vibrates at about 69 kHz and sub-microsecond control of illumination light using an LED. By turning on the LED for only a few hundred nanoseconds when the refractive power of the lens is at a certain value, an image can be presented with this specific refractive power. The optical system is combined with a DMD to form an image at each depth. 3D information consisting of multiple planes in the depth direction can be presented at a high refresh rate by switching the images and changing the refractive power at high speed. A proof-of-concept system was developed to show the validity of the proposed display principle. The system successfully displayed 3D information consisting of six binary images at an update rate of 1000 volume/s.

Augmented Reality and Virtual Reality Displays: Perspectives and Challenges

As one of the most promising candidates for next-generation mobile platform, augmented reality (AR) and virtual reality (VR) have potential to revolutionize the ways we perceive and interact with various digital information. In the meantime, recent advances in display and optical technologies, together with the rapidly developing digital processers, offer new development directions to advancing the near-eye display systems further. In this perspective paper, we start by analyzing the optical requirements in near-eye displays poised by the human visual system and then compare it against the specifications of state-of-the-art devices, which reasonably shows the main challenges in near-eye displays at the present stage. Afterward, potential solutions to address these challenges in both AR and VR displays are presented case by case, including the most recent optical research and development, which are already or have the potential to be industrialized for extended reality displays.

Geometrical-lightguide-based head-mounted lightfield displays using polymer-dispersed liquid-crystal films

Integrating the promising waveguide or lightguide optical combiners to head-mounted lightfield display (LF-HMD) systems offers a great opportunity to achieve both a compact optical see-through capability required for augmented or mixed reality applications and true 3D scene with correct focus cues required for mitigating the well-known vergence-accommodation conflict. Due to the non-sequential ray propagation nature of these flat combiners and the ray construction nature of a lightfield display engine, however, adapting these two technologies to each other confronts several significant challenges. In this paper, we explore the feasibility of combining an integral-imaging-based lightfield display engine with a geometrical lightguide based on microstructure mirror arrays. The image artifacts and the key challenges in a lightguide-based LF-HMD system are systematically analyzed and are further quantified via a non-sequential ray tracing simulation. We further propose to utilize polymer-dispersed liquid-crystal (PDLC) films to address the inherent problems associated with a lightguide combiner such as increasing the viewing density and improving the image coupling uniformity. We finally demonstrate, to our best knowledge, the first lightguide-based LF-HMD system that takes the advantages of both the compact form factor of a lightguide combiner and the true 3D virtual image rendering capability of a lightfield display.

Development of an ultra-compact optical combiner for augmented reality using geometric phase lenses

We present an ultra-compact optical combiner using a waveguide and geometric phase lenses (GPL) for augmented reality displays. By sandwiching the output coupler of a planar waveguide between two flat, thin GPLs, we create two optical sub-systems with different optical powers for displaying the virtual objects and transmitting the ambient light rays, respectively. We implemented our method in a scanning-based Maxwellian display and demonstrated the augmentation of an all-in-focus Maxwellian-view image with real-world objects within a 15° field of view. Our device is light (50 g) and thin (4 mm), making it well suited for wearable applications.

Retinal projection type lightguide-based near-eye display with switchable viewpoints

We present a retinal-projection-based near-eye display with switchable multiple viewpoints by polarization-multiplexing. Active switching of viewpoints is provided by the polarization grating, multiplexed holographic optical elements and polarization-dependent eyepiece lens that can generate one of the dual-divided focus groups according to the pupil position. The lightguide-combined optical devices have a potential to enable a wide field of view (FOV) and short eye relief with compact form factor. Our proposed system can support a pupil movement with an extended eyebox and mitigate image problem caused by duplicated viewpoints. We discuss the optical design for guiding system and demonstrate that proof-of-concept system provides all-in-focus images with 37 degrees FOV and 16 mm eyebox in horizontal direction.

Holographic near-eye display with continuously expanded eyebox using two-dimensional replication and angular spectrum wrapping

Holographic near-eye displays present true three-dimensional images with full monocular depth cues. In this paper, we propose a technique to expand the eyebox of the holographic near-eye displays. The base eyebox of the holographic near-eye displays is determined by the space bandwidth product of a spatial light modulator. The proposed technique replicates and stitches the base eyebox by the combined use of a holographic optical element and high order diffractions of the spatial light modulator, achieving horizontally and vertically expanded eyebox. An angular spectrum wrapping technique is also applied to alleviate image distortions observed at the boundaries between the replicated base eyeboxes.

Computational holographic Maxwellian near-eye display with an expanded eyebox

The Maxwellian near-eye displays have attracted growing interest in various applications. By using a confined pupil, a Maxwellian display presents an all-in-focus image to the viewer where the image formed on the retina is independent of the optical power of the eye. Despite being a promising technique, current Maxwellian near-eye displays suffer from various limitations such as a small eyebox, a bulky setup and a high cost. To overcome these drawbacks, we present a holographic Maxwellian near-eye display based on computational imaging. By encoding a complex wavefront into amplitude-only signals, we can readily display the computed histogram on a widely-accessible device such as a liquid-crystal or digital light processing display, creating an all-in-focus virtual image augmented on the real-world objects. Additionally, to expand the eyebox, we multiplex the hologram with multiple off-axis plane waves, duplicating the pupils into an array. The resultant method features a compact form factor because it requires only one active electronic component, lending credence to its wearable applications.