Design of lensless retinal scanning display with diffractive optical element

Augmented reality system based on the integration of polarization-independent metalens and micro-LEDs

Augmented reality (AR), a technology that superimposes virtual information onto a user's direct view of real-world scenes, is considered one of the next-generation display technologies and has been attracting considerable attention. Here, we propose a flat optic AR system that synergistically integrates a polarization-independent metalens with micro light-emitting diodes (LEDs). A key component is a meticulously designed metalens with a numerical aperture of 0.25, providing a simulated focusing efficiency of approximately 76.5% at a wavelength of 532 nm. Furthermore, the laser measurement system substantiates that the fabricated metalens achieves a focusing efficiency of 70.8%. By exploiting the reversibility of light characteristics, the metalens transforms the divergent light from green micro-LEDs into a collimated beam that passes through the pupil and images on the retina. Monochromatic pixels with a size of 5×5 µm2 and a pitch of 10 µm can be distinctly resolved with a power efficiency of 50%. This work illustrates the feasibility of integrating the metalens with microdisplays, realizing a high-efficiency AR device without the need for additional optical components and showcasing great potential for the development of near-eye display applications.

Framework for optimizing AR waveguide in-coupler architectures

Waveguide displays have been shown to exhibit multiple interactions of light at the in-coupler diffractive surface, leading to light loss. Any losses at the in-coupler set a fundamental upper limit on the full-system efficiency. Furthermore, these losses vary spatially across the beam for each field, significantly decreasing the displayed image quality. We present a framework for alleviating the losses based on irradiance, efficiency, and MTF maps. We then derive and quantify the innate tradeoff between the in-coupling efficiency and the achievable modulation transfer function (MTF) characterizing image quality. Applying the framework, we show a new in-coupler architecture that mitigates the efficiency vs image quality tradeoff. In the example architecture, we demonstrate a computation speed that is 2,000 times faster than that of a commercial non-sequential ray tracer, enabling faster optimization and more thorough exploration of the parameter space. Results show that with this architecture, the in-coupling efficiency still meets the fundamental limit, while the MTF achieves the diffraction limit up to and including 30 cycles/deg, equivalent to 20/20 vision.

A Depth-Enhanced Holographic Super Multi-View Display Based on Depth Segmentation

A super multi-view (SMV) near-eye display (NED) effectively provides depth cues for three-dimensional (3D) display by projecting multiple viewpoint or parallax images onto the retina simultaneously. Previous SMV NED have suffered from a limited depth of field (DOF) due to a fixed image plane. In this paper, a holographic SMV Maxwellian display based on depth segmentation is proposed to enhance the DOF. The proposed approach involves capturing a set of parallax images and their corresponding depth maps. According to the depth maps, the parallax images are segmented into N sub-parallax images at different depth ranges. These sub-parallax images are then projected onto N image-recording planes (IRPs) of the corresponding depth for hologram computation. The wavefront at each IRP is calculated by multiplying the sub-parallax images with the corresponding spherical wave phases. Then, they are propagated to the hologram plane and added together to form a DOF-enhanced hologram. The simulation and experimental results are obtained to validate the effectiveness of the proposed method in extending the DOF of the holographic SMV displays, while accurately preserving occlusion.

Depth-Enhanced Holographic Super Multi-View Maxwellian Display Based on Variable Filter Aperture

The super multi-view (SMV) near-eye display (NED) effectively provides depth cues for three-dimensional (3D) displays by projecting multiple viewpoint images or parallax images onto the retina simultaneously. Previous SMV NED suffers from a limited depth of field (DOF) due to the fixed image plane. Aperture filtering is widely used to enhance the DOF; however, an invariably sized aperture may have opposite effects on objects with different reconstruction depths. In this paper, a holographic SMV display based on the variable filter aperture is proposed to enhance the DOF. In parallax image acquisition, multiple groups of parallax images, each group recording a part of the 3D scene on a fixed depth range, are captured first. In the hologram calculation, each group of wavefronts at the image recording plane (IRP) is calculated by multiplying the parallax images with the corresponding spherical wave phase. Then, they are propagated to the pupil plane and multiplied by the corresponding aperture filter function. The size of the filter aperture is variable which is determined by the depth of the object. Finally, the complex amplitudes at the pupil plane are back-propagated to the holographic plane and added together to form the DOF-enhanced hologram. Simulation and experimental results verify the proposed method could improve the DOF of holographic SMV display, which will contribute to the application of 3D NED.

Implantable metaverse with retinal prostheses and bionic vision processing

We present an implantable metaverse featuring retinal prostheses in association with bionic vision processing. Unlike conventional retinal prostheses, whose electrodes are spaced equidistantly, our solution is to rearrange the electrodes to match the distribution of ganglion cells. To naturally imitate the human vision, a scheme of bionic vision processing is developed. On top of a three-dimensional eye model, our bionic vision processing is able to visualize the monocular image, binocular image fusion, and parallax-induced depth map.

Lensless phase-only holographic retinal projection display based on the error diffusion algorithm

Holographic retinal projection display (RPD) can project images directly onto the retina without any lens by encoding a convergent spherical wave phase with the target images. Conventional amplitude-type holographic RPD suffers from strong zero-order light and conjugate. In this paper, a lensless phase-only holographic RPD based on error diffusion algorithm is demonstrated. It is found that direct error diffusion of the complex Fresnel hologram leads to low image quality. Thus, a post-addition phase method is proposed based on angular spectrum diffraction. The spherical wave phase is multiplied after error diffusion process, and acts as an imaging lens. In this way, the error diffusion functions better due to reduced phase difference between adjacent pixels, and a virtual image with improved quality is produced. The viewpoint is easily deflected just by changing the post-added spherical phase. A full-color holographic RPD with adjustable eyebox is demonstrated experimentally with time-multiplexing technique.

Near-eye display with a triple-channel waveguide for metaverse

We present a near-eye display featuring a triple-channel waveguide with chiral liquid crystal gratings. Our triple-channel waveguide is capable of dividing one field of view into three through both the polarization orthogonality and angular separation. To illustrate its principle, a k-space diagram, which takes into account the aspect ratio of field of view, is depicted. Our results demonstrate that its diagonal field of view reaches 90°, eye relief is 10 mm, exit pupil is 4.9 × 4.9 mm², transmittance is 4.9%, and uniformity is 89%.

Wide-Field-of-View Near-Eye Display with Dual-Channel Waveguide

We propose a wide-field-of-view near-eye display featuring a dual-channel waveguide with cholesteric liquid crystal gratings. Our dual-channel waveguide is capable of splitting the field of view through the orthogonal polarization division multiplexing. To explain its mechanism, a diagram of k-domain, which factors into both the waveguide size and the number of pupils, is depicted. Our results demonstrate that the diagonal field of view reaches up to 80°, eye relief is 10 mm, exit pupil is 4 × 3 mm2, and uniformity is 79%.

Lensless full-color holographic Maxwellian near-eye display with a horizontal eyebox expansion

A lensless full-color holographic Maxwellian near-eye display using a single amplitude-type spatial light modulator is proposed in this Letter. The color holographic image is directly projected onto the retina without any eyepiece. The color crosstalk is clearly separated from the signal in the space owing to the encoded spherical wave and carrier wave. An aperture numerical filter and a real polarized filter are used at the pupil plane to accurately stop the crosstalk light. A high-quality dynamic speckless color holographic image was produced in the mid-air within a specific depth range. The horizontal eyebox expansion is achieved simply through multiple spherical wave encoding and verified through an optical experiment. The proposed display is compact and promising as the augmented reality near-eye display.

Electrically Tuneable Optical Diffraction Gratings Based on a Polymer Scaffold Filled with a Nematic Liquid Crystal

We present an experimental and theoretical investigation of the optical diffractive properties of electrically tuneable optical transmission gratings assembled as stacks of periodic slices from a conventional nematic liquid crystal (E7) and a standard photoresist polymer (SU-8). The external electric field causes a twist-type reorientation of the LC molecules toward a perpendicular direction with respect to initial orientation. The associated field-induced modification of the director field is determined numerically and analytically by minimization of the Landau–de Gennes free energy. The optical diffraction properties of the associated periodically modulated structure are calculated numerically on the basis of rigorous coupled-wave analysis (RCWA). A comparison of experimental and theoretical results suggests that polymer slices provoke planar surface anchoring of the LC molecules with the inhomogeneous surface anchoring energy varying in the range 5–20 μJ/m². The investigated structures provide a versatile approach to fabricating LC-polymer-based electrically tuneable diffractive optical elements (DOEs).

Hybrid holographic Maxwellian near-eye display based on spherical wave and plane wave reconstruction for augmented reality display

The holographic Maxwellian display is a promising technique for augmented reality (AR) display because it solves the vergence-accommodation conflict while presenting a high-resolution display. However, conventional holographic Maxwellian display has the inherent trade-off between depth of field (DOF) and image quality. In this paper, two types of holographic Maxwellian displays, the spherical wave type and the plane wave type, are proposed and analyzed. The spherical wavefront and the plane wavefront are produced by a spatial light modulator (SLM) for Maxwellian display. Due to the focusing properties of different wavefronts, the two types of display have complementary DOF ranges. A hybrid approach combining the spherical wavefront and plane wavefront is proposed for a large DOF with high image quality. An optical experiment with AR display is demonstrated to verify the proposed method.

Eye-box extended retinal projection type near-eye display with multiple independent viewpoints [Invited]

We introduce an approach to expand the eye-box in a retinal-projection-based near-eye display. The retinal projection display has the advantage of providing clear images in a wide depth range; however, it has difficulty in practical use with a narrow eye-box. Here, we propose a method to enhance the eye-box of the retinal projection display by generating multiple independent viewpoints, maintaining a wide depth of field. The method prevents images projected from multiple viewpoints from overlapping one other in the retina. As a result, our proposed system can provide a continuous image over a wide viewing angle without an eye tracker or image update. We discuss the optical design for the proposed method and verify its feasibility through simulation and experiment.

Augmented reality display system using modulated moiré imaging technique

To enhance the depth rendering ability of augmented reality (AR) display systems, a modulated moiré imaging technique is used to render the true three-dimensional (3D) images for AR display systems. 3D images with continuous depth information and large depth of field are rendered and superimposed on the real scene. The proposed AR system consists of a modulated moiré imaging subsystem and an optical combiner. The modulated moiré imaging subsystem employs modulated point light sources, a display device, and a microlens array to generate 3D images. A defocussing equal period moiré imaging structure is used, which gives a chance for the point light sources to modulate the depth position of 3D images continuously. The principles of the imaging system are deduced analytically. A custom-designed transparent off-axis spherical reflective lens is used as an optical combiner to project the 3D images into the real world. An experimental AR system that provides continuous 3D images with depth information ranging from 0.5 to 2.5 m is made to verify the feasibility of the proposed technique.

First Evaluation of a Retinal Imaging Laser Eyewear System Based Low Vision Aid

Purpose

We conducted this trial to investigate a new wearable laser-eyewear (LEW). Images of an integrated camera are projected to the retina by a RGB-Laser (<1µW) and MEMS-mirror system. This enables a full-color live video as augmented reality embedded in the field of vision of the wearer. Thin parallel laser beams are projected following the principle of Maxwellian view through the center of the ocular lens to ensure independency of refractive errors. We performed a study with healthy subjects to test this independency.

Materials and Methods

LEW was tested in 20 healthy subjects (aged between 21 and 60 years) with hyperopia, myopia, astigmatism and/or presbyopia. Subjects underwent standardized visual acuity (VA) measurements (ETDRS) without any correction, with LEW and with best correction.

Results

We found no significant correlation between refraction and VA while using LEW in linear regression (r=0.17). Still, younger participants performed better in terms of VA with the device compared to older participants despite no differences in BCVA (p<0.01). The achievable VA with LEW was in general reduced compared to uncorrected VA (0.50 vs 0.40 logMAR). Only myopic subjects reached a significantly higher VA using LEW (p<0.001). Presbyopic subjects showed enhanced near VA (0.25 logMAR) by reading at 15cm with LEW without any further necessary refractive correction. Nearly all patients (80%) showed stereopsis without need for additional adjustments.

Conclusion

Our investigation could verify the independence of LEW of refractive errors. Therefore, the technology seems to be especially useful in patients with untreatable corneal conditions, e.g., after corneal burns, to achieve higher VA since the thin laser should penetrate even corneal opacities with less scattering.

Correcting the Proximity Effect in Nanophotonic Phased Arrays

Thermally modulated Nanophotonic Phased Arrays (NPAs) can be used as phase-only holographic displays. Compared to the holographic displays based on Liquid Crystal on Silicon Spatial Light Modulators (LCoS SLMs), NPAs have the advantage of integrated light source and high refresh rate. However, the formation of the desired wavefront requires accurate modulation of the phase which is distorted by the thermal proximity effect. This problem has been largely overlooked and existing approaches to similar problems are either slow or do not provide a good result in the setting of NPAs. We propose two new algorithms based on the iterative phase retrieval algorithm and the proximal algorithm to address this challenge. We have carried out computational simulations to compare and contrast various algorithms in terms of image quality and computational efficiency. This work is going to benefit the research on NPAs and enable the use of large-scale NPAs as holographic displays.

See-Through Near-Eye Display with Built-in Prescription and Two-Dimensional Exit Pupil Expansion

We propose a see-through near-eye display featuring an exit pupil expander (EPE), which is composed of two multiplexed slanted gratings. Via a two-dimensional expansion, the exit pupil (EP) is able to be enlarged up to 10 × 8 mm2. Besides, the prescription for correcting the refractive errors can be integrated as well. The design rules are set forth in detail, followed by the results and discussion regarding the efficiency, field of view (FOV), exit pupil, angular resolution (AR), modulation transfer function (MTF), contrast ratio (CR), distortion, and simulated imaging.

Design of foveated contact lens display for augmented reality

We present a design of a contact lens display, which features an array of collimated light-emitting diodes and a contact lens, for the augmented reality. By building the infrastructure directly on top of the eye, eye is allowed to move or rotate freely without the need of exit pupil expansion nor eye tracking. The resolution of light-emitting diodes is foveated to match with the density of cones on the retina. In this manner, the total number of pixels as well as the latency of image processing can be significantly reduced. Based on the simulation, the device performance is quantitatively analyzed. For the real image, modulation transfer functions is 0.669757 at 30 cycle/degree, contrast ratio is 5, and distortion is 10%. For the virtual image, the field of view is 82°, best angular resolution is 0.38', modulation transfer function is above 0.999999 at 30 cycle/degree, contrast ratio is 4988, and distortion is 6%.

Multifunction reflector controlled by liquid piston for optical switch and beam steering

This paper presents a multifunction reflector controlled by liquid piston for optical switch and beam steering. The multifunction reflector consists of two liquid cavities that are designed with microchannels. Two holes covered with elastic membranes are fabricated on the upper surface of the liquid cavities. When the liquid cavity is injected with liquid, the shape of the elastic membrane changes to form a liquid piston in the position of the holes accordingly. The magnetic base covered with a reflector is fixed on the surface. We can adjust the active number and height of the liquid pistons to drive the reflector deflecting to different directions. Our experiments show that the multifunction reflector can realize the function of 2×2 optical switch. It can also deflect the light beam through an angle of 0°∼72° in two directions. The multifunction reflector has potential applications of free-space optical communications, laser detections and variable optical attenuators.