Three-dimensional deeply generated holography [Invited]

Non-convex optimization for inverse problem solving in computer-generated holography

Computer-generated holography is a promising technique that modulates user-defined wavefronts with digital holograms. Computing appropriate holograms with faithful reconstructions is not only a problem closely related to the fundamental basis of holography but also a long-standing challenge for researchers in general fields of optics. Finding the exact solution of a desired hologram to reconstruct an accurate target object constitutes an ill-posed inverse problem. The general practice of single-diffraction computation for synthesizing holograms can only provide an approximate answer, which is subject to limitations in numerical implementation. Various non-convex optimization algorithms are thus designed to seek an optimal solution by introducing different constraints, frameworks, and initializations. Herein, we overview the optimization algorithms applied to computer-generated holography, incorporating principles of hologram synthesis based on alternative projections and gradient descent methods. This is aimed to provide an underlying basis for optimized hologram generation, as well as insights into the cutting-edge developments of this rapidly evolving field for potential applications in virtual reality, augmented reality, head-up display, data encryption, laser fabrication, and metasurface design.

Computer-generated holography with ordinary display

We propose a method of computer-generated holography (CGH) using incoherent light emitted from a mobile phone screen. In this method, we suppose a cascade of holograms in which the first hologram is a color image displayed on the mobile phone screen. The hologram cascade is synthesized by solving an inverse problem with respect to the propagation of incoherent light. We demonstrate a three-dimensional color image reproduction using a two-layered hologram cascade composed of an iPhone and a spatial light modulator.

Deep hologram converter from low-precision to middle-precision holograms

We propose a deep hologram converter based on deep learning to convert low-precision holograms into middle-precision holograms. The low-precision holograms were calculated using a shorter bit width. It can increase the amount of data packing for single instruction/multiple data in the software approach and the number of calculation circuits in the hardware approach. One small and one large deep neural network (DNN) are investigated. The large DNN exhibited better image quality, whereas the smaller DNN exhibited a faster inference time. Although the study demonstrated the effectiveness of point-cloud hologram calculations, this scheme could be extended to various other hologram calculation algorithms.

From picture to 3D hologram: end-to-end learning of real-time 3D photorealistic hologram generation from 2D image input

In this Letter, we demonstrate a deep-learning-based method capable of synthesizing a photorealistic 3D hologram in real-time directly from the input of a single 2D image. We design a fully automatic pipeline to create large-scale datasets by converting any collection of real-life images into pairs of 2D images and corresponding 3D holograms and train our convolutional neural network (CNN) end-to-end in a supervised way. Our method is extremely computation-efficient and memory-efficient for 3D hologram generation merely from the knowledge of on-hand 2D image content. We experimentally demonstrate speckle-free and photorealistic holographic 3D displays from a variety of scene images, opening up a way of creating real-time 3D holography from everyday pictures.

Phase-only hologram generated by a convolutional neural network trained using low-frequency mixed noise

A phase-only hologram generated through the convolution neutral network (CNN) which is trained by the low-frequency mixed noise (LFMN) is proposed. Compared with CNN based computer-generated holograms, the proposed training dataset named LFMN includes different kinds of noise images after low-frequency processing. This dataset was used to replace the real images used in the conventional hologram to train CNN in a simple and flexible approach. The results revealed that the proposed method could generate a hologram of 2160 × 3840 pixels at a speed of 0.094 s/frame on the DIV2K valid dataset, and the average peak signal-to-noise ratio of the reconstruction was approximately 29.2 dB. The results of optical experiments validated the theoretical prediction. The reconstructed images obtained using the proposed method exhibited higher quality than those obtained using the conventional methods. Furthermore, the proposed method considerably mitigated artifacts of the reconstructed images.

End-to-end learning of 3D phase-only holograms for holographic display

Computer-generated holography (CGH) provides volumetric control of coherent wavefront and is fundamental to applications such as volumetric 3D displays, lithography, neural photostimulation, and optical/acoustic trapping. Recently, deep learning-based methods emerged as promising computational paradigms for CGH synthesis that overcome the quality-runtime tradeoff in conventional simulation/optimization-based methods. Yet, the quality of the predicted hologram is intrinsically bounded by the dataset's quality. Here we introduce a new hologram dataset, MIT-CGH-4K-V2, that uses a layered depth image as a data-efficient volumetric 3D input and a two-stage supervised+unsupervised training protocol for direct synthesis of high-quality 3D phase-only holograms. The proposed system also corrects vision aberration, allowing customization for end-users. We experimentally show photorealistic 3D holographic projections and discuss relevant spatial light modulator calibration procedures. Our method runs in real-time on a consumer GPU and 5 FPS on an iPhone 13 Pro, promising drastically enhanced performance for the applications above.

Incoherent computer-generated holography

We present a method for computer-generated holography (CGH) using spatially and temporally incoherent light. The proposed method synthesizes a hologram cascade by solving an inverse problem for the propagation of incoherent light. The spatial incoherence removes speckle noise in CGH, and the temporal incoherence simplifies the optical setup, including the light source. We demonstrate two- and three-dimensional color image reproductions by a two-layer grayscale hologram cascade with a chip-on-board white light-emitting diode.

Variable-intensity line 3D images drawn using kinoform-type electroholography superimposed with phase error

Three-dimensional (3D) display using electroholography is a promising technology for next-generation television systems; however, its applicability is limited by the heavy computational load for obtaining computer-generated holograms (CGHs). The CG-line method is an algorithm that calculates CGHs to display 3D line-drawn objects at a very high computational speed but with limited expressiveness; for instance, the intensity along the line must be constant. Herein, we propose an extension for drawing gradated 3D lines using the CG-line method by superimposing phase noise. Consequently, we succeeded in drawing gradated 3D lines while maintaining the high computational speed of the original CG-line method.

Deep-learning-based computer-generated hologram from a stereo image pair

We propose a deep-learning-based approach to producing computer-generated holograms (CGHs) of real-world scenes. We design an end-to-end convolutional neural network (the Stereo-to-Hologram Network, SHNet) framework that takes a stereo image pair as input and efficiently synthesizes a monochromatic 3D complex hologram as output. The network is able to rapidly and straightforwardly calculate CGHs from the directly recorded images of real-world scenes, eliminating the need for time-consuming intermediate depth recovery and diffraction-based computations. We demonstrate the 3D reconstructions with clear depth cues obtained from the SHNet-based CGHs by both numerical simulations and optical holographic virtual reality display experiments.

Compressive propagation with coherence

In this Letter, we present wave propagation models of spatially partially coherent (or spatially incoherent) light to compress the computational load of forward and back propagations in inverse problems. In our model, partially coherent light is approximated as a set of random or plane wavefronts passing through spatial bandpass filters, which corresponds to an illumination pupil, and each wave coherently propagates onto a sensor plane through object space. We show that our models reduce the number of coherent propagations in inverse problems, which are essential in optical control and sensing, such as computer-generated holography (CGH) and quantitative phase imaging. We verify the proposed models by numerical and experimental demonstrations of CGH incorporating spatially partially coherent light.

Real-Time Computation of 3D Wireframes in Computer-Generated Holography

Computer-Generated Holography (CGH) algorithms simulate numerical diffraction, being applied in particular for holographic display technology. Due to the wave-based nature of diffraction, CGH is highly computationally intensive, making it especially challenging for driving high-resolution displays in real-time. To this end, we propose a technique for efficiently calculating holograms of 3D line segments. We express the solutions analytically and devise an efficiently computable approximation suitable for massively parallel computing architectures. The algorithms are implemented on a GPU (with CUDA), and we obtain a 70-fold speedup over the reference point-wise algorithm with almost imperceptible quality loss. We report real-time frame rates for CGH of complex 3D line-drawn objects, and validate the algorithm in both a simulation environment as well as on a holographic display setup.

Roadmap on digital holography [Invited]

This Roadmap article on digital holography provides an overview of a vast array of research activities in the field of digital holography. The paper consists of a series of 25 sections from the prominent experts in digital holography presenting various aspects of the field on sensing, 3D imaging and displays, virtual and augmented reality, microscopy, cell identification, tomography, label-free live cell imaging, and other applications. Each section represents the vision of its author to describe the significant progress, potential impact, important developments, and challenging issues in the field of digital holography.