Fourier-inspired neural module for real-time and high-fidelity computer-generated holography

Quality improvement of unfiltered holography by optimizing high diffraction orders with fill factor

Computer-generated holography (CGH) suffers from high diffraction orders (HDOs) due to the pixelated nature of spatial light modulators (SLMs), typically requiring bulky optical filtering systems. To address this issue, a novel unfiltered holography approach known as the high-order gradient descent (HOGD) algorithm was previously introduced to optimize HDOs without optical filtering, enabling compact holographic displays. However, this algorithm overlooks a crucial physical parameter of SLMs-the fill factor-leading to limited optical quality. Here, we introduce a fill factor-based HOGD (FF-HOGD) algorithm, specifically designed to improve the quality of unfiltered holography by incorporating the fill factor into the optimization process. The quality advantage of FF-HOGD is demonstrated through numerical simulations and optical experiments.

Holo-U2Net for High-Fidelity 3D Hologram Generation

Traditional methods of hologram generation, such as point-, polygon-, and layer-based physical simulation approaches, suffer from substantial computational overhead and generate low-fidelity holograms. Deep learning-based computer-generated holography demonstrates effective performance in terms of speed and hologram fidelity. There is potential to enhance the network's capacity for fitting and modeling in the context of computer-generated holography utilizing deep learning methods. Specifically, the ability of the proposed network to simulate Fresnel diffraction based on the provided hologram dataset requires further improvement to meet expectations for high-fidelity holograms. We propose a neural architecture called Holo-U2Net to address the challenge of generating a high-fidelity hologram within an acceptable time frame. Holo-U2Net shows notable performance in hologram evaluation metrics, including an average structural similarity of 0.9988, an average peak signal-to-noise ratio of 46.75 dB, an enhanced correlation coefficient of 0.9996, and a learned perceptual image patch similarity of 0.0008 on the MIT-CGH-4K large-scale hologram dataset.

Solution to the issue of high-order diffraction images for cylindrical computer-generated holograms

The cylindrical computer-generated hologram (CCGH), featuring a 360° viewing zone, has garnered widespread attention. However, the issue of high-order diffraction images due to pixelated structure in CCGH has not been previously reported and solved. For a cylindrical model offering a 360° viewing zone in the horizontal direction, the high-order diffraction images always overlap with the reconstruction image, leading to quality degradation. Furthermore, the 4f system is commonly used to eliminate high-order diffraction images in planar CGH, but its implementation is predictably complex for a cylindrical model. In this paper, we propose a solution to the issue of high-order diffraction images for CCGH. We derive the cylindrical diffraction formula from the outer hologram surface to the inner object surface in the spectral domain, and based on this, we subsequently analyze the effects brought by the pixel structure and propose the high-order diffraction model. Based on the proposed high-order diffraction model, we use the gradient descent method to optimize CCGH accounting for all diffraction orders simultaneously. Furthermore, we discuss the issue of circular convolution due to the periodicity of the Fast Fourier Transform (FFT) in cylindrical diffraction. The correctness of the proposed high-order diffraction model and the effectiveness of the proposed optimization method are demonstrated by numerical simulation. To our knowledge, this is the first time that the issue of high-order diffraction images in CCGH has been proposed, and we believe our solution can offer valuable guidance to practitioners in the field.

Real-time 4K computer-generated hologram based on encoding conventional neural network with learned layered phase

Learning-based computer-generated hologram (CGH) demonstrates great potential for real-time high-quality holographic displays. However, real-time 4K CGH generation for 3D scenes remains a challenge due to the computational burden. Here, a variant conventional neural network (CNN) is presented for CGH encoding with learned layered initial phases for layered CGH generation. Specifically, the CNN predicts the CGH based on the input complex amplitude on the CGH plane, and the learned initial phases act as a universal phase for any target images at the target depth layer. These phases are generated during the training process of the coding CNN to further optimize the quality. The CNN is trained to learn encoding 3D CGH by randomly selecting the depth layer in the training process, and contains only 938 parameters. The generation time for a 2D 4K CGH is 18 ms, and is increased by 12 ms for each layer in a layered 3D scene. The average Peak Signal to Noise Ratio (PSNR) of each layer is above 30dB in the depth range from 160 to 210 mm. Experiments verify that our method can achieve real-time layered 4K CGH generation.

DCPNet: a dual-channel parallel deep neural network for high quality computer-generated holography

Recent studies have demonstrated that a learning-based computer-generated hologram (CGH) has great potential for real-time, high-quality holographic displays. However, most existing algorithms treat the complex-valued wave field as a two-channel spatial domain image to facilitate mapping onto real-valued kernels, which does not fully consider the computational characteristics of complex amplitude. To address this issue, we proposed a dual-channel parallel neural network (DCPNet) for generating phase-only holograms (POHs), taking inspiration from the double phase amplitude encoding method. Instead of encoding the complex-valued wave field in the SLM plane as a two-channel image, we encode it into two real-valued phase elements. Then the two learned sub-POHs are sampled by the complementary 2D binary grating to synthesize the desired POH. Simulation and optical experiments are carried out to verify the feasibility and effectiveness of the proposed method. The simulation results indicate that the DCPNet is capable of generating high-fidelity 2k POHs in 36 ms. The optical experiments reveal that the DCPNet has excellent ability to preserve finer details, suppress speckle noise and improve uniformity in the reconstructed images.

Asymmetrical neural network for real-time and high-quality computer-generated holography

Computer-generated holography based on neural network holds great promise as a real-time hologram generation method. However, existing neural network-based approaches prioritize lightweight networks to achieve real-time display, which limits their capacity for network fitting. Here, we propose an asymmetrical neural network with a non-end-to-end structure that enhances fitting capacity and delivers superior real-time display quality. The non-end-to-end structure decomposes the overall task into two sub-tasks: phase prediction and hologram encoding. The asymmetrical design tailors each sub-network to its specific sub-task using distinct basic net-layers rather than relying on similar net-layers. This method allows for a sub-network with strong feature extraction and inference capabilities to match the phase predictor, while another sub-network with efficient coding capability matches the hologram encoder. By matching network functions to tasks, our method enhances the overall network's fitting capacity while maintaining a lightweight architecture. Both numerical reconstructions and optical experiments validate the reliability and effectiveness of our proposed method.

Diffraction model-driven neural network trained using hybrid domain loss for real-time and high-quality computer-generated holography

Learning-based computer-generated holography (CGH) has demonstrated great potential in enabling real-time, high-quality holographic displays. However, most existing learning-based algorithms still struggle to produce high-quality holograms, due to the difficulty of convolutional neural networks (CNNs) in learning cross-domain tasks. Here, we present a diffraction model-driven neural network (Res-Holo) using hybrid domain loss for phase-only hologram (POH) generation. Res-Holo utilizes the weights of the pretrained ResNet34 as the initialization during the encoder stage of the initial phase prediction network to extract more generic features and also to help prevent overfitting. Also, frequency domain loss is added to further constrain the information that the spatial domain loss is insensitive. The peak signal-to-noise ratio (PSNR) of the reconstructed image is improved by 6.05 dB using hybrid domain loss compared to using spatial domain loss alone. Simulation results show that the proposed Res-Holo can generate high-fidelity 2 K resolution POHs with an average PSNR of 32.88 dB at 0.014 seconds/frame on the DIV2K validation set. Both monochrome and full-color optical experiments show that the proposed method can effectively improve the quality of reproduced images and suppress image artifacts.

Deep learning empowered highly compressive SS-OCT via learnable spectral-spatial sub-sampling

With the rapid advances of light source technology, the A-line imaging rate of swept-source optical coherence tomography (SS-OCT) has experienced a great increase in the past three decades. The bandwidths of data acquisition, data transfer, and data storage, which can easily reach several hundred megabytes per second, have now been considered major bottlenecks for modern SS-OCT system design. To address these issues, various compression schemes have been previously proposed. However, most of the current methods focus on enhancing the capability of the reconstruction algorithm and can only provide a data compression ratio (DCR) up to 4 without impairing the image quality. In this Letter, we proposed a novel design paradigm, in which the sub-sampling pattern for interferogram acquisition is jointly optimized with the reconstruction algorithm in an end-to-end manner. To validate the idea, we retrospectively apply the proposed method on an ex vivo human coronary optical coherence tomography (OCT) dataset. The proposed method could reach a maximum DCR of ∼62.5 with peak signal-to-noise ratio (PSNR) of 24.2 dB, while a DCR of ∼27.78 could yield a visually pleasant image with a PSNR of ∼24.6 dB. We believe the proposed system could be a viable remedy for the ever-growing data issue in SS-OCT.