Single-shot lensless imaging with fresnel zone aperture and incoherent illumination

Non-invasive estimation of the powder size distribution from a single speckle image

Non-invasive characterization of powders may take one of two approaches: imaging and counting individual particles; or relying on scattered light to estimate the particle size distribution (PSD) of the ensemble. The former approach runs into practical difficulties, as the system must conform to the working distance and other restrictions of the imaging optics. The latter approach requires an inverse map from the speckle autocorrelation to the particle sizes. The principle relies on the pupil function determining the basic sidelobe shape, whereas the particle size spread modulates the sidelobe intensity. We recently showed that it is feasible to invert the speckle autocorrelation and obtain the PSD using a neural network, trained efficiently through a physics-informed semi-generative approach. In this work, we eliminate one of the most time-consuming steps of our previous method by engineering the pupil function. By judiciously blocking portions of the pupil, we sacrifice some photons but in return we achieve much enhanced sidelobes and, hence, higher sensitivity to the change of the size distribution. The result is a 60 × reduction in total acquisition and processing time, or 0.25 seconds per frame in our implementation. Almost real-time operation in our system is not only more appealing toward rapid industrial adoption, it also paves the way for quantitative characterization of complex spatial or temporal dynamics in drying, blending, and other chemical and pharmaceutical manufacturing processes.

Address model mismatch and defocus in FZA lensless imaging via model-driven CycleGAN

Mask-based lensless imaging systems suffer from model mismatch and defocus. In this Letter, we propose a model-driven CycleGAN, MDGAN, to reconstruct objects within a long distance. MDGAN includes two translation cycles for objects and measurements respectively, each consisting of a forward propagation and a backward reconstruction module. The backward module resembles the Wiener-U-Net, and the forward module consists of the estimated image formation model of a Fresnel zone aperture camera (FZACam), followed by CNN to compensate for the model mismatch. By imposing cycle consistency, the backward module can adaptively match the actual depth-varying imaging process. We demonstrate that MDGAN based on either a simulated or calibrated imaging model produces a higher-quality image compared to existing methods. Thus, it can be applied to other mask-based systems.

Quantitative phase imaging based on holography: trends and new perspectives

In 1948, Dennis Gabor proposed the concept of holography, providing a pioneering solution to a quantitative description of the optical wavefront. After 75 years of development, holographic imaging has become a powerful tool for optical wavefront measurement and quantitative phase imaging. The emergence of this technology has given fresh energy to physics, biology, and materials science. Digital holography (DH) possesses the quantitative advantages of wide-field, non-contact, precise, and dynamic measurement capability for complex-waves. DH has unique capabilities for the propagation of optical fields by measuring light scattering with phase information. It offers quantitative visualization of the refractive index and thickness distribution of weak absorption samples, which plays a vital role in the pathophysiology of various diseases and the characterization of various materials. It provides a possibility to bridge the gap between the imaging and scattering disciplines. The propagation of wavefront is described by the complex amplitude. The complex-value in the complex-domain is reconstructed from the intensity-value measurement by camera in the real-domain. Here, we regard the process of holographic recording and reconstruction as a transformation between complex-domain and real-domain, and discuss the mathematics and physical principles of reconstruction. We review the DH in underlying principles, technical approaches, and the breadth of applications. We conclude with emerging challenges and opportunities based on combining holographic imaging with other methodologies that expand the scope and utility of holographic imaging even further. The multidisciplinary nature brings technology and application experts together in label-free cell biology, analytical chemistry, clinical sciences, wavefront sensing, and semiconductor production.

CFZA camera: a high-resolution lensless imaging technique based on compound Fresnel zone aperture

In lensless imaging using a Fresnel zone aperture (FZA), it is generally believed that the resolution is limited by the outermost ring breadth of the FZA. The limitation has the potential to be broken according to the multi-order property of binary FZAs. In this Letter, we propose to use a high-order component of the FZA as the point spread function (PSF) to develop a high-order transfer function backpropagation (HBP) algorithm to enhance the resolution. The proportion of high-order diffraction energy is low, leading to severe defocus noise in the reconstructed image. To address this issue, we propose a Compound FZA (CFZA), which merges two partial FZAs operating at different orders as the mask to strike a balance between the noise and resolution. Experimental results verify that the CFZA-based camera has a resolution that is double that of a traditional FZA-based camera with an identical outer ring breadth and can be reconstructed with high quality by a single HBP without calibration. Our method offers a cost-effective solution for achieving high-resolution imaging, expanding the potential applications of FZA-based lensless imaging in a variety of areas.

Enhanced imaging with binary circular Dammann Fresnel zone plate

We introduce a novel approach in optical engineering by combining Dammann gratings with binary Fresnel zone plates to create a unique hybrid optical element with enhanced energetic efficiency of its focal spots. Traditionally, binary Fresnel zone plates focus light at multiple points with varying intensities, while Dammann gratings are renowned for their efficient and uniform light splitting capabilities. Our innovation lies in merging these two elements and generating a binary circular Dammann (varying along the radial direction) Fresnel zone plate that concentrates most of the incident light into a small and desired number of focused points with equal intensities, rather than distributing light's energy non-equally across multiple points. This novel design significantly enhances the efficiency and precision of light manipulation. It opens new possibilities in applications requiring high-intensity focal points, such as in advanced medical imaging and in accurate scientific measurements. By redefining the conventional roles of these optical elements, our research contributes an advancement to the field, paving the way for innovative solutions in various optical applications.

Chromatic Aberration Correction in Harmonic Diffractive Lenses Based on Compressed Sensing Encoding Imaging

Large-aperture, lightweight, and high-resolution imaging are hallmarks of major optical systems. To eliminate aberrations, traditional systems are often bulky and complex, whereas the small volume and light weight of diffractive lenses position them as potential substitutes. However, their inherent diffraction mechanism leads to severe dispersion, which limits their application in wide spectral bands. Addressing the dispersion issue in diffractive lenses, we propose a chromatic aberration correction algorithm based on compressed sensing. Utilizing the diffractive lens's focusing ability at the reference wavelength and its degradation performance at other wavelengths, we employ compressed sensing to reconstruct images from incomplete image information. In this work, we design a harmonic diffractive lens with a diffractive order of M=150, an aperture of 40 mm, a focal length f0=320 mm, a reference wavelength λ0=550 nm, a wavelength range of 500-800 nm, and 7 annular zones. Through algorithmic recovery, we achieve clear imaging in the visible spectrum, with a peak signal-to-noise ratio (PSNR) of 22.85 dB, a correlation coefficient of 0.9596, and a root mean square error (RMSE) of 0.02, verifying the algorithm's effectiveness.

Extended depth of field for Fresnel zone aperture camera via fast passive depth estimation

The lensless camera with incoherent illumination has gained significant research interest for its thin and flexible structure. However, it faces challenges in resolving scenes with a wide depth of field (DoF) due to its depth-dependent point spread function (PSF). In this paper, we present a single-shot method for extending the DoF in Fresnel zone aperture (FZA) cameras at visible wavelengths through passive depth estimation. The improved ternary search method is utilized to determine the depth of targets rapidly by evaluating the sharpness of the back propagation reconstruction. Based on the depth estimation results, a set of reconstructed images focused on targets at varying depths are derived from the encoded image. After that, the DoF is extended through focus stacking. The experimental results demonstrate an 8-fold increase compared with the calibrated DoF at 130 mm depth. Moreover, our depth estimation method is five times faster than the traversal method, while maintaining the same level of accuracy. The proposed method facilitates the development of lensless imaging in practical applications such as photography, microscopy, and surveillance.

Fresnel incoherent compressive holography toward 3D videography via dual-channel simultaneous phase-shifting interferometry

Fresnel incoherent correlation holography (FINCH) enables high-resolution 3D imaging of objects from several 2D holograms under incoherent light and has many attractive applications in motionless 3D fluorescence imaging. However, FINCH has difficulty implementing 3D imaging of dynamic scenes since multiple phase-shifting holograms need to be recorded for removing the bias term and twin image in the reconstructed scene, which requires the object to remain static during this progress. Here, we propose a dual-channel Fresnel noncoherent compressive holography method. First, a pair of holograms with π phase shifts obtained in a single shot are used for removing the bias term noise. Then, a physic-driven compressive sensing (CS) algorithm is used to achieve twin-image-free reconstruction. In addition, we analyze the reconstruction effect and suitability of the CS algorithm and two-step phase-shift filtering algorithm for objects with different complexities. The experimental results show that the proposed method can record hologram videos of 3D dynamic objects and scenes without sacrificing the imaging field of view or resolution. Moreover, the system refocuses images at arbitrary depth positions via computation, hence providing a new method for fast high-throughput incoherent 3D imaging.

Physics-driven universal twin-image removal network for digital in-line holographic microscopy

Digital in-line holographic microscopy (DIHM) enables efficient and cost-effective computational quantitative phase imaging with a large field of view, making it valuable for studying cell motility, migration, and bio-microfluidics. However, the quality of DIHM reconstructions is compromised by twin-image noise, posing a significant challenge. Conventional methods for mitigating this noise involve complex hardware setups or time-consuming algorithms with often limited effectiveness. In this work, we propose UTIRnet, a deep learning solution for fast, robust, and universally applicable twin-image suppression, trained exclusively on numerically generated datasets. The availability of open-source UTIRnet codes facilitates its implementation in various DIHM systems without the need for extensive experimental training data. Notably, our network ensures the consistency of reconstruction results with input holograms, imparting a physics-based foundation and enhancing reliability compared to conventional deep learning approaches. Experimental verification was conducted among others on live neural glial cell culture migration sensing, which is crucial for neurodegenerative disease research.

An Improved 3D OPC Method for the Fabrication of High-Fidelity Micro Fresnel Lenses

Based on three-dimensional optical proximity correction (3D OPC), recent advancements in 3D lithography have enabled the high-fidelity customization of 3D micro-optical elements. However, the micron-to-millimeter-scale structures represented by the Fresnel lens design bring more stringent requirements for 3D OPC, which poses significant challenges to the accuracy of models and the efficiency of algorithms. Thus, a lithographic model based on optical imaging and photochemical reaction curves is developed in this paper, and a subdomain division method with a statistics principle is proposed to improve the efficiency and accuracy of 3D OPC. Both the simulation and the experimental results show the superiority of the proposed 3D OPC method in the fabrication of Fresnel lenses. The computation memory requirements of the 3D OPC are reduced to below 1%, and the profile error of the fabricated Fresnel lens is reduced 79.98%. Applying the Fresnel lenses to an imaging system, the average peak signal to noise ratio (PSNR) of the image is increased by 18.92%, and the average contrast of the image is enhanced by 36%. We believe that the proposed 3D OPC method can be extended to the fabrication of vision-correcting ophthalmological lenses.

MWDNs: reconstruction in multi-scale feature spaces for lensless imaging

Lensless cameras, consisting of only a sensor and a mask, are small and flexible enough to be used in many applications with stringent scale constraints. These mask-based imagers encode scenes in caustic patterns. Most existing reconstruction algorithms rely on multiple iterations based on physical model for deconvolution followed by deep learning for perception, among which the main limitation of reconstruction quality is the mismatch between the ideal and the real model. To solve the problem, we in this work learned a class of multi Wiener deconvolution networks (MWDNs), deconvoluting in multi-scale feature spaces with Wiener filters to reduce the information loss and improving the accuracy of the given model by correcting the inputs. A comparison between the proposed and the state-of-the-art algorithms shows that ours achieves much better images and performs well in real-world environments. In addition, our method takes greater advantage of the computational time due to the abandonment of iterations.

Chiral metasurface zone plate for transmission-reflection focusing of circularly polarized terahertz waves

The properties of traditional Fresnel zone plates have been greatly enhanced by metasurfaces, which allow the control of polarization, orbital angular momentum, or other parameters on the basis of focusing. In this Letter, a new, to the best of our knowledge, method for circularly polarized wave manipulation based on a zone plate is proposed. Chiral meta-atoms and binary geometric phase are used for the simultaneous focusing of reflected and transmitted terahertz waves. The silicon-based dielectric chiral units, which show great performance of spin-selective transmission near 0.54 THz, separate the orthogonal circularly polarized components. A binary Pancharatnam-Berry (P-B) phase gradient is obtained by rotating the unit 90 degrees, then the phase zone plate can be easily designed. The simulation results show that the proposed chiral metasurface zone plate has the function of reflection-transmission separation and focusing for the circularly polarized terahertz waves. In addition, we also demonstrate the possibility of using a 1064-nm continuous infrared laser to adjust the intensity of our devices, based on photo-generated carriers in silicon. The design principle of the chiral metasurface zone plates can be extended to other wavelengths, providing new ideas for the regulation of circularly polarized light.

Lensless computationally defined confocal incoherent imaging with a Fresnel zone plane as coded aperture

We present a Fresnel zone plate (FZP) mask-based system for single-shot lensless confocal imaging. The system uses an FZP as coded aperture, which allows each point source to cast a unique pattern onto the sensor, representing their horizontal and axial positions. This results in a 2D sensor measurement comprising a series of FZP patterns, which records the spatial intensity distribution of the incoherent illuminant. The reconstruction process is facilitated by an algorithm based on compress sensing (CS) theory and the use of the nuclear norm of gradient scanning and hologram segmentation technology for autofocusing. The simulative and experimental results of this study align well with the expectation that every layered scene can be accurately recovered at the corresponding depth, without undesirable signals from other layers. Additionally, we analyze the deviation of the reconstruction results in the experiment, which emphasizes the need to consider the thickness of the FZP for a precise forward propagation model.

Amp-vortex edge-camera: a lensless multi-modality imaging system with edge enhancement

We demonstrate a lensless imaging system with edge-enhanced imaging constructed with a Fresnel zone aperture (FZA) mask placed 3 mm away from a CMOS sensor. We propose vortex back-propagation (vortex-BP) and amplitude vortex-BP algorithms for the FZA-based lensless imaging system to remove the noise and achieve the fast reconstruction of high contrast edge enhancement. Directionally controlled anisotropic edge enhancement can be achieved with our proposed superimposed vortex-BP algorithm. With different reconstruction algorithms, the proposed amp-vortex edge-camera in this paper can achieve 2D bright filed imaging, isotropic, and directional controllable anisotropic edge-enhanced imaging with incoherent light illumination, by a single-shot captured hologram. The effect of edge detection is the same as optical edge detection, which is the re-distribution of light energy. Noise-free in-focus edge detection can be achieved by using back-propagation, without a de-noise algorithm, which is an advantage over other lensless imaging technologies. This is expected to be widely used in autonomous driving, artificial intelligence recognition in consumer electronics, etc.

Deep Learning Empowered Fresnel-based Lensless Fluorescence Microscopy. ANNUAL INTERNATIONAL CONFERENCE OF THE IEEE ENGINEERING IN MEDICINE AND BIOLOGY SOCIETY. IEEE ENGINEERING IN MEDICINE AND BIOLOGY SOCIETY. ANNUAL INTERNATIONAL CONFERENCE 2023;2023:1-4. [PMID: 38082985 DOI: 10.1109/embc40787.2023.10339990] [Citation(s) in RCA: 0] [Impact Index Per Article: 0]

Miniaturized fluorescence microscopy has revolutionized the way neuroscientists study the brain in-vivo. Recent developments in computational lensless imaging promise a next generation of miniaturized microscopes in lensless fluorescence microscopy. We developed a microscope prototype using an optimized Fresnel amplitude mask. While many lensless imaging modalities have reported excellent performance using Deep Learning (DL) approaches, DL application in fluorescence imaging has been left untouched. We generated a computational dataset based on experimental system calibration to evaluate DL capabilities on biological cell morphologies. We show that our DL-assisted microscope can provide high-quality imaging with a structural similarity index of 89%. The least absolute error was decreased by 63% using the DL-assisted method compared with the classical models. The state-of-the-art performance of this prototype enhances the expected potential of amplitude masks in lensless microscopy applications, which are critical for robust in-vivo flat microscopy with engineered image sensors.Clinical Relevance- This study aids in advancing miniaturized fluorescence microscopy, which greatly impacts long-term brain circuit and disease studies in freely moving animal models.

Light People: Professor Liangcai Cao

EDITORIAL

Holography utilizes the principles of wave interference and diffraction to record and reconstruct images, which can highly restore the three-dimensional features of objects and provide an immersive visual experience. Dennis Gabor proposed the concept of holography in 1947 and was awarded the Nobel Prize in Physics in 1971. Holography has gradually developed into two major research directions: digital holography (DH) and computer-generated holography (CGH). Holography has empowered the development of fields such as 6G communication, intelligent healthcare, and commercial MR headsets. In recent years, the general solution to optical inverse problems contained in holography also provides theoretical support for its wide integration with computational lithography, optical metamaterials, optical neural networks, orbital angular momentum (OAM), and other areas. This demonstrates its enormous potential for research and application. We are delighted to invite Professor Liangcai Cao from Tsinghua University, a leading scientist in the field of holography, to give us a profound interpretation of the opportunities and challenges of holography. In the interview, Prof. Cao will take us on a journey through the history of holography, share fascinating stories from his academic visits and exchanges, and shed light on the mentor and tutor culture in teaching. Through this episode of "Light People," we will have the privilege of getting to know Prof. Cao on a deeper level.

Multi-phase FZA lensless imaging via diffusion model

Lensless imaging shifts the burden of imaging from bulky and expensive hardware to computing, which enables new architectures for portable cameras. However, the twin image effect caused by the missing phase information in the light wave is a key factor limiting the quality of lensless imaging. Conventional single-phase encoding methods and independent reconstruction of separate channels pose challenges in removing twin images and preserving the color fidelity of the reconstructed image. In order to achieve high-quality lensless imaging, the multiphase lensless imaging via diffusion model (MLDM) is proposed. A multi-phase FZA encoder integrated on a single mask plate is used to expand the data channel of a single-shot image. The information association between the color image pixel channel and the encoded phase channel is established by extracting prior information of the data distribution based on multi-channel encoding. Finally, the reconstruction quality is improved through the use of the iterative reconstruction method. The results show that the proposed MLDM method effectively removes the influence of twin images and produces high-quality reconstructed images compared with traditional methods, and the results reconstructed using MLDM have higher structural similarity and peak signal-to-noise ratio.

Toward a see-through camera via AR lightguide

As the foundation of virtual content generation, cameras are crucial for augmented reality (AR) applications, yet their integration with transparent displays has remained a challenge. Prior efforts to develop see-through cameras have struggled to achieve high resolution and seamless integration with AR displays. In this work, we present LightguideCam, a compact and flexible see-through camera based on an AR lightguide. To address the overlapping artifacts in measurement, we present a compressive sensing algorithm based on an equivalent imaging model that minimizes computational consumption and calibration complexity. We validate our design using a commercial AR lightguide and demonstrate a field of view of 23.1° and an angular resolution of 0.1° in the prototype. Our LightguideCam has great potential as a plug-and-play extensional imaging component in AR head-mounted displays, with promising applications for eye-gaze tracking, eye-position perspective photography, and improved human-computer interaction devices, such as full-screen mobile phones.

Autofocusing of Fresnel zone aperture lensless imaging for QR code recognition

Fresnel zone aperture (FZA) lensless imaging encodes the incident light into a hologram-like pattern, so that the scene image can be numerically focused at a long imaging range by the back propagation method. However, the target distance is uncertain. The inaccurate distance causes blurs and artifacts in the reconstructed images. This brings difficulties for the target recognition applications, such as quick response code scanning. We propose an autofocusing method for FZA lensless imaging. By incorporating the image sharpness metrics into the back propagation reconstruction process, the method can acquire the desired focusing distance and reconstruct noise-free high-contrast images. By combining the Tamura of the gradient metrics and nuclear norm of gradient, the relative error of estimated object distance is only 0.95% in the experiment. The proposed reconstruction method significantly improves the mean recognition rate of QR code from 4.06% to 90.00%. It paves the way for designing intelligent integrated sensors.

Wave-optics-based image synthesis for super resolution reconstruction of a FZA lensless camera

A Fresnel Zone Aperture (FZA) mask for a lensless camera, an ultra-thin and functional computational imaging system, is beneficial because the FZA pattern makes it easy to model the imaging process and reconstruct captured images through a simple and fast deconvolution. However, diffraction causes a mismatch between the forward model used in the reconstruction and the actual imaging process, which affects the recovered image's resolution. This work theoretically analyzes the wave-optics imaging model of an FZA lensless camera and focuses on the zero points caused by diffraction in the frequency response. We propose a novel idea of image synthesis to compensate for the zero points through two different realizations based on the linear least-mean-square-error (LMSE) estimation. Results from computer simulation and optical experiments verify a nearly two-fold improvement in spatial resolution from the proposed methods compared with the conventional geometrical-optics-based method.

Monocular metasurface camera for passive single-shot 4D imaging

It is a grand challenge for an imaging system to simultaneously obtain multi-dimensional light field information, such as depth and polarization, of a scene for the accurate perception of the physical world. However, such a task would conventionally require bulky optical components, time-domain multiplexing, and active laser illumination. Here, we experimentally demonstrate a compact monocular camera equipped with a single-layer metalens that can capture a 4D image, including 2D all-in-focus intensity, depth, and polarization of a target scene in a single shot under ambient illumination conditions. The metalens is optimized to have a conjugate pair of polarization-decoupled rotating single-helix point-spread functions that are strongly dependent on the depth of the target object. Combined with a straightforward, physically interpretable image retrieval algorithm, the camera can simultaneously perform high-accuracy depth sensing and high-fidelity polarization imaging over an extended depth of field for both static and dynamic scenes in both indoor and outdoor environments. Such a compact multi-dimensional imaging system could enable new applications in diverse areas ranging from machine vision to microscopy.

3D single shot lensless incoherent optical imaging using coded phase aperture system with point response of scattered airy beams

Interferenceless coded aperture correlation holography (I-COACH) techniques have revolutionized the field of incoherent imaging, offering multidimensional imaging capabilities with a high temporal resolution in a simple optical configuration and at a low cost. The I-COACH method uses phase modulators (PMs) between the object and the image sensor, which encode the 3D location information of a point into a unique spatial intensity distribution. The system usually requires a one-time calibration procedure in which the point spread functions (PSFs) at different depths and/or wavelengths are recorded. When an object is recorded under identical conditions as the PSF, the multidimensional image of the object is reconstructed by processing the object intensity with the PSFs. In the previous versions of I-COACH, the PM mapped every object point to a scattered intensity distribution or random dot array pattern. The scattered intensity distribution results in a low SNR compared to a direct imaging system due to optical power dilution. Due to the limited focal depth, the dot pattern reduces the imaging resolution beyond the depth of focus if further multiplexing of phase masks is not performed. In this study, I-COACH has been realized using a PM that maps every object point into a sparse random array of Airy beams. Airy beams during propagation exhibit a relatively high focal depth with sharp intensity maxima that shift laterally following a curved path in 3D space. Therefore, sparse, randomly distributed diverse Airy beams exhibit random shifts with respect to one another during propagation, generating unique intensity distributions at different distances while retaining optical power concentrations in small areas on the detector. The phase-only mask displayed on the modulator was designed by random phase multiplexing of Airy beam generators. The simulation and experimental results obtained for the proposed method are significantly better in SNR than in the previous versions of I-COACH.

Unrolled primal-dual networks for lensless cameras

Conventional models for lensless imaging assume that each measurement results from convolving a given scene with a single experimentally measured point-spread function. These models fail to simulate lensless cameras truthfully, as these models do not account for optical aberrations or scenes with depth variations. Our work shows that learning a supervised primal-dual reconstruction method results in image quality matching state of the art in the literature without demanding a large network capacity. We show that embedding learnable forward and adjoint models improves the reconstruction quality of lensless images (+5dB PSNR) compared to works that assume a fixed point-spread function.

Diffractive lensless imaging with optimized Voronoi-Fresnel phase

Lensless cameras are a class of imaging devices that shrink the physical dimensions to the very close vicinity of the image sensor by replacing conventional compound lenses with integrated flat optics and computational algorithms. Here we report a diffractive lensless camera with spatially-coded Voronoi-Fresnel phase to achieve superior image quality. We propose a design principle of maximizing the acquired information in optics to facilitate the computational reconstruction. By introducing an easy-to-optimize Fourier domain metric, Modulation Transfer Function volume (MTFv), which is related to the Strehl ratio, we devise an optimization framework to guide the optimization of the diffractive optical element. The resulting Voronoi-Fresnel phase features an irregular array of quasi-Centroidal Voronoi cells containing a base first-order Fresnel phase function. We demonstrate and verify the imaging performance for photography applications with a prototype Voronoi-Fresnel lensless camera on a 1.6-megapixel image sensor in various illumination conditions. Results show that the proposed design outperforms existing lensless cameras, and could benefit the development of compact imaging systems that work in extreme physical conditions.

Accurate PSF determination in x ray image restoration

When a source's focal spot is known, an x ray image can be significantly improved by a deconvolution algorithm with the point spread function (PSF). We propose a simple method to measure the PSF for image restoration using x ray speckle imaging. In this method, the PSF is reconstructed with intensity and total variation constraints from a single x ray speckle of an ordinary diffuser. Compared with the traditional time-consuming measurement with a pinhole camera, the speckle imaging method is fast and easy to execute. When the PSF is available, we use a deconvolution algorithm to reconstruct the sample's radiography image, and the result provides more structural details than the original images.

Learned lensless 3D camera

Single-shot three-dimensional (3D) imaging with compact device footprint, high imaging quality, and fast processing speed is challenging in computational imaging. Mask-based lensless imagers, which replace the bulky optics with customized thin optical masks, are portable and lightweight, and can recover 3D object from a snap-shot image. Existing lensless imaging typically requires extensive calibration of its point spread function and heavy computational resources to reconstruct the object. Here we overcome these challenges and demonstrate a compact and learnable lensless 3D camera for real-time photorealistic imaging. We custom designed and fabricated the optical phase mask with an optimized spatial frequency support and axial resolving ability. We developed a simple and robust physics-aware deep learning model with adversarial learning module for real-time depth-resolved photorealistic reconstructions. Our lensless imager does not require calibrating the point spread function and has the capability to resolve depth and "see-through" opaque obstacles to image features being blocked, enabling broad applications in computational imaging.

Shortwave infrared single-pixel spectral imaging based on a GSST phase-change metasurface

Shortwave infrared (SWIR) spectral imaging obtains spectral fingerprints corresponding to overtones of molecular vibrations invisible to conventional silicon-based imagers. However, SWIR imaging is challenged by the excessive cost of detectors. Single-pixel imaging based on compressive sensing can alleviate the problem but meanwhile presents new difficulties in spectral modulations, which are prerequisite in compressive sampling. In this work, we theoretically propose a SWIR single-pixel spectral imaging system with spectral modulations based on a Ge₂Sb₂Se₄Te₁ (GSST) phase-change metasurface. The transmittance spectra of the phase-change metasurface are tuned through wavelength shifts of multipole resonances by varying crystallinities of GSST, validated by the multipole decompositions and electromagnetic field distributions. The spectral modulations constituted by the transmittance spectra corresponding to the 11 phases of GSST are sufficient for the compressive sampling on the spectral domain of SWIR hyperspectral images, indicated by the reconstruction in false color and point spectra. Moreover, the feasibility of optimization on phase-change metasurface via coherence minimization is demonstrated through the designing of the GSST pillar height. The concept of spectral modulation with phase-change metasurface overcomes the static limitation in conventional modulators, whose integratable and reconfigurable features may pave the way for high-efficient, low-cost, and miniaturized computational imaging based on nanophotonics.

Tunable liquid crystal grating based holographic 3D display system with wide viewing angle and large size

As one of the most ideal display approaches, holographic 3-dimensional (3D) display has always been a research hotspot since the holographic images reproduced in such system are very similar to what humans see the actual environment. However, current holographic 3D displays suffer from critical bottlenecks of narrow viewing angle and small size. Here, we propose a tunable liquid crystal grating-based holographic 3D display system with wide viewing angle and large size. Our tunable liquid crystal grating, providing an adjustable period and the secondary diffraction of the reconstructed image, enables to simultaneously implement two different hologram generation methods in achieving wide viewing angle and enlarged size, respectively. By using the secondary diffraction mechanism of the tunable liquid crystal grating, the proposed system breaks through the limitations of narrow viewing angle and small size of holographic 3D display. The proposed system shows a viewing angle of 57.4°, which is nearly 7 times of the conventional case with a single spatial light modulator, and the size of the reconstructed image is enlarged by about 4.2. The proposed system will have wide applications in medical diagnosis, advertising, education and entertainment and other fields.

Multidimension-multiplexed full-phase-encoding holography

I propose a multidimension-multiplexed imaging method with which multiple physical quantities of light are simultaneously obtained as interference fringe images. The varieties of light are distinguished by exploiting the proposed phase-encoding technique. Neither measurements of point spread functions in advance, nor iterative calculations to derive multidimensional information, nor a laser light source is required. Multidimensional imaging of an object and simultaneous three-dimensional image recording of self-luminous light and light transmitted from an object are experimentally demonstrated. A palm-sized interferometer based on the proposed holography is developed for the experiments to show its portability and physical-filter-free multidimensional imaging ability without an antivibration structure.

Explicit-restriction convolutional framework for lensless imaging

Mask-based lensless cameras break the constraints of traditional lens-based cameras, introducing highly flexible imaging systems. However, the inherent restrictions of imaging devices lead to low reconstruction quality. To overcome this challenge, we propose an explicit-restriction convolutional framework for lensless imaging, whose forward model effectively incorporates multiple restrictions by introducing the linear and noise-like nonlinear terms. As examples, numerical and experimental reconstructions based on the limitation of sensor size, pixel pitch, and bit depth are analyzed. By tailoring our framework for specific factors, better perceptual image quality or reconstructions with 4× pixel density can be achieved. This proposed framework can be extended to lensless imaging systems with different masks or structures.

High-performance imaging with an advanced non-imaging lens based on full-path optical diffraction calculation in two-dimensional space

High-performance image-forming systems often require high system complexity due to the overdetermined nature of optical aberration correction. What we present here is a novel computational imaging modality which can achieve high-performance imaging using a simple non-image-forming optical system. The presented optical system contains an aspherical non-imaging lens which is designed with the optimal transfer of light radiation between an object and a detector. All spatial frequencies of the object collected by the non-imaging lens are delivered to the detector. No image is formed on the detector, and a full-path optical diffraction calculation method is developed to recover a high-quality image of the object from multiple intensity measurements. The effectiveness and high performance of the proposed imaging modality is verified by the examples.

Recent Advances in Lensless Imaging

Lensless imaging provides opportunities to design imaging systems free from the constraints imposed by traditional camera architectures. Thanks to advances in imaging hardware, fabrication techniques, and new algorithms, researchers have recently developed lensless imaging systems that are extremely compact, lightweight or able to image higher-dimensional quantities. Here we review these recent advances and describe the design principles and their effects that one should consider when developing and using lensless imaging systems.

Chromatic confocal measurement method using a phase Fresnel zone plate

A chromatic confocal measurement method based on a phase Fresnel zone plate (FZP) is described. Strong dispersion of FZP results in significant axial focal shift. The axial dispersion curve is close to linear within a certain wavelength range determined by the quantitative calculation using the vectorial angular spectrum theory. A 11.27 mm diameter phase FZP with a primary focal length of 50 mm was processed using standard photolithography technology and used as the dispersive objective in a homemade chromatic confocal measurement system. The calibrated axial measurement range exceeds 16 mm, the axial resolution reaches 0.8 µm, and the measurement accuracy of displacement is better than 0.4%. This chromatic confocal sensor has been practically used in the measurement of step height, glass thickness, and 3D surface profile. The proposed method has the obvious characteristics of simplicity, greatly reduced cost and superior performance. It is believed that this sensing method has broad application prospects in glass, coating, machinery, electronics, optics and other industries.

Single-path single-shot phase-shifting digital holographic microscopy without a laser light source

We propose single-path single-shot phase-shifting digital holographic microscopy (SSP-DHM) in which the quantitative phase information of an object wave is acquired without a laser light source. Multiple phase-shifted holograms are simultaneously obtained using a linear polarizer, a liquid crystal on a silicon spatial light modulator (LCoS-SLM), and a polarization-imaging camera. Complex amplitude imaging of a USAF1951 test target and phase imaging of transparent HeLa cells are performed to show its quantitative phase-imaging ability. We also conduct an experiment for the motion-picture imaging of transparent particles to highlight the single-shot imaging ability of SSP-DHM.

Efficient compressive holographic reconstruction based on hologram segmentation

Compressive holography can successfully reconstruct a three-dimensional layered object from a two-dimensional hologram. However, the extremely time-consuming reconstruction limits its range of applications. We propose a dimension reduction of measurement matrix (DRMM) method to accelerate compressive holographic reconstruction. The calculation time is substantially reduced while the reconstruction quality is improved by DRMM, which is implemented by a hologram segmentation approach and a parallel computing technique. Holograms of specific target objects are segmented from the hologram of a three-dimensional layered object, and the reconstruction can be implemented in parallel using multicore processors. We present both simulation and experimental results to show the feasibility and effectiveness of the proposed method.

Hyperuniform scalar random fields for lensless, multispectral imaging systems

We propose a novel framework for the systematic design of lensless imaging systems based on the hyperuniform random field solutions of nonlinear reaction-diffusion equations from pattern formation theory. Specifically, we introduce a new class of imaging point-spread functions (PSFs) with enhanced isotropic behavior and controllable sparsity. We investigate PSFs and modulated transfer functions for a number of nonlinear models and demonstrate that two-phase isotropic random fields with hyperuniform disorder are ideally suited to construct imaging PSFs with improved performances compared to PSFs based on Perlin noise. Additionally, we introduce a phase retrieval algorithm based on non-paraxial Rayleigh-Sommerfeld diffraction theory and introduce diffractive phase plates with PSFs designed from hyperuniform random fields, called hyperuniform phase plates (HPPs). Finally, using high-fidelity object reconstruction, we demonstrate improved image quality using engineered HPPs across the visible range. The proposed framework is suitable for high-performance lensless imaging systems for on-chip microscopy and spectroscopy applications.

Investigation of the effective aperture: towards high-resolution Fresnel incoherent correlation holography

Fresnel incoherent correlation holography (FINCH) shows great advantages of coherent-light-source-free, high lateral resolution, no scanning, and easy integration, and has exhibited great potential in recording three-dimensional information of objects. Despite the rapid advances in the resolution of the FINCH system, little attention has been paid to the influence of the effective aperture of the system. Here, the effective aperture of the point spread function (PSF) has been investigated both theoretically and experimentally. It is found that the effective aperture is mainly restricted by the aperture of the charge-coupled device (CCD), the pixel size of the CCD, and the actual aperture of the PSF at different recording distances. It is also found that the optimal spatial resolution exists only for a small range of recording distance, while this range would become smaller as the imaging wavelength gets longer, leading to the result that the optimal spatial resolution is solely determined by the actual aperture of the PSF. By further combining the FINCH system with a microscopy system and optimizing the recording distance, a spatial resolution as high as 0.78 μm at the wavelength of 633 nm has been obtained, enabling a much higher quality imaging of unstained living biological cells compared to the commercial optical microscope. The results of this work may provide some helpful insights into the design of high-resolution FINCH systems and pave the way for their application in biomedical imaging.

Incoherent digital holography simulation based on scalar diffraction theory

Incoherent digital holography (IDH) enables passive 3D imaging through the self-interference of incoherent light. IDH imaging properties are dictated by the numerical aperture and optical layout in a complex manner [Opt. Express27, 33634 (2019)OPEXFF1094-408710.1364/OE.27.033634]. We develop an IDH simulation model to provide insight into its basic operation and imaging properties. The simulation is based on the scalar diffraction theory. Incoherent irradiance and self-interference holograms are numerically represented by the intensity-based summation of each propagation through finite aperture optics from independent point sources. By comparing numerical and experimental results, the applicability, accuracy, and limitation of the simulation are discussed. The developed simulation would be useful in optimizing the IDH setup.

Holographic optical field recovery using a regularized untrained deep decoder network

Image reconstruction using minimal measured information has been a long-standing open problem in many computational imaging approaches, in particular in-line holography. Many solutions are devised based on compressive sensing (CS) techniques with handcrafted image priors or supervised deep neural networks (DNN). However, the limited performance of CS methods due to lack of information about the image priors and the requirement of an enormous amount of per-sample-type training resources for DNNs has posed new challenges over the primary problem. In this study, we propose a single-shot lensless in-line holographic reconstruction method using an untrained deep neural network which is incorporated with a physical image formation algorithm. We demonstrate that by modifying a deep decoder network with simple regularizers, a Gabor hologram can be inversely reconstructed via a minimization process that is constrained by a deep image prior. The outcoming model allows to accurately recover the phase and amplitude images without any training dataset, excess measurements, or specific assumptions about the object's or the measurement's characteristics.

Influence of sparse constraint functions on compressive holographic tomography

In this paper, we quantified and analyzed the impact of the l₁ norm and total variation (TV) norm sparse constraints on the reconstruction quality under different interlayer spacings, sampling rates, and signal-to-noise ratios. For high-quality holograms, the results of compressive-sensing reconstruction using l₁ norm achieved higher quality than those by the TV norm. In contrast, for low-quality holograms, the quality of TV-norm-based reconstruction results was relatively stable and better than that of l₁ norm. In addition, we explained why interlayer spacing cannot be smaller and recommend the use of axial resolution of the digital holography system as the interlayer spacing. The conclusions are valuable in the choice of sparse constraints in compressive holographic tomography.

DNN-FZA camera: a deep learning approach toward broadband FZA lensless imaging

In mask-based lensless imaging, iterative reconstruction methods based on the geometric optics model produce artifacts and are computationally expensive. We present a prototype of a lensless camera that uses a deep neural network (DNN) to realize rapid reconstruction for Fresnel zone aperture (FZA) imaging. A deep back-projection network (DBPN) is connected behind a U-Net providing an error feedback mechanism, which realizes the self-correction of features to recover the image detail. A diffraction model generates the training data under conditions of broadband incoherent imaging. In the reconstructed results, blur caused by diffraction is shown to have been ameliorated, while the computing time is 2 orders of magnitude faster than the traditional iterative image reconstruction algorithms. This strategy could drastically reduce the design and assembly costs of cameras, paving the way for integration of portable sensors and systems.

Superresolved image reconstruction in FZA lensless camera by color-channel synthesis

The Fresnel-zone-aperture lensless camera using a fringe-scanning technique allows non-iterative well-conditioned image reconstruction; however, the spatial resolution is limited by the mathematical reconstruction model that ignores diffraction. To solve this resolution problem, we propose a novel image-reconstruction algorithm using the wave-optics-based design of the deconvolution filter and color-channel image synthesis. We verify a two-fold improvement of the effective angular resolution by conducting numerical simulations and optical experiments with a prototype.

Three-dimensional phase optical transfer function in axially symmetric microscopic quantitative phase imaging

Three-dimensional quantitative phase imaging (3D QPI) is widely recognized as a potentially high-impact microscopic modality. Central to determining the resolution capability of 3D QPI is the phase optical transfer function (POTF). The magnitude of the POTF over its spatial frequency coverage (SFC) specifies the intensity of the response for each allowed spatial frequency. In this paper, a detailed analysis of the POTF for an axially symmetric optical configuration is presented. First, a useful geometric interpretation of the SFC, which enables its visualization, is presented. Second, a closed-form 1D integral expression is derived for the POTF in the general nonparaxial case, which enables rapid calculation of the POTF. Third, this formulation is applied to disk, annular, multi-annuli, and Gaussian illuminations as well as to an annular objective. Taken together, these contributions enable the visualization and simplified calculation of the 3D axially symmetric POTF and provide a basis for optimizing QPI in a wide range of applications.

AI on a chip

Artificial intelligence (AI) has dramatically changed the landscape of science, industry, defence, and medicine in the last several years. Supported by considerably enhanced computational power and cloud storage, the field of AI has shifted from mostly theoretical studies in the discipline of computer science to diverse real-life applications such as drug design, material discovery, speech recognition, self-driving cars, advertising, finance, medical imaging, and astronomical observation, where AI-produced outcomes have been proven to be comparable or even superior to the performance of human experts. In these applications, what is essentially important for the development of AI is the data needed for machine learning. Despite its prominent importance, the very first process of the AI development, namely data collection and data preparation, is typically the most laborious task and is often a limiting factor of constructing functional AI algorithms. Lab-on-a-chip technology, in particular microfluidics, is a powerful platform for both the construction and implementation of AI in a large-scale, cost-effective, high-throughput, automated, and multiplexed manner, thereby overcoming the above bottleneck. On this platform, high-throughput imaging is a critical tool as it can generate high-content information (e.g., size, shape, structure, composition, interaction) of objects on a large scale. High-throughput imaging can also be paired with sorting and DNA/RNA sequencing to conduct a massive survey of phenotype-genotype relations whose data is too complex to analyze with traditional computational tools, but is analyzable with the power of AI. In addition to its function as a data provider, lab-on-a-chip technology can also be employed to implement the developed AI for accurate identification, characterization, classification, and prediction of objects in mixed, heterogeneous, or unknown samples. In this review article, motivated by the excellent synergy between AI and lab-on-a-chip technology, we outline fundamental elements, recent advances, future challenges, and emerging opportunities of AI with lab-on-a-chip technology or "AI on a chip" for short.

Phase extraction neural network (PhENN) with coherent modulation imaging (CMI) for phase retrieval at low photon counts

Imaging with low-dose light is of importance in various fields, especially when minimizing radiation-induced damage onto samples is desirable. The raw image captured at the detector plane is then predominantly a Poisson random process with Gaussian noise added due to the quantum nature of photo-electric conversion. Under such noisy conditions, highly ill-posed problems such as phase retrieval from raw intensity measurements become prone to strong artifacts in the reconstructions; a situation that deep neural networks (DNNs) have already been shown to be useful at improving. Here, we demonstrate that random phase modulation on the optical field, also known as coherent modulation imaging (CMI), in conjunction with the phase extraction neural network (PhENN) and a Gerchberg-Saxton-Fienup (GSF) approximant, further improves resilience to noise of the phase-from-intensity imaging problem. We offer design guidelines for implementing the CMI hardware with the proposed computational reconstruction scheme and quantify reconstruction improvement as function of photon count.