Single shot holographic super-resolution microscopy

Single-pixel Fresnel incoherent correlation holography compressed imaging using a Trumpet network

Fresnel incoherent correlation holography (FINCH) can achieve high-precision and non-scanning 3D imaging. However, as a holographic imaging technology, the huge bandwidth requirements and the amount of holographic data transmitted have always been one of the important factors limiting its application. In addition, the hardware cost of pixel array-based CCD or CMOS imaging is very high under high resolution or specific wavelength conditions. Accordingly, a single-pixel Fresnel incoherent correlation holography (SP-FINCH) compressed imaging method is proposed, which replaces pixel array detector with single-pixel detector and designs a Trumpet network to achieve low-cost and high-resolution imaging. Firstly, a modified FINCH imaging system is constructed and data acquisition is carried out using a single-pixel detector. Secondly, a Trumpet network is constructed to directly map the relationship between one-dimensional sampled data and two-dimensional image in an end-to-end manner. Moreover, by comparing the reconstructed images using neural network with that using commonly used single-pixel reconstruction methods, the results indicate that the proposed SP-FINCH compressed imaging method can significantly improve the quality of image reconstruction at lower sampling rate and achieve imaging without phase-shifting operation. The proposed method has been shown to be feasible and advantageous through numerical simulations and optical experiment results.

Fresnel incoherent compressive holography toward 3D videography via dual-channel simultaneous phase-shifting interferometry

Fresnel incoherent correlation holography (FINCH) enables high-resolution 3D imaging of objects from several 2D holograms under incoherent light and has many attractive applications in motionless 3D fluorescence imaging. However, FINCH has difficulty implementing 3D imaging of dynamic scenes since multiple phase-shifting holograms need to be recorded for removing the bias term and twin image in the reconstructed scene, which requires the object to remain static during this progress. Here, we propose a dual-channel Fresnel noncoherent compressive holography method. First, a pair of holograms with π phase shifts obtained in a single shot are used for removing the bias term noise. Then, a physic-driven compressive sensing (CS) algorithm is used to achieve twin-image-free reconstruction. In addition, we analyze the reconstruction effect and suitability of the CS algorithm and two-step phase-shift filtering algorithm for objects with different complexities. The experimental results show that the proposed method can record hologram videos of 3D dynamic objects and scenes without sacrificing the imaging field of view or resolution. Moreover, the system refocuses images at arbitrary depth positions via computation, hence providing a new method for fast high-throughput incoherent 3D imaging.

General phase-difference imaging of incoherent digital holography

The hologram formed by incoherent holography based on self-interference should preserve the phase difference information of the object, such as the phase difference between the mutually orthogonal polarizations of anisotropic object. How to decode this phase difference from this incoherent hologram, i.e., phase-difference imaging, is of great significance for studying the properties of the measured object. However, there is no general phase-difference imaging theory due to both diverse incoherent holography systems and the complicated reconstruction process from holograms based on the diffraction theory. To realize phase-difference image in incoherent holography, the relationship between the phase difference of the object and the image reconstructed by holograms is derived using a general physical model of incoherent holographic systems, and then the additional phase that will distort this relationship in actual holographic systems is analyzed and eliminated. Finally, the phase-difference imaging that is suitable for the most incoherent holographic systems is realized and the general theory is experimentally verified. This technology can be applied to phase-difference imaging of anisotropic objects, and has potential applications in materials science, biomedicine, polarized optics and other fields.

Deep learning-based incoherent holographic camera enabling acquisition of real-world holograms for holographic streaming system

While recent research has shown that holographic displays can represent photorealistic 3D holograms in real time, the difficulty in acquiring high-quality real-world holograms has limited the realization of holographic streaming systems. Incoherent holographic cameras, which record holograms under daylight conditions, are suitable candidates for real-world acquisition, as they prevent the safety issues associated with the use of lasers; however, these cameras are hindered by severe noise due to the optical imperfections of such systems. In this work, we develop a deep learning-based incoherent holographic camera system that can deliver visually enhanced holograms in real time. A neural network filters the noise in the captured holograms, maintaining a complex-valued hologram format throughout the whole process. Enabled by the computational efficiency of the proposed filtering strategy, we demonstrate a holographic streaming system integrating a holographic camera and holographic display, with the aim of developing the ultimate holographic ecosystem of the future.

Single-shot Fresnel incoherent correlation holography via deep learning based phase-shifting technology

Fresnel incoherent correlation holography (FINCH) realizes non-scanning three-dimension (3D) images using spatial incoherent illumination, but it requires phase-shifting technology to remove the disturbance of the DC term and twin term that appears in the reconstruction field, thus increasing the complexity of the experiment and limits the real-time performance of FINCH. Here, we propose a single-shot Fresnel incoherent correlation holography via deep learning based phase-shifting (FINCH/DLPS) method to realize rapid and high-precision image reconstruction using only a collected interferogram. A phase-shifting network is designed to implement the phase-shifting operation of FINCH. The trained network can conveniently predict two interferograms with the phase shift of 2/3 π and 4/3 π from one input interferogram. Using the conventional three-step phase-shifting algorithm, we can conveniently remove the DC term and twin term of the FINCH reconstruction and obtain high-precision reconstruction through the back propagation algorithm. The Mixed National Institute of Standards and Technology (MNIST) dataset is used to verify the feasibility of the proposed method through experiments. In the test with the MNIST dataset, the reconstruction results demonstrate that in addition to high-precision reconstruction, the proposed FINCH/DLPS method also can effectively retain the 3D information by calibrating the back propagation distance in the case of reducing the complexity of the experiment, further indicating the feasibility and superiority of the proposed FINCH/DLPS method.

Volumetric imaging of fast cellular dynamics with deep learning enhanced bioluminescence microscopy

Bioluminescence microscopy is an appealing alternative to fluorescence microscopy, because it does not depend on external illumination, and consequently does neither produce spurious background autofluorescence, nor perturb intrinsically photosensitive processes in living cells and animals. The low photon emission of known luciferases, however, demands long exposure times that are prohibitive for imaging fast biological dynamics. To increase the versatility of bioluminescence microscopy, we present an improved low-light microscope in combination with deep learning methods to image extremely photon-starved samples enabling subsecond exposures for timelapse and volumetric imaging. We apply our method to image subcellular dynamics in mouse embryonic stem cells, epithelial morphology during zebrafish development, and DAF-16 FoxO transcription factor shuttling from the cytoplasm to the nucleus under external stress. Finally, we concatenate neural networks for denoising and light-field deconvolution to resolve intracellular calcium dynamics in three dimensions of freely moving Caenorhabditis elegans.

Grating-based in-line geometric-phase-shifting incoherent digital holographic system toward 3D videography

Incoherent digital holography (IDH) with a sequential phase-shifting method enables high-definition 3D imaging under incoherent lights. However, sequential recording of multiple holograms renders IDH impractical for 3D videography. In this study, we propose grating-based in-line geometric-phase-shifting IDH. Our method divides orthogonal circularly polarized lights into four copies with a fabricated phase grating and subsequently creates self-interference holograms with geometric phases introduced by a segmented linear polarizer. This enables single-shot recording of holograms without the need for a specially designed image sensor, such as a polarization-sensitive sensor. Moreover, the achievable spatial resolution is higher than that of off-axis methods. As a proof-of-principle experiment, we demonstrated snapshot and video recording of 3D reflective objects using our IDH method. The results confirmed the feasibility of the proposed method.

Historical development of FINCH from the beginning to single-shot 3D confocal imaging beyond optical resolution [Invited]

We chronicle a 15-year development effort of Fresnel incoherent correlation holography (FINCH) since its first description to its current 3D current microscopic wide-field or confocal imaging that doubles optical resolution beyond the Rayleigh limit to about 100 nm in a single snapshot. The path from the original demonstration of FINCH [Opt. Lett.32, 912 (2007) OPLEDP0146-959210.1364/OL.32.000912] to its current picture-perfect imaging of multicolor fluorescent biological specimens and reference test patterns by fluorescence or reflected light imaging is described.

Voxel characteristic estimation of integral imaging display system using self-interference incoherent digital holography

Three-dimensional (3D) images reconstructed by integral imaging display are captured as a complex hologram using self-interference incoherent digital holography (SIDH) and analyzed for the volumetric image characteristics. The integrated images can present 3D perception through not only binocular disparity but also volumetric property, which is represented in forming a volume picture element, called 'voxel', and an important criterion to distinguish the integral imaging from the multiview 3D display. Since SIDH can record the complex hologram under incoherent lighting conditions, the SIDH camera system has the advantage to measure the voxel formed with the incoherent light fields. In this paper, we propose a technique to estimate and analyze the voxel characteristics of the integral imaging system such as the depth location and resolution. The captured holograms of the integrated images are numerically reconstructed by depth for the voxel analysis. The depth location of the integrated image can be calculated and obtained using the autofocus algorithms and the focus metrics values, which also show the modalities of depth resolution. The estimation method of this paper can be applied to the accurate and quantitative analysis of the volumetric characteristics of light field 3D displays.

Fast Image Reconstruction Technique for Parallel Phase-Shifting Digital Holography

For incoherent and coherent digital holography, the parallel phase-shifting technique has been used to reduce the number of exposures required for the phase-shifting technique which eliminates zero-order diffraction and conjugates image components. Although the parallel phase-shifting technique can decrease the hologram recording time, the image interpolations require additional calculation time. In this study, we propose a technique that reduces the calculation time for image interpolations; this technique is based on the convolution theorem. We experimentally verified the proposed technique and compared it with the conventional technique. The proposed technique is more effective for more precise interpolation algorithms because the calculation time does not depend on the size of interpolation kernels.

Roadmap on Recent Progress in FINCH Technology

Fresnel incoherent correlation holography (FINCH) was a milestone in incoherent holography. In this roadmap, two pathways, namely the development of FINCH and applications of FINCH explored by many prominent research groups, are discussed. The current state-of-the-art FINCH technology, challenges, and future perspectives of FINCH technology as recognized by a diverse group of researchers contributing to different facets of research in FINCH have been presented.

Digital Incoherent Compressive Holography Using a Geometric Phase Metalens

We propose a compressive self-interference incoherent digital holography (SIDH) with a geometric phase metalens for section-wise holographic object reconstruction. We specify the details of the SIDH with a geometric phase metalens design that covers the visible wavelength band, analyze a spatial distortion problem in the SIDH and address a process of a compressive holographic section-wise reconstruction with analytic spatial calibration. The metalens allows us to realize a compressive SIDH system in the visible wavelength band using an image sensor with relatively low bandwidth. The operation of the proposed compressive SIDH is verified through numerical simulations.

Incoherent digital holography simulation based on scalar diffraction theory

Incoherent digital holography (IDH) enables passive 3D imaging through the self-interference of incoherent light. IDH imaging properties are dictated by the numerical aperture and optical layout in a complex manner [Opt. Express27, 33634 (2019)OPEXFF1094-408710.1364/OE.27.033634]. We develop an IDH simulation model to provide insight into its basic operation and imaging properties. The simulation is based on the scalar diffraction theory. Incoherent irradiance and self-interference holograms are numerically represented by the intensity-based summation of each propagation through finite aperture optics from independent point sources. By comparing numerical and experimental results, the applicability, accuracy, and limitation of the simulation are discussed. The developed simulation would be useful in optimizing the IDH setup.

Coherence aperture restricted spatial resolution for an arbitrary depth plane in incoherent digital holography

Incoherent digital holography (IDH) requires no spatial coherence; however, it requires high temporal coherence for a light source to capture holograms with high spatial resolution. Temporal coherence is often enhanced with a bandpass filter, reducing the light utilization efficiency. Thus, there is a trade-off between spatial resolution and light utilization efficiency. In this paper, we derive a relationship between spatial resolution and temporal coherence by including a conceptual aperture, determined by temporal coherence, in our previous theory of spatial resolution for arbitrary depth planes [Opt. Express27, 33634 (2019)OPEXFF1094-408710.1364/OE.27.033634]. Experimental evaluations verified the effectiveness of our theory, which is useful for the optimization of IDH setups and avoiding the trade-off.