Snapshot ptychography on array cameras

Temporal structured illumination and vision-transformer enables large field-of-view binary snapshot ptychography

Ptychography, a widely used computational imaging method, generates images by processing coherent interference patterns scattered from an object of interest. In order to capture scenes with large field-of-view (FoV) and high spatial resolution simultaneously in a single shot, we propose a temporal-compressive structured-light Ptychography system. A novel three-step reconstruction algorithm composed of multi-frame spectra reconstruction, phase retrieval, and multi-frame image stitching is developed, where we employ the emerging Transformer-based network in the first step. Experimental results demonstrate that our system can expand the FoV by 20× without losing spatial resolution. Our results offer huge potential for enabling lensless imaging of molecules with large FoV as well as high spatial-temporal resolutions. We also notice that due to the loss of low-intensity information caused by the compressed sensing process, our method so far is only applicable to binary targets.

Unsupervised adaptive coded illumination Fourier ptychographic microscopy based on a physical neural network

Fourier Ptychographic Microscopy (FPM) is a computational technique that achieves a large space-bandwidth product imaging. It addresses the challenge of balancing a large field of view and high resolution by fusing information from multiple images taken with varying illumination angles. Nevertheless, conventional FPM framework always suffers from long acquisition time and a heavy computational burden. In this paper, we propose a novel physical neural network that generates an adaptive illumination mode by incorporating temporally-encoded illumination modes as a distinct layer, aiming to improve the acquisition and calculation efficiency. Both simulations and experiments have been conducted to validate the feasibility and effectiveness of the proposed method. It is worth mentioning that, unlike previous works that obtain the intensity of a multiplexed illumination by post-combination of each sequentially illuminated and obtained low-resolution images, our experimental data is captured directly by turning on multiple LEDs with a coded illumination pattern. Our method has exhibited state-of-the-art performance in terms of both detail fidelity and imaging velocity when assessed through a multitude of evaluative aspects.

Optical ptychography for biomedical imaging: recent progress and future directions [Invited]

Ptychography is an enabling microscopy technique for both fundamental and applied sciences. In the past decade, it has become an indispensable imaging tool in most X-ray synchrotrons and national laboratories worldwide. However, ptychography's limited resolution and throughput in the visible light regime have prevented its wide adoption in biomedical research. Recent developments in this technique have resolved these issues and offer turnkey solutions for high-throughput optical imaging with minimum hardware modifications. The demonstrated imaging throughput is now greater than that of a high-end whole slide scanner. In this review, we discuss the basic principle of ptychography and summarize the main milestones of its development. Different ptychographic implementations are categorized into four groups based on their lensless/lens-based configurations and coded-illumination/coded-detection operations. We also highlight the related biomedical applications, including digital pathology, drug screening, urinalysis, blood analysis, cytometric analysis, rare cell screening, cell culture monitoring, cell and tissue imaging in 2D and 3D, polarimetric analysis, among others. Ptychography for high-throughput optical imaging, currently in its early stages, will continue to improve in performance and expand in its applications. We conclude this review article by pointing out several directions for its future development.

Learning-based single-shot long-range synthetic aperture Fourier ptychographic imaging with a camera array

In this Letter, we report a new long-range synthetic aperture Fourier ptychographic imaging technique, termed learning-based single-shot synthetic aperture imaging (LSS-SAI). LSS-SAI uses a camera array to record low-resolution intensity images corresponding to different non-overlapping spectral regions in parallel, which are synthesized to reconstruct a super-resolved high-quality image based on a physical model-based dual-regression deep neural network. Compared with conventional macroscopic Fourier ptychographic imaging, LSS-SAI overcomes the stringent requirement on a large amount of raw data with a high spectral overlapping ratio for high-resolution, high signal-to-noise imaging of reflective objects with diffuse surfaces, making single-shot long-range synthetic aperture imaging possible. Experimental results on rough reflective samples show that our approach can improve the peak signal-to-noise ratio (PSNR) and structural similarity (SSIM) by 10.56 dB and 0.26, respectively. We also demonstrate the single-shot ptychography capability of the proposed approach by the synthetic aperture imaging of a dynamic scene at a camera-limited speed (30 fps). To the best of our knowledge, this is the first demonstration of macroscopic Fourier ptychography to single-shot synthetic aperture imaging of dynamic events.

Pose correction scheme for camera-scanning Fourier ptychography based on camera calibration and homography transform

Fourier ptychography (FP), as a computational imaging method, is a powerful tool to improve imaging resolution. Camera-scanning Fourier ptychography extends the application of FP from micro to macro creatively. Due to the non-ideal scanning of the camera driven by the mechanical translation stage, the pose error of the camera occurs, greatly degrading the reconstruction quality, while a precise translation stage is expensive and not suitable for wide-range imaging. Here, to improve the imaging performance of camera-scanning Fourier ptychography, we propose a pose correction scheme based on camera calibration and homography transform approaches. The scheme realizes the accurate alignment of data set and location error correction in the frequency domain. Simulation and experimental results demonstrate this method can optimize the reconstruction results and realize high-quality imaging effectively. Combined with the feature recognition algorithm, the scheme provides the possibility for applying FP in remote sensing imaging and space imaging.