Aperture scanning Fourier ptychographic microscopy

Depth-of-field extended Fourier ptychographic microscopy without defocus distance priori

Fourier ptychographic microscopy (FPM) provides a solution of high-throughput phase imaging. Thanks to its coherent imaging model, FPM has the capacity of depth-of-field (DOF) extension by simultaneously recovering the sample's transmittance function and pupil aberration, which contains a defocus term. However, existing phase retrieval algorithms (PRs) often struggle in the presence of a significant defocus. In this Letter, different PRs with embedded pupil recovery are compared, and the one based on the alternating direction multiplier method (ADMM-FPM) demonstrates promising potential for reconstructing highly defocused FPM images. Besides, we present a plug-and-play framework that integrates ADMM-FPM and total variation or Hessian denoiser for pupil function enhancement. Both simulations and experiments demonstrate that this framework enables robust reconstruction of defocused FPM images without any prior knowledge of defocus distance or sample characteristics. In experiments involving USAF 1951 targets and pathologic slides, ADMM-FPM combined with the Hessian denoiser successfully corrected the defocus up to approximately 200 µm, i.e., extending the DOF to 400 µm.

Dual-wavelength Fourier ptychographic microscopy for topographic measurement

Topographic measurements of micro- or nanostructures are essential in cutting-edge scientific disciplines such as optical communications, metrology, and structural biology. Despite the advances in surface metrology, measuring micron-scale steps with wide field of view (FOV) and high-resolution remains difficult. This study demonstrates a dual-wavelength Fourier ptychographic microscopy for high-resolution topographic measurement across a wide FOV using an aperture scanning structure. This structure enables the capture of a three-dimensional (3D) sample's scattered field with two different wavelength lasers, thus allowing the axial measurement range growing from nano- to micro-scale with enhanced lateral resolution. To suppress the unavoidable noises and artifacts caused by temporal coherence, system vibration, etc., a total variation (TV) regularization algorithm is introduced for phase retrieval. A blazed grating with micron-scale steps is used as the sample to validate the performance of our method. The agreement between the high-resolution reconstructed topography with our method and that with atomic force microscopy verified the effectiveness. Meanwhile, numerical simulations suggest that the method has the potential to characterize samples with high aspect-ratio steps.

Multi-wavelength interference phase imaging for automatic breast cancer detection and delineation using diffuse reflection imaging

Millions of women globally are impacted by the major health problem of breast cancer (BC). Early detection of BC is critical for successful treatment and improved survival rates. In this study, we provide a progressive approach for BC detection using multi-wavelength interference (MWI) phase imaging based on diffuse reflection hyperspectral (HS) imaging. The proposed findings are based on the measurement of the interference pattern between the blue (446.6 nm) and red (632 nm) wavelengths. We consider implementing a comprehensive image processing and categorization method based on the use of Fast Fourier (FF) transform analysis pertaining to a change in the refractive index between tumor and normal tissue. We observed that cancer growth affects tissue organization dramatically, as seen by persistently increased refractive index variance in tumors compared normal areas. Both malignant and normal tissue had different depth data collected from it that was analyzed. To enhance the categorization of ex-vivo BC tissue, we developed and validated a training classifier algorithm specifically designed for categorizing HS cube data. Following the application of signal normalization with the FF transform algorithm, our methodology achieved a high level of performance with a specificity (Spec) of 94% and a sensitivity (Sen) of 90.9% for the 632 nm acquired image categorization, based on preliminary findings from breast specimens under investigation. Notably, we successfully leveraged unstained tissue samples to create 3D phase-resolved images that effectively highlight the distinctions in diffuse reflectance features between cancerous and healthy tissue. Preliminary data revealed that our imaging method might be able to assist specialists in safely excising malignant areas and assessing the tumor bed following resection automatically at different depths. This preliminary investigation might result in an effective "in-vivo" disease description utilizing optical technology using a typical RGB camera with wavelength-specific operation with our quantitative phase MWI imaging methodology.

Non-interferometric stand-alone single-shot holographic camera using reciprocal diffractive imaging

An ideal holographic camera measures the amplitude and phase of the light field so that the focus can be numerically adjusted after the acquisition, and depth information about an imaged object can be deduced. The performance of holographic cameras based on reference-assisted holography is significantly limited owing to their vulnerability to vibration and complex optical configurations. Non-interferometric holographic cameras can resolve these issues. However, existing methods require constraints on an object or measurement of multiple-intensity images. In this paper, we present a holographic image sensor that reconstructs the complex amplitude of scattered light from a single-intensity image using reciprocal diffractive imaging. We experimentally demonstrate holographic imaging of three-dimensional diffusive objects and suggest its potential applications by imaging a variety of samples under both static and dynamic conditions.

High-precision Fourier ptychographic microscopy based on Gaussian apodization coherent transfer function constraints

Fourier ptychographic microscopy (FPM) combines the concepts of phase retrieval algorithms and synthetic apertures and can solve the problem in which it is difficult to combine a large field of view with high resolution. However, the use of the coherent transfer function in conventional calculations to describe the linear transfer process of an imaging system can lead to ringing artifacts. In addition, the Gerchberg-Saxton iterative algorithm can cause the phase retrieval part of the FPM algorithm to fall into a local optimum. In this paper, Gaussian apodization coherent transfer function is proposed to describe the imaging process and is combined with an iterative method based on amplitude weighting and phase gradient descent to reduce the presence of ringing artifacts while ensuring the accuracy of the reconstructed results. In simulated experiments, the proposed algorithm is shown to give a smaller mean square error and higher structural similarity, both in the presence and absence of noise. Finally, the proposed algorithm is validated in terms of giving reconstruction results with high accuracy and high resolution, using images acquired with a new microscope system and open-source images.

Optical ptychography for biomedical imaging: recent progress and future directions [Invited]

Ptychography is an enabling microscopy technique for both fundamental and applied sciences. In the past decade, it has become an indispensable imaging tool in most X-ray synchrotrons and national laboratories worldwide. However, ptychography's limited resolution and throughput in the visible light regime have prevented its wide adoption in biomedical research. Recent developments in this technique have resolved these issues and offer turnkey solutions for high-throughput optical imaging with minimum hardware modifications. The demonstrated imaging throughput is now greater than that of a high-end whole slide scanner. In this review, we discuss the basic principle of ptychography and summarize the main milestones of its development. Different ptychographic implementations are categorized into four groups based on their lensless/lens-based configurations and coded-illumination/coded-detection operations. We also highlight the related biomedical applications, including digital pathology, drug screening, urinalysis, blood analysis, cytometric analysis, rare cell screening, cell culture monitoring, cell and tissue imaging in 2D and 3D, polarimetric analysis, among others. Ptychography for high-throughput optical imaging, currently in its early stages, will continue to improve in performance and expand in its applications. We conclude this review article by pointing out several directions for its future development.

Fourier ptychographic microscopy with untrained deep neural network priors

We propose a physics-assisted deep neural network scheme in Fourier ptychographic microscopy (FPM) using untrained deep neural network priors (FPMUP) to achieve a high-resolution image reconstruction from multiple low-resolution images. Unlike the traditional training type of deep neural network that requires a large labelled dataset, this proposed scheme does not require training and instead outputs the high-resolution image by optimizing the parameters of neural networks to fit the experimentally measured low-resolution images. Besides the amplitude and phase of the sample function, another two parallel neural networks that generate the general pupil function and illumination intensity factors are incorporated into the carefully designed neural networks, which effectively improves the image quality and robustness when both the aberration and illumination intensity fluctuation are present in FPM. Reconstructions using simulated and experimental datasets are demonstrated, showing that the FPMUP scheme has better image quality than the traditional iterative algorithms, especially for the phase recovery, but at the expense of increasing computational cost. Most importantly, it is found that the FPMUP scheme can predict the Fourier spectrum of the sample outside synthetic aperture of FPM and thus eliminate the ringing effect of the recovered images due to the spectral truncation. Inspired by deep image prior in the field of image processing, we may impute the expansion of Fourier spectrums to the deep prior rooted in the architecture of the careful designed four parallel deep neural networks. We envisage that the resolution of FPM will be further enhanced if the Fourier spectrum of the sample outside the synthetic aperture of FPM is accurately predicted.

Low-voltage driving high-resistance liquid crystal micro-lens with electrically tunable depth of field for the light field imaging system

Light field imaging (LFI) based on Liquid crystal microlens array (LC MLAs) are emerging as a significant area for 3D imaging technology in the field of upcoming Internet of things and artificial intelligence era. However, in scenes of LFI through conventional MLAs, such as biological imaging and medicine imaging, the quality of imaging reconstruction will be severely reduced due to the limited depth of field. Here, we are proposed a low-voltage driving LC MLAs with electrically tunable depth of field (DOF) for the LFI system. An aluminum-doped zinc oxide (AZO) film was deposited on the top of the hole-patterned driven-electrode arrays and used as a high resistance (Hi-R) layer, a uniform gradient electric field was obtained across the sandwiched LC cell. Experimental results confirm that the proposed LC MLAs possess high-quality interference rings and tunable focal length at a lower working voltage. In addition, the focal lengths are tunable from 3.93 to 2.62 mm and the DOF are adjustable from 15.60 to 1.23 mm. The experiments demonstrated that the LFI system based on the proposed structure can clearly capture 3D information of the insets with enlarged depths by changing the working voltage and driving frequency, which indicates that the tunable DOF LC MLAs have a potential application prospects for the biological and medical imaging.

High-speed multi-objective Fourier ptychographic microscopy

The ability of a microscope to rapidly acquire wide-field, high-resolution images is limited by both the optical performance of the microscope objective and the bandwidth of the detector. The use of multiple detectors can increase electronic-acquisition bandwidth, but the use of multiple parallel objectives is problematic since phase coherence is required across the multiple apertures. We report a new synthetic-aperture microscopy technique based on Fourier ptychography, where both the illumination and image-space numerical apertures are synthesized, using a spherical array of low-power microscope objectives that focus images onto mutually incoherent detectors. Phase coherence across apertures is achieved by capturing diffracted fields during angular illumination and using ptychographic reconstruction to synthesize wide-field, high-resolution, amplitude and phase images. Compared to conventional Fourier ptychography, the use of multiple objectives reduces image acquisition times by increasing the area for sampling the diffracted field. We demonstrate the proposed scaleable architecture with a nine-objective microscope that generates an 89-megapixel, 1.1 µm resolution image nine-times faster than can be achieved with a single-objective Fourier-ptychographic microscope. New calibration procedures and reconstruction algorithms enable the use of low-cost 3D-printed components for longitudinal biological sample imaging. Our technique offers a route to high-speed, gigapixel microscopy, for example, imaging the dynamics of large numbers of cells at scales ranging from sub-micron to centimetre, with an enhanced possibility to capture rare phenomena.

Integration of Fourier ptychography with machine learning: an alternative scheme

As the core task of the reconstruction in conventional ptychography (CP) and Fourier ptychographic microscopy (FPM), the meticulous design of ptychographical iterative engine (PIE) largely affects the performance of reconstruction algorithms. Compared to traditional PIE algorithms, the paradigm of combining with machine learning to cross a local optimum has recently achieved significant progress. Nevertheless, existing designed engines still suffer drawbacks such as excessive hyper-parameters, heavy tuning work and lack of compatibility, which greatly limit their practical applications. In this work, we present a complete set of alternative schemes comprised of a kind of new perspective, a uniform design template, and a fusion framework, to naturally integrate Fourier ptychography (FP) with machine learning concepts. The new perspective, Dynamic Physics, is taken as the preferred tool to analyze a path (algorithm) at the physical level; the uniform design template, T-FP, clarifies the physical significance and optimization part in a path; the fusion framework follows two workable guidelines that are specially designed to keep convergence and make later localized modification for a new path, and further establishes a link between FP iterations and the gradient update in machine learning. Our scheme is compatible with both traditional FP paths and machine learning concepts. By combining ideas in both fields, we offer two design examples, MaFP and AdamFP. Results for both simulations and experiments show that designed algorithms following our scheme obtain better, faster (converge at the early stage after a few iterations) and more stable recovery with only minimal tuning hyper-parameters, demonstrating the effectiveness and superiority of our scheme.

Experimental Study on the Exploration of Camera Scanning Reflective Fourier Ptychography Technology for Far-Field Imaging

Fourier ptychography imaging is a powerful phase retrieval method that can be used to realize super-resolution. In this study, we establish a mathematical model of long-distance camera scanning based on reflective Fourier ptychography imaging. In order to guarantee the effective recovery of a high-resolution image in the experiment, we analyze the influence of laser coherence in different modes and the surface properties of diverse materials for diffused targets. For the analysis, we choose a single-mode fiber laser as the illumination source and metal materials with high diffused reflectivity as the experimental targets to ensure the validity of the experimental results. Based on the above, we emulate camera scanning with a single camera attached to an X-Y translation stage, and an experimental system with a working distance of 3310 mm is used as an example to image a fifty-cent coin. We also perform speckle analysis for rough targets and calculate the average speckle size using a normalized autocorrelation function in different positions. The method of calculating the average speckle size for everyday objects provides the premise for subsequent research on image quality evaluation; meanwhile, the coherence of the light field and the targets with high reflectivity under this experiment provide an application direction for the further development of the technique, such as computer vision, surveillance and remote sensing.

Analysis, Simulations, and Experiments for Far-Field Fourier Ptychography Imaging Using Active Coherent Synthetic-Aperture

Fourier ptychography (FP) is a powerful phase retrieval method that can be used to reconstruct missing high-frequency details and high-space-bandwidth products in microscopy. In this study, we further advanced the application of FP in microscopic imaging to the field of macroscopic far-field imaging, incorporating camera scanning for spatial resolution improvement. First, on the basis of the Fraunhofer diffraction mechanism and the transmission imaging model, we found the analysis of the associated theoretical fundamentals via simulations and experiments to be crucially relevant to the far-field of FP imaging. Second, we built an experimental device with long-distance imaging and experimentally demonstrated the relationship between the spectrum overlap ratio and the reconstructed high-resolution image. The simulation and experimental results showed that an overlap ratio higher than 50% had a good reconstruction effect. Third, camera scanning was used to obtain low-resolution intensity images in this study, for which the scanning range was wide and spherical wave illumination was satisfied, and therefore different positions corresponded to different aberrations of low-resolution intensity images, and even different positions of the same image had aberration differences, leading to inconsistencies in the aberrations of different images. Therefore, in the reconstruction process, we further overcame the effect of the inconsistency of aberrations of different images using the partition reconstruction method, which involves cutting the image into smaller parts for reconstruction. Finally, with the proposed partition reconstruction algorithm, we were able to resolve 40 μm line width of GBA1 resolution object and obtain a spatial resolution gain of 4× with a working distance of 2 m.

Snapshot ptychography on array cameras

We use convolutional neural networks to recover images optically down-sampled by 6.7 × using coherent aperture synthesis over a 16 camera array. Where conventional ptychography relies on scanning and oversampling, here we apply decompressive neural estimation to recover full resolution image from a single snapshot, although as shown in simulation multiple snapshots can be used to improve signal-to-noise ratio (SNR). In place training on experimental measurements eliminates the need to directly calibrate the measurement system. We also present simulations of diverse array camera sampling strategies to explore how snapshot compressive systems might be optimized.

High-resolution and large field-of-view Fourier ptychographic microscopy and its applications in biomedicine

Fourier ptychographic microscopy (FPM) is a promising and fast-growing computational imaging technique with high resolution, wide field-of-view (FOV) and quantitative phase recovery, which effectively tackles the problems of phase loss, aberration-introduced artifacts, narrow depth-of-field and the trade-off between resolution and FOV in conventional microscopy simultaneously. In this review, we provide a comprehensive roadmap of microscopy, the fundamental principles, advantages, and drawbacks of existing imaging techniques, and the significant roles that FPM plays in the development of science. Since FPM is an optimization problem in nature, we discuss the framework and related work. We also reveal the connection of Euler's formula between FPM and structured illumination microscopy. We review recent advances in FPM, including the implementation of high-precision quantitative phase imaging, high-throughput imaging, high-speed imaging, three-dimensional imaging, mixed-state decoupling, and introduce the prosperous biomedical applications. We conclude by discussing the challenging problems and future applications. FPM can be extended to a kind of framework to tackle the phase loss and system limits in the imaging system. This insight can be used easily in speckle imaging, incoherent imaging for retina imaging, large-FOV fluorescence imaging, etc.

Fourier ptychography: current applications and future promises

Traditional imaging systems exhibit a well-known trade-off between the resolution and the field of view of their captured images. Typical cameras and microscopes can either "zoom in" and image at high-resolution, or they can "zoom out" to see a larger area at lower resolution, but can rarely achieve both effects simultaneously. In this review, we present details about a relatively new procedure termed Fourier ptychography (FP), which addresses the above trade-off to produce gigapixel-scale images without requiring any moving parts. To accomplish this, FP captures multiple low-resolution, large field-of-view images and computationally combines them in the Fourier domain into a high-resolution, large field-of-view result. Here, we present details about the various implementations of FP and highlight its demonstrated advantages to date, such as aberration recovery, phase imaging, and 3D tomographic reconstruction, to name a few. After providing some basics about FP, we list important details for successful experimental implementation, discuss its relationship with other computational imaging techniques, and point to the latest advances in the field while highlighting persisting challenges.

Single-shot optical multiple-image encryption by jointly using wavelength multiplexing and position multiplexing

A single shot large-capacity optical multiple-image encryption method based on wavelength multiplexing and position multiplexing is proposed. In the encryption process of the proposed method, multiple plane waves of different wavelengths are adopted to illuminate secret images that are placed at different positions along the optical axis. All the secret images are encoded into a single grayscale intensity-only image that is recorded by a monochromic camera by applying a diffractive-imaging based double random phase encoding (DRPE) system. In the decryption process, high accuracy images are decrypted without crosstalk from the intensity image through a multimode phase retrieval algorithm and a two-step iterative shrinkage/thresholding (TwIST) algorithm. The feasibility of the proposed method is demonstrated by numerical simulations.

Computational aberration correction of VIS-NIR multispectral imaging microscopy based on Fourier ptychography

Due to the chromatic dispersion properties inherent in all optical materials, even the best-designed multispectral objective will exhibit residual chromatic aberration. Here, we demonstrate a multispectral microscope with a computational scheme based on the Fourier ptychographic microscopy (FPM) to correct these effects in order to render undistorted, in-focus images. The microscope consists of 4 spectral channels ranging from 405 nm to 1552 nm. After the computational aberration correction, it can achieve isotropic resolution enhancement as verified with the Siemens star sample. We image a flip-chip to show the promise of our system to conduct fault detection on silicon chips. This computational approach provides a cost-efficient strategy for high quality multispectral imaging over a broad spectral range.

Phase-space deconvolution for light field microscopy

Light field microscopy, featuring with snapshot large-scale three-dimensional (3D) fluorescence imaging, has aroused great interests in various biological applications, especially for high-speed 3D calcium imaging. Traditional 3D deconvolution algorithms based on the beam propagation model facilitate high-resolution 3D reconstructions. However, such a high-precision model is not robust enough for the experimental data with different system errors such as optical aberrations and background fluorescence, which bring great periodic artifacts and reduce the image contrast. In order to solve this problem, here we propose a phase-space deconvolution method for light field microscopy, which fully exploits the smoothness prior in the phase-space domain. By modeling the imaging process in the phase-space domain, we convert the spatially-nonuniform point spread function (PSF) into a spatially-uniform one with a much smaller size. Experiments on various biological samples and resolution charts are demonstrated to verify the contrast enhancement with much fewer artifacts and 10-times less computational cost by our method without any hardware modifications required.

Single-shot aperture-scanning Fourier ptychography

Aperture-scanning Fourier ptychography [Opt. Express22, 13586 (2014)] is a promising non-interferometric wavefront measurement technique. It eliminates the thin-sample requirement in typical Fourier ptychography employing angle-varying illumination. However, as aperture-scanning Fourier ptychography is based on step-by-step scanning, it requires long data acquisition time and a high-stability optical system. In this paper, we propose a single-shot aperture-scanning Fourier ptychography method. In our method, multiple low-resolution images are collected in a single shot by inserting a Dammann grating at a certain distance before the aperture, and the images are subsequently converted to a high-resolution complex wavefront. Compared with scanning-based aperture-scanning Fourier ptychography, the total acquisition time of the proposed method is dramatically reduced. The feasibility of our proposed method is demonstrated by proof-of-concept experiments.

Dual-wavelength Fourier ptychography using a single LED

We propose dual-wavelength Fourier ptychography for topographic measurement. To extend the axial measurement range, a single light-emitting diode (LED) and two appropriate bandpass filters are employed. This provides a speckle-free phase image, and reduces the possibility of a systematic error, which yields a high-quality topographic image. The proposed system can measure the surface topography in the range of nano- to micro-structures. The performance of the system is experimentally verified.

System calibration method for Fourier ptychographic microscopy

Fourier ptychographic microscopy (FPM) is a recently proposed computational imaging technique with both high-resolution and wide field of view. In current FPM imaging platforms, systematic error sources come from aberrations, light-emitting diode (LED) intensity fluctuation, parameter imperfections, and noise, all of which may severely corrupt the reconstruction results with similar artifacts. Therefore, it would be unlikely to distinguish the dominating error from these degraded reconstructions without any preknowledge. In addition, systematic error is generally a mixture of various error sources in the real situation, and it cannot be separated due to their mutual restriction and conversion. To this end, we report a system calibration procedure, termed SC-FPM, to calibrate the mixed systematic errors simultaneously from an overall perspective, based on the simulated annealing algorithm, the LED intensity correction method, the nonlinear regression process, and the adaptive step-size strategy, which involves the evaluation of an error metric at each iteration step, followed by the re-estimation of accurate parameters. The performance achieved both in simulations and experiments demonstrates that the proposed method outperforms other state-of-the-art algorithms. The reported system calibration scheme improves the robustness of FPM, relaxes the experiment conditions, and does not require any preknowledge, which makes the FPM more pragmatic.

SAVI: Synthetic apertures for long-range, subdiffraction-limited visible imaging using Fourier ptychography

Synthetic aperture radar is a well-known technique for improving resolution in radio imaging. Extending these synthetic aperture techniques to the visible light domain is not straightforward because optical receivers cannot measure phase information. We propose to use macroscopic Fourier ptychography (FP) as a practical means of creating a synthetic aperture for visible imaging to achieve subdiffraction-limited resolution. We demonstrate the first working prototype for macroscopic FP in a reflection imaging geometry that is capable of imaging optically rough objects. In addition, a novel image space denoising regularization is introduced during phase retrieval to reduce the effects of speckle and improve perceptual quality of the recovered high-resolution image. Our approach is validated experimentally where the resolution of various diffuse objects is improved sixfold.

Wide-field Fourier ptychographic microscopy using laser illumination source

Fourier ptychographic (FP) microscopy is a coherent imaging method that can synthesize an image with a higher bandwidth using multiple low-bandwidth images captured at different spatial frequency regions. The method's demand for multiple images drives the need for a brighter illumination scheme and a high-frame-rate camera for a faster acquisition. We report the use of a guided laser beam as an illumination source for an FP microscope. It uses a mirror array and a 2-dimensional scanning Galvo mirror system to provide a sample with plane-wave illuminations at diverse incidence angles. The use of a laser presents speckles in the image capturing process due to reflections between glass surfaces in the system. They appear as slowly varying background fluctuations in the final reconstructed image. We are able to mitigate these artifacts by including a phase image obtained by differential phase contrast (DPC) deconvolution in the FP algorithm. We use a 1-Watt laser configured to provide a collimated beam with 150 mW of power and beam diameter of 1 cm to allow for the total capturing time of 0.96 seconds for 96 raw FPM input images in our system, with the camera sensor's frame rate being the bottleneck for speed. We demonstrate a factor of 4 resolution improvement using a 0.1 NA objective lens over the full camera field-of-view of 2.7 mm by 1.5 mm.

Quantitative phase imaging and complex field reconstruction by pupil modulation differential phase contrast

Differential phase contrast (DPC) is a non-interferometric quantitative phase imaging method achieved by using an asymmetric imaging procedure. We report a pupil modulation differential phase contrast (PMDPC) imaging method by filtering a sample's Fourier domain with half-circle pupils. A phase gradient image is captured with each half-circle pupil, and a quantitative high resolution phase image is obtained after a deconvolution process with a minimum of two phase gradient images. Here, we introduce PMDPC quantitative phase image reconstruction algorithm and realize it experimentally in a 4f system with an SLM placed at the pupil plane. In our current experimental setup with the numerical aperture of 0.36, we obtain a quantitative phase image with a resolution of 1.73μm after computationally removing system aberrations and refocusing. We also extend the depth of field digitally by 20 times to ±50μm with a resolution of 1.76μm.