Aperture-scanning Fourier ptychography for 3D refocusing and super-resolution macroscopic imaging

Ptycho-endoscopy on a lensless ultrathin fiber bundle tip

Synthetic aperture radar (SAR) utilizes an aircraft-carried antenna to emit electromagnetic pulses and detect the returning echoes. As the aircraft travels across a designated area, it synthesizes a large virtual aperture to improve image resolution. Inspired by SAR, we introduce synthetic aperture ptycho-endoscopy (SAPE) for micro-endoscopic imaging beyond the diffraction limit. SAPE operates by hand-holding a lensless fiber bundle tip to record coherent diffraction patterns from specimens. The fiber cores at the distal tip modulate the diffracted wavefield within a confined area, emulating the role of the 'airborne antenna' in SAR. The handheld operation introduces positional shifts to the tip, analogous to the aircraft's movement. These shifts facilitate the acquisition of a ptychogram and synthesize a large virtual aperture extending beyond the bundle's physical limit. We mitigate the influences of hand motion and fiber bending through a low-rank spatiotemporal decomposition of the bundle's modulation profile. Our tests demonstrate the ability to resolve a 548-nm linewidth on a resolution target. The achieved space-bandwidth product is ~1.1 million effective pixels, representing a 36-fold increase compared to that of the original fiber bundle. Furthermore, SAPE's refocusing capability enables imaging over an extended depth of field exceeding 2 cm. The aperture synthesizing process in SAPE surpasses the diffraction limit set by the probe's maximum collection angle, opening new opportunities for both fiber-based and distal-chip endoscopy in applications such as medical diagnostics and industrial inspection.

Simultaneous Multifocal Plane Fourier Ptychographic Microscopy Utilizing a Standard RGB Camera

Fourier ptychographic microscopy (FPM) is a computational imaging technology that can acquire high-resolution large-area images for applications ranging from biology to microelectronics. In this study, we utilize multifocal plane imaging to enhance the existing FPM technology. Using an RGB light emitting diode (LED) array to illuminate the sample, raw images are captured using a color camera. Then, exploiting the basic optical principle of wavelength-dependent focal length variation, three focal plane images are extracted from the raw image through simple R, G, and B channel separation. Herein, a single aspherical lens with a numerical aperture (NA) of 0.15 was used as the objective lens, and the illumination NA used for FPM image reconstruction was 0.08. Therefore, simultaneous multifocal plane FPM with a synthetic NA of 0.23 was achieved. The multifocal imaging performance of the enhanced FPM system was then evaluated by inspecting a transparent organic light-emitting diode (OLED) sample. The FPM system was able to simultaneously inspect the individual OLED pixels as well as the surface of the encapsulating glass substrate by separating R, G, and B channel images from the raw image, which was taken in one shot.

Spatially-coded Fourier ptychography: flexible and detachable coded thin films for quantitative phase imaging with uniform phase transfer characteristics

Fourier ptychography (FP) is an enabling imaging technique that produces high-resolution complex-valued images with extended field coverages. However, when FP images a phase object with any specific spatial frequency, the captured images contain only constant values, rendering the recovery of the corresponding linear phase ramp impossible. This challenge is not unique to FP but also affects other common microscopy techniques -- a rather counterintuitive outcome given their widespread use in phase imaging. The underlying issue originates from the non-uniform phase transfer characteristic inherent in microscope systems, which impedes the conversion of object wavefields into discernible intensity variations. To address this challenge, we present spatially-coded Fourier ptychography (scFP), a new method that synergizes FP with spatial-domain coded detection for true quantitative phase imaging. In scFP, a flexible and detachable coded thin film is attached atop the image sensor in a regular FP setup. The spatial modulation of this thin film ensures a uniform phase response across the entire synthetic bandwidth. It improves reconstruction quality and corrects refractive index underestimation issues prevalent in conventional FP and related tomographic implementations. The inclusion of the coded thin film further adds a new dimension of measurement diversity in the spatial domain. The development of scFP is expected to catalyse new research directions and applications for phase imaging, emphasizing the need for true quantitative accuracy with uniform frequency response.

Dual-wavelength Fourier ptychographic microscopy for topographic measurement

Topographic measurements of micro- or nanostructures are essential in cutting-edge scientific disciplines such as optical communications, metrology, and structural biology. Despite the advances in surface metrology, measuring micron-scale steps with wide field of view (FOV) and high-resolution remains difficult. This study demonstrates a dual-wavelength Fourier ptychographic microscopy for high-resolution topographic measurement across a wide FOV using an aperture scanning structure. This structure enables the capture of a three-dimensional (3D) sample's scattered field with two different wavelength lasers, thus allowing the axial measurement range growing from nano- to micro-scale with enhanced lateral resolution. To suppress the unavoidable noises and artifacts caused by temporal coherence, system vibration, etc., a total variation (TV) regularization algorithm is introduced for phase retrieval. A blazed grating with micron-scale steps is used as the sample to validate the performance of our method. The agreement between the high-resolution reconstructed topography with our method and that with atomic force microscopy verified the effectiveness. Meanwhile, numerical simulations suggest that the method has the potential to characterize samples with high aspect-ratio steps.

Fourier Ptychographic Microscopy 10 Years on: A Review

Fourier ptychographic microscopy (FPM) emerged as a prominent imaging technique in 2013, attracting significant interest due to its remarkable features such as precise phase retrieval, expansive field of view (FOV), and superior resolution. Over the past decade, FPM has become an essential tool in microscopy, with applications in metrology, scientific research, biomedicine, and inspection. This achievement arises from its ability to effectively address the persistent challenge of achieving a trade-off between FOV and resolution in imaging systems. It has a wide range of applications, including label-free imaging, drug screening, and digital pathology. In this comprehensive review, we present a concise overview of the fundamental principles of FPM and compare it with similar imaging techniques. In addition, we present a study on achieving colorization of restored photographs and enhancing the speed of FPM. Subsequently, we showcase several FPM applications utilizing the previously described technologies, with a specific focus on digital pathology, drug screening, and three-dimensional imaging. We thoroughly examine the benefits and challenges associated with integrating deep learning and FPM. To summarize, we express our own viewpoints on the technological progress of FPM and explore prospective avenues for its future developments.

Multi-wavelength interference phase imaging for automatic breast cancer detection and delineation using diffuse reflection imaging

Millions of women globally are impacted by the major health problem of breast cancer (BC). Early detection of BC is critical for successful treatment and improved survival rates. In this study, we provide a progressive approach for BC detection using multi-wavelength interference (MWI) phase imaging based on diffuse reflection hyperspectral (HS) imaging. The proposed findings are based on the measurement of the interference pattern between the blue (446.6 nm) and red (632 nm) wavelengths. We consider implementing a comprehensive image processing and categorization method based on the use of Fast Fourier (FF) transform analysis pertaining to a change in the refractive index between tumor and normal tissue. We observed that cancer growth affects tissue organization dramatically, as seen by persistently increased refractive index variance in tumors compared normal areas. Both malignant and normal tissue had different depth data collected from it that was analyzed. To enhance the categorization of ex-vivo BC tissue, we developed and validated a training classifier algorithm specifically designed for categorizing HS cube data. Following the application of signal normalization with the FF transform algorithm, our methodology achieved a high level of performance with a specificity (Spec) of 94% and a sensitivity (Sen) of 90.9% for the 632 nm acquired image categorization, based on preliminary findings from breast specimens under investigation. Notably, we successfully leveraged unstained tissue samples to create 3D phase-resolved images that effectively highlight the distinctions in diffuse reflectance features between cancerous and healthy tissue. Preliminary data revealed that our imaging method might be able to assist specialists in safely excising malignant areas and assessing the tumor bed following resection automatically at different depths. This preliminary investigation might result in an effective "in-vivo" disease description utilizing optical technology using a typical RGB camera with wavelength-specific operation with our quantitative phase MWI imaging methodology.

Long-range Fourier ptychographic imaging of the dynamic object with a single camera

Fourier ptychographic imaging technology is a new imaging method proposed in recent years. This technology captures multiple low-resolution images, and synthesizes them into a high-resolution image in the Fourier domain by a phase retrieval algorithm, breaking through the diffraction limit of the lens. In the field of macroscopic Fourier ptychographic imaging, most of the existing research generally focus on high-resolution imaging of static objects, and applying Fourier ptychographic imaging technology to dynamic objects is a hot research area now. At present, most of the researches are to use camera arrays combined with multiplexed lighting, deep learning or other algorithms, but the implementation of these methods is complicated or costly. Based on the diffraction theory of Fourier optics, this paper proposes that by expanding and focusing the illumination area, we can apply Fourier ptychographic imaging technology with a single camera to moving objects within a certain range. Theoretical analysis and experiments prove the feasibility of the proposed method. We successfully achieve high-resolution imaging of the dynamic object, increasing the resolution by about 2.5 times. This paper also researches the impact of speckles in the illuminated area on imaging results and proposes a processing method to reduce the impact of speckles.

Spatial- and Fourier-domain ptychography for high-throughput bio-imaging

First envisioned for determining crystalline structures, ptychography has become a useful imaging tool for microscopists. However, ptychography remains underused by biomedical researchers due to its limited resolution and throughput in the visible light regime. Recent developments of spatial- and Fourier-domain ptychography have successfully addressed these issues and now offer the potential for high-resolution, high-throughput optical imaging with minimal hardware modifications to existing microscopy setups, often providing an excellent trade-off between resolution and field of view inherent to conventional imaging systems, giving biomedical researchers the best of both worlds. Here, we provide extensive information to enable the implementation of ptychography by biomedical researchers in the visible light regime. We first discuss the intrinsic connections between spatial-domain coded ptychography and Fourier ptychography. A step-by-step guide then provides the user instructions for developing both systems with practical examples. In the spatial-domain implementation, we explain how a large-scale, high-performance blood-cell lens can be made at negligible expense. In the Fourier-domain implementation, we explain how adding a low-cost light source to a regular microscope can improve the resolution beyond the limit of the objective lens. The turnkey operation of these setups is suitable for use by professional research laboratories, as well as citizen scientists. Users with basic experience in optics and programming can build the setups within a week. The do-it-yourself nature of the setups also allows these procedures to be implemented in laboratory courses related to Fourier optics, biomedical instrumentation, digital image processing, robotics and capstone projects.

High-precision Fourier ptychographic microscopy based on Gaussian apodization coherent transfer function constraints

Fourier ptychographic microscopy (FPM) combines the concepts of phase retrieval algorithms and synthetic apertures and can solve the problem in which it is difficult to combine a large field of view with high resolution. However, the use of the coherent transfer function in conventional calculations to describe the linear transfer process of an imaging system can lead to ringing artifacts. In addition, the Gerchberg-Saxton iterative algorithm can cause the phase retrieval part of the FPM algorithm to fall into a local optimum. In this paper, Gaussian apodization coherent transfer function is proposed to describe the imaging process and is combined with an iterative method based on amplitude weighting and phase gradient descent to reduce the presence of ringing artifacts while ensuring the accuracy of the reconstructed results. In simulated experiments, the proposed algorithm is shown to give a smaller mean square error and higher structural similarity, both in the presence and absence of noise. Finally, the proposed algorithm is validated in terms of giving reconstruction results with high accuracy and high resolution, using images acquired with a new microscope system and open-source images.

Undersampled Fourier ptychography for reflective-based long range imaging

Fourier ptychography (FP) can be a promising technique for long-range and high-resolution imaging. In this work, we explore reconstructions with undersampled data for meter-scale reflective based Fourier ptychographic imaging. To reconstruct with under-sampling captures, we propose a novel cost function for FP phase retrieval and design a new optimization algorithm based on gradient descent. To verify the proposed methods, we perform the high-fidelity reconstruction of the targets with sampling parameter less than one. Compared to the state-of-the-art alternative-projectionbased FP algorithm, the proposed one can achieve the same performance but with much less data.

Reduction in required volume of imaging data and image reconstruction time for adaptive-illumination Fourier ptychographic microscopy

Fourier ptychographic microscopy (FPM) is a promising super-resolution computational imaging technology. It stitches a series of low-resolution (LR) images in the Fourier domain by an iterative method. Thus, it obtains a large field of view and high-resolution quantitative phase images. Owing to its capability to perform high-spatial bandwidth product imaging, FPM is widely used in the reconstruction of conventional static samples. However, the influence of the FPM imaging mechanism limits its application in high-speed dynamic imaging. To solve this problem, an adaptive-illumination FPM scheme using regional energy estimation is proposed. Starting with several captured real LR images, the energy distribution of all LR images is estimated, and select the measurement images with large information to perform FPM reconstruction. Simulation and experimental results show that the method produces efficient imaging performance and reduces the required volume of data to more than 65% while ensuring the quality of FPM reconstruction.

Optical ptychography for biomedical imaging: recent progress and future directions [Invited]

Ptychography is an enabling microscopy technique for both fundamental and applied sciences. In the past decade, it has become an indispensable imaging tool in most X-ray synchrotrons and national laboratories worldwide. However, ptychography's limited resolution and throughput in the visible light regime have prevented its wide adoption in biomedical research. Recent developments in this technique have resolved these issues and offer turnkey solutions for high-throughput optical imaging with minimum hardware modifications. The demonstrated imaging throughput is now greater than that of a high-end whole slide scanner. In this review, we discuss the basic principle of ptychography and summarize the main milestones of its development. Different ptychographic implementations are categorized into four groups based on their lensless/lens-based configurations and coded-illumination/coded-detection operations. We also highlight the related biomedical applications, including digital pathology, drug screening, urinalysis, blood analysis, cytometric analysis, rare cell screening, cell culture monitoring, cell and tissue imaging in 2D and 3D, polarimetric analysis, among others. Ptychography for high-throughput optical imaging, currently in its early stages, will continue to improve in performance and expand in its applications. We conclude this review article by pointing out several directions for its future development.

Parallel Fourier ptychographic microscopy reconstruction method based on FPGA

Fourier ptychographic microscopy (FPM) can bypass the limitation of spatial bandwidth product to get images with large field-of-view and high resolution. The complicated sequential iterative calculation in the FPM reconstruction process reduces the reconstruction efficiency of the FPM. Therefore, we propose a parallel FPM reconstruction method based on field programmable gate array (FPGA) to accelerate the FPM reconstruction process. Using this method, multiple sub-regions in the Fourier domain can be computed in parallel and we customize a dedicated high-performance computational architecture for this approach. We deploy 4 FPM reconstruct computing architectures with a parallelism of 4 in a FPGA to compute the FPM reconstruction process, achieving the speed nearly 180 times faster than traditional methods. The proposed method provides a new perspective of parallel computing for FPM reconstruction.

Learning-based single-shot long-range synthetic aperture Fourier ptychographic imaging with a camera array

In this Letter, we report a new long-range synthetic aperture Fourier ptychographic imaging technique, termed learning-based single-shot synthetic aperture imaging (LSS-SAI). LSS-SAI uses a camera array to record low-resolution intensity images corresponding to different non-overlapping spectral regions in parallel, which are synthesized to reconstruct a super-resolved high-quality image based on a physical model-based dual-regression deep neural network. Compared with conventional macroscopic Fourier ptychographic imaging, LSS-SAI overcomes the stringent requirement on a large amount of raw data with a high spectral overlapping ratio for high-resolution, high signal-to-noise imaging of reflective objects with diffuse surfaces, making single-shot long-range synthetic aperture imaging possible. Experimental results on rough reflective samples show that our approach can improve the peak signal-to-noise ratio (PSNR) and structural similarity (SSIM) by 10.56 dB and 0.26, respectively. We also demonstrate the single-shot ptychography capability of the proposed approach by the synthetic aperture imaging of a dynamic scene at a camera-limited speed (30 fps). To the best of our knowledge, this is the first demonstration of macroscopic Fourier ptychography to single-shot synthetic aperture imaging of dynamic events.

Batch-based alternating direction methods of multipliers for Fourier ptychography

Fourier ptychography (FP) has been developed as a general imaging tool for various applications. However, the redundancy data has to be enforced to get a stable recovery, leading to a large dataset and a high computational cost. Based on the additive property of the optical pupils in FP recovery, we report batch-based alternating direction methods of multipliers (ADMM) for FP reconstruction. The reported scheme is performed by implementing partial updates in sub-problems of the standard ADMM. We validate the reconstruction performance using both simulated and experimental measurements. Compared with the embedded pupil function recovery (EPRY) algorithm, the proposed algorithms can converge faster and produce higher-quality images.

High-speed multi-objective Fourier ptychographic microscopy

The ability of a microscope to rapidly acquire wide-field, high-resolution images is limited by both the optical performance of the microscope objective and the bandwidth of the detector. The use of multiple detectors can increase electronic-acquisition bandwidth, but the use of multiple parallel objectives is problematic since phase coherence is required across the multiple apertures. We report a new synthetic-aperture microscopy technique based on Fourier ptychography, where both the illumination and image-space numerical apertures are synthesized, using a spherical array of low-power microscope objectives that focus images onto mutually incoherent detectors. Phase coherence across apertures is achieved by capturing diffracted fields during angular illumination and using ptychographic reconstruction to synthesize wide-field, high-resolution, amplitude and phase images. Compared to conventional Fourier ptychography, the use of multiple objectives reduces image acquisition times by increasing the area for sampling the diffracted field. We demonstrate the proposed scaleable architecture with a nine-objective microscope that generates an 89-megapixel, 1.1 µm resolution image nine-times faster than can be achieved with a single-objective Fourier-ptychographic microscope. New calibration procedures and reconstruction algorithms enable the use of low-cost 3D-printed components for longitudinal biological sample imaging. Our technique offers a route to high-speed, gigapixel microscopy, for example, imaging the dynamics of large numbers of cells at scales ranging from sub-micron to centimetre, with an enhanced possibility to capture rare phenomena.

Pose correction scheme for camera-scanning Fourier ptychography based on camera calibration and homography transform

Fourier ptychography (FP), as a computational imaging method, is a powerful tool to improve imaging resolution. Camera-scanning Fourier ptychography extends the application of FP from micro to macro creatively. Due to the non-ideal scanning of the camera driven by the mechanical translation stage, the pose error of the camera occurs, greatly degrading the reconstruction quality, while a precise translation stage is expensive and not suitable for wide-range imaging. Here, to improve the imaging performance of camera-scanning Fourier ptychography, we propose a pose correction scheme based on camera calibration and homography transform approaches. The scheme realizes the accurate alignment of data set and location error correction in the frequency domain. Simulation and experimental results demonstrate this method can optimize the reconstruction results and realize high-quality imaging effectively. Combined with the feature recognition algorithm, the scheme provides the possibility for applying FP in remote sensing imaging and space imaging.

Spectrum sampling optimization for quantitative phase imaging based on Kramers-Kronig relations

Annular-illumination quantitative phase imaging based on space-domain Kramers-Kronig relations (AIKK) is a newly developed technique that is object-independent and non-iterative reconstructed inherently. Only capturing four low-resolution images, the AIKK system gains a resolution enhancement of nearly twofold. Under matching constraints between the illumination wave vector and pupil function aperture, we set a spectrum sampling criterion and establish a spectrum effective utilization model to search for the optimal solution of spectrum distribution for the specific annular structure. In view of the square spectrum structure, a diagonal-expanded sampling based AIKK method (DES-AIKK) is presented to get rid of the pixel aliasing problem. It is worth noting that the space-bandwidth-time product (SBP-T) further increases to 439.51 megapixels (1.8× of AIKK). Our work provides the guidelines and insights for designing the most suitable AIKK platform for high-throughput microscopic applications in pathology and real-time dynamic observation.

Experimental Study on the Exploration of Camera Scanning Reflective Fourier Ptychography Technology for Far-Field Imaging

Fourier ptychography imaging is a powerful phase retrieval method that can be used to realize super-resolution. In this study, we establish a mathematical model of long-distance camera scanning based on reflective Fourier ptychography imaging. In order to guarantee the effective recovery of a high-resolution image in the experiment, we analyze the influence of laser coherence in different modes and the surface properties of diverse materials for diffused targets. For the analysis, we choose a single-mode fiber laser as the illumination source and metal materials with high diffused reflectivity as the experimental targets to ensure the validity of the experimental results. Based on the above, we emulate camera scanning with a single camera attached to an X-Y translation stage, and an experimental system with a working distance of 3310 mm is used as an example to image a fifty-cent coin. We also perform speckle analysis for rough targets and calculate the average speckle size using a normalized autocorrelation function in different positions. The method of calculating the average speckle size for everyday objects provides the premise for subsequent research on image quality evaluation; meanwhile, the coherence of the light field and the targets with high reflectivity under this experiment provide an application direction for the further development of the technique, such as computer vision, surveillance and remote sensing.

Blood-Coated Sensor for High-Throughput Ptychographic Cytometry on a Blu-ray Disc

The Blu-ray drive is an engineering masterpiece that integrates disc rotation, pickup head translation, and three lasers in a compact and portable format. Here, we integrate a blood-coated image sensor with a modified Blu-ray drive for high-throughput cytometric analysis of various biospecimens. In this device, samples are mounted on the rotating Blu-ray disc and illuminated by the built-in lasers from the pickup head. The resulting coherent diffraction patterns are then recorded by the blood-coated image sensor. The rich spatial features of the blood-cell monolayer help down-modulate the object information for sensor detection, thus forming a high-resolution computational biolens with a theoretically unlimited field of view. With the acquired data, we develop a lensless coherent diffraction imaging modality termed rotational ptychography for image reconstruction. We show that our device can resolve the 435 nm line width on the resolution target and has a field of view only limited by the size of the Blu-ray disc. To demonstrate its applications, we perform high-throughput urinalysis by locating disease-related calcium oxalate crystals over the entire microscope slide. We also quantify different types of cells on a blood smear with an acquisition speed of ∼10,000 cells per second. For in vitro experiments, we monitor live bacterial cultures over the entire Petri dish with single-cell resolution. Using biological cells as a computational lens could enable new intriguing imaging devices for point-of-care diagnostics. Modifying a Blu-ray drive with the blood-coated sensor further allows the spread of high-throughput optical microscopy from well-equipped laboratories to citizen scientists worldwide.

Robust Fourier ptychographic microscopy via a physics-based defocusing strategy for calibrating angle-varied LED illumination

Fourier ptychographic microscopy (FPM) is a recently developed computational imaging technique for wide-field, high-resolution microscopy with a high space-bandwidth product. It integrates the concepts of synthetic aperture and phase retrieval to surpass the resolution limit imposed by the employed objective lens. In the FPM framework, the position of each sub-spectrum needs to be accurately known to ensure the success of the phase retrieval process. Different from the conventional methods with mechanical adjustment or data-driven optimization strategies, here we report a physics-based defocusing strategy for correcting large-scale positional deviation of the LED illumination in FPM. Based on a subpixel image registration process with a defocused object, we can directly infer the illumination parameters including the lateral offsets of the light source, the in-plane rotation angle of the LED array, and the distance between the sample and the LED board. The feasibility and effectiveness of our method are validated with both simulations and experiments. We show that the reported strategy can obtain high-quality reconstructions of both the complex object and pupil function even the LED array is randomly placed under the sample with both unknown lateral offsets and rotations. As such, it enables the development of robust FPM systems by reducing the requirements on fine mechanical adjustment and data-driven correction in the construction process.

Analysis, Simulations, and Experiments for Far-Field Fourier Ptychography Imaging Using Active Coherent Synthetic-Aperture

Fourier ptychography (FP) is a powerful phase retrieval method that can be used to reconstruct missing high-frequency details and high-space-bandwidth products in microscopy. In this study, we further advanced the application of FP in microscopic imaging to the field of macroscopic far-field imaging, incorporating camera scanning for spatial resolution improvement. First, on the basis of the Fraunhofer diffraction mechanism and the transmission imaging model, we found the analysis of the associated theoretical fundamentals via simulations and experiments to be crucially relevant to the far-field of FP imaging. Second, we built an experimental device with long-distance imaging and experimentally demonstrated the relationship between the spectrum overlap ratio and the reconstructed high-resolution image. The simulation and experimental results showed that an overlap ratio higher than 50% had a good reconstruction effect. Third, camera scanning was used to obtain low-resolution intensity images in this study, for which the scanning range was wide and spherical wave illumination was satisfied, and therefore different positions corresponded to different aberrations of low-resolution intensity images, and even different positions of the same image had aberration differences, leading to inconsistencies in the aberrations of different images. Therefore, in the reconstruction process, we further overcame the effect of the inconsistency of aberrations of different images using the partition reconstruction method, which involves cutting the image into smaller parts for reconstruction. Finally, with the proposed partition reconstruction algorithm, we were able to resolve 40 μm line width of GBA1 resolution object and obtain a spatial resolution gain of 4× with a working distance of 2 m.

Snapshot ptychography on array cameras

We use convolutional neural networks to recover images optically down-sampled by 6.7 × using coherent aperture synthesis over a 16 camera array. Where conventional ptychography relies on scanning and oversampling, here we apply decompressive neural estimation to recover full resolution image from a single snapshot, although as shown in simulation multiple snapshots can be used to improve signal-to-noise ratio (SNR). In place training on experimental measurements eliminates the need to directly calibrate the measurement system. We also present simulations of diverse array camera sampling strategies to explore how snapshot compressive systems might be optimized.

Ptychographic sensor for large-scale lensless microbial monitoring with high spatiotemporal resolution

Traditional microbial detection methods often rely on the overall property of microbial cultures and cannot resolve individual growth event at high spatiotemporal resolution. As a result, they require bacteria to grow to confluence and then interpret the results. Here, we demonstrate the application of an integrated ptychographic sensor for lensless cytometric analysis of microbial cultures over a large scale and with high spatiotemporal resolution. The reported device can be placed within a regular incubator or used as a standalone incubating unit for long-term microbial monitoring. For longitudinal study where massive data are acquired at sequential time points, we report a new temporal-similarity constraint to increase the temporal resolution of ptychographic reconstruction by 7-fold. With this strategy, the reported device achieves a centimeter-scale field of view, a half-pitch spatial resolution of 488 nm, and a temporal resolution of 15-s intervals. For the first time, we report the direct observation of bacterial growth in a 15-s interval by tracking the phase wraps of the recovered images, with high phase sensitivity like that in interferometric measurements. We also characterize cell growth via longitudinal dry mass measurement and perform rapid bacterial detection at low concentrations. For drug-screening application, we demonstrate proof-of-concept antibiotic susceptibility testing and perform single-cell analysis of antibiotic-induced filamentation. The combination of high phase sensitivity, high spatiotemporal resolution, and large field of view is unique among existing microscopy techniques. As a quantitative and miniaturized platform, it can improve studies with microorganisms and other biospecimens at resource-limited settings.

Quantitative multi-height phase retrieval via a coded image sensor

Multi-height phase retrieval introduces different object-to-detector distances for obtaining phase diversity measurements. In the acquisition process, the slow-varying phase information, however, cannot be converted to intensity variations for detection. Therefore, the low-frequency contents of the phase profile are lost during acquisition and cannot be properly restored via phase retrieval. Here, we demonstrate the use of a coded image sensor for addressing this challenge in multi-height phase retrieval. In our scheme, we add a coded layer on top of the image sensor for encoding the slow-varying complex wavefronts into intensity variations of the modulated patterns. Inspired by the concept of blind ptychography, we report a reconstruction scheme to jointly recover the complex object and the unknown coded layer using multi-height measurements. With both simulation and experimental results, we show that the recovered phase is quantitative and the slow-varying phase profiles can be properly restored using lensless multi-height measurements. We also show that the image quality using the coded sensor is better than that of a regular image sensor. For demonstrations, we validate the reported scheme with various biospecimens and compare the results to those of regular lensless multi-height phase retrieval. The use of a coded image sensor may enable true quantitative phase imaging for the lensless multi-height, multi-wavelength, and transport-of-intensity equation approaches.

Dither removing Fourier ptychographic microscope based on a two-axis rotation stage

SIGNIFICANCE

Large space-bandwidth product is highly desirable in many biomedical imaging. Fourier ptychographic microscopy (FPM) is a computational imaging technique that can significantly increase the space-bandwidth product of a standard microscope. The illuminator of a Fourier ptychographic microscope is not flexible at present, and it is inconvenient to meet different imaging needs.

AIM

An illuminator based on a two-axis motorized rotation stage was presented to provide a more flexible illuminating way with the goal of meeting different imaging needs.

APPROACH

The illuminator adopts a concentric illuminating method to provide coherent illumination in any direction on the sample plane. The sampling pattern can be freely designed and changed according to the parameters of the imaging system. A dither removing algorithm was proposed to remove the potential dither influence introduced in the image acquisition process.

RESULTS

The illuminator could be conveniently integrated into different imaging systems. The feasibility and flexibility were demonstrated by applying it to imaging systems with numerical aperture of 0.045 and 0.01. The resolution gain is about 4- and 13-fold, respectively. The effectiveness of the dither removing algorithm was validated in both simulation and experiment.

CONCLUSIONS

A more flexible illuminator for FPM was presented to meet different imaging needs. A dither removing algorithm was proposed to remove dither influence.

Optical scanning Fourier ptychographic microscopy

We propose a lower-cost and practical active scanning optical scanning Fourier ptychographic microscopy (OSFPM). Featured is a simple setup of Galvo mirrors capable of scanning large-sized objects. The active scanning laser beam is projected onto the sample in a circular pattern to form multiple lower-resolution images. With multiple lower-resolution images, a higher-resolution image is subsequently reconstructed. The OSFPM is able to more precisely control the overlap of the incident light illumination as compared to that in conventional LED-based or other laser-based scanning FPM systems. The proposed microscope is also suitable for applications where a larger size of the object needs to be imaged with efficient illumination.

Coherent synthetic aperture imaging for visible remote sensing via reflective Fourier ptychography

Synthetic aperture radar can measure the phase of a microwave with an antenna, which cannot be directly extended to visible light imaging due to phase lost. In this Letter, we report an active remote sensing with visible light via reflective Fourier ptychography, termed coherent synthetic aperture imaging (CSAI), achieving high resolution, a wide field-of-view (FOV), and phase recovery. A proof-of-concept experiment is reported with laser scanning and a collimator for the infinite object. Both smooth and rough objects are tested, and the spatial resolution increased from 15.6 to 3.48 µm with a factor of 4.5. The speckle noise can be suppressed obviously, which is important for coherent imaging. Meanwhile, the CSAI method can tackle the aberration induced from the optical system by one-step deconvolution and shows the potential to replace the adaptive optics for aberration removal of atmospheric turbulence.

Autofocusing technologies for whole slide imaging and automated microscopy

Whole slide imaging (WSI) has moved digital pathology closer to diagnostic practice in recent years. Due to the inherent tissue topography variability, accurate autofocusing remains a critical challenge for WSI and automated microscopy systems. The traditional focus map surveying method is limited in its ability to acquire a high degree of focus points while still maintaining high throughput. Real-time approaches decouple image acquisition from focusing, thus allowing for rapid scanning while maintaining continuous accurate focus. This work reviews the traditional focus map approach and discusses the choice of focus measure for focal plane determination. It also discusses various real-time autofocusing approaches including reflective-based triangulation, confocal pinhole detection, low-coherence interferometry, tilted sensor approach, independent dual sensor scanning, beam splitter array, phase detection, dual-LED illumination and deep-learning approaches. The technical concepts, merits and limitations of these methods are explained and compared to those of a traditional WSI system. This review may provide new insights for the development of high-throughput automated microscopy imaging systems that can be made broadly available and utilizable without loss of capacity.

High-resolution and large field-of-view Fourier ptychographic microscopy and its applications in biomedicine

Fourier ptychographic microscopy (FPM) is a promising and fast-growing computational imaging technique with high resolution, wide field-of-view (FOV) and quantitative phase recovery, which effectively tackles the problems of phase loss, aberration-introduced artifacts, narrow depth-of-field and the trade-off between resolution and FOV in conventional microscopy simultaneously. In this review, we provide a comprehensive roadmap of microscopy, the fundamental principles, advantages, and drawbacks of existing imaging techniques, and the significant roles that FPM plays in the development of science. Since FPM is an optimization problem in nature, we discuss the framework and related work. We also reveal the connection of Euler's formula between FPM and structured illumination microscopy. We review recent advances in FPM, including the implementation of high-precision quantitative phase imaging, high-throughput imaging, high-speed imaging, three-dimensional imaging, mixed-state decoupling, and introduce the prosperous biomedical applications. We conclude by discussing the challenging problems and future applications. FPM can be extended to a kind of framework to tackle the phase loss and system limits in the imaging system. This insight can be used easily in speckle imaging, incoherent imaging for retina imaging, large-FOV fluorescence imaging, etc.

Long-Distance Sub-Diffraction High-Resolution Imaging Using Sparse Sampling

How to perform imaging beyond the diffraction limit has always been an essential subject for the research of optical systems. One effective way to achieve this purpose is Fourier ptychography, which has been widely used in microscopic imaging. However, microscopic imaging measurement technology cannot be directly extended to imaging macro objects at long distances. In this paper, a reconstruction algorithm is proposed to solve the need for oversampling low-resolution images, and it is successfully applied to macroscopic imaging. Compared with the traditional FP technology, the proposed sub-sampling method can significantly reduce the number of iterations in reconstruction. Experiments prove that the proposed method can reconstruct low-resolution images captured by the camera and achieve high-resolution imaging of long-range macroscopic objects.

Fourier ptychography: current applications and future promises

Traditional imaging systems exhibit a well-known trade-off between the resolution and the field of view of their captured images. Typical cameras and microscopes can either "zoom in" and image at high-resolution, or they can "zoom out" to see a larger area at lower resolution, but can rarely achieve both effects simultaneously. In this review, we present details about a relatively new procedure termed Fourier ptychography (FP), which addresses the above trade-off to produce gigapixel-scale images without requiring any moving parts. To accomplish this, FP captures multiple low-resolution, large field-of-view images and computationally combines them in the Fourier domain into a high-resolution, large field-of-view result. Here, we present details about the various implementations of FP and highlight its demonstrated advantages to date, such as aberration recovery, phase imaging, and 3D tomographic reconstruction, to name a few. After providing some basics about FP, we list important details for successful experimental implementation, discuss its relationship with other computational imaging techniques, and point to the latest advances in the field while highlighting persisting challenges.

Wide-field, high-resolution lensless on-chip microscopy via near-field blind ptychographic modulation

We report a novel lensless on-chip microscopy platform based on near-field blind ptychographic modulation. In this platform, we place a thin diffuser in between the object and the image sensor for light wave modulation. By blindly scanning the unknown diffuser to different x-y positions, we acquire a sequence of modulated intensity images for quantitative object recovery. Different from previous ptychographic implementations, we employ a unit magnification configuration with a Fresnel number of ∼50 000, which is orders of magnitude higher than those of previous ptychographic setups. The unit magnification configuration allows us to have the entire sensor area, 6.4 mm by 4.6 mm, as the imaging field of view. The ultra-high Fresnel number enables us to directly recover the positional shift of the diffuser in the phase retrieval process, addressing the positioning accuracy issue plaguing regular ptychographic experiments. In our implementation, we use a low-cost, DIY scanning stage to perform blind diffuser modulation. Precise mechanical scanning that is critical in conventional ptychography experiments is no longer needed in our setup. We further employ an up-sampling phase retrieval scheme to bypass the resolution limit set by the imager pixel size and demonstrate a half-pitch resolution of 0.78 μm. We validate the imaging performance via in vitro cell cultures, transparent and stained tissue sections, and a thick biological sample. We show that the recovered quantitative phase map can be used to perform effective cell segmentation of a dense yeast culture. We also demonstrate 3D digital refocusing of the thick biological sample based on the recovered wavefront. The reported platform provides a cost-effective and turnkey solution for large field-of-view, high-resolution, and quantitative on-chip microscopy. It is adaptable for a wide range of point-of-care-, global-health-, and telemedicine-related applications.

Imaging Correlography Using Ptychography

Imaging correlography, an effective method for long-distance imaging, recovers an object using only the knowledge of the Fourier modulus, without needing phase information. It is not sensitive to atmospheric turbulence or optical imperfections. However, the unreliability of traditional phase retrieval algorithms in imaging correlography has hindered their development. In this work, we join imaging correlography and ptychography together to overcome such obstacles. Instead of detecting the whole object, the object is measured part-by-part with a probe moving in a ptychographic way. A flexible optimization framework is proposed to reconstruct the object rapidly and reliably within a few iterations. In addition, novel image space denoising regularization is plugged into the loss function to reduce the effects of input noise and improve the perceptual quality of the recovered image. Experiments demonstrate that four-fold resolution gains are achievable for the proposed imaging method. We can obtain satisfactory results for both visual and quantitative metrics with one-sixth of the measurements in the conventional imaging correlography. Therefore, the proposed imaging technique is more suitable for long-range practical applications.

Computational aberration correction of VIS-NIR multispectral imaging microscopy based on Fourier ptychography

Due to the chromatic dispersion properties inherent in all optical materials, even the best-designed multispectral objective will exhibit residual chromatic aberration. Here, we demonstrate a multispectral microscope with a computational scheme based on the Fourier ptychographic microscopy (FPM) to correct these effects in order to render undistorted, in-focus images. The microscope consists of 4 spectral channels ranging from 405 nm to 1552 nm. After the computational aberration correction, it can achieve isotropic resolution enhancement as verified with the Siemens star sample. We image a flip-chip to show the promise of our system to conduct fault detection on silicon chips. This computational approach provides a cost-efficient strategy for high quality multispectral imaging over a broad spectral range.

Phase and amplitude imaging with quantum correlations through Fourier Ptychography

Extracting as much information as possible about an object when probing with a limited number of photons is an important goal with applications from biology and security to metrology. Imaging with a few photons is a challenging task as the detector noise and stray light are then predominant, which precludes the use of conventional imaging methods. Quantum correlations between photon pairs has been exploited in a so called ‘heralded imaging scheme’ to eliminate this problem. However these implementations have so-far been limited to intensity imaging and the crucial phase information is lost in these methods. In this work, we propose a novel quantum-correlation enabled Fourier Ptychography technique, to capture high-resolution amplitude and phase images with a few photons. This is enabled by the heralding of single photons combined with Fourier ptychographic reconstruction. We provide experimental validation and discuss the advantages of our technique that include the possibility of reaching a higher signal to noise ratio and non-scanning Fourier Ptychographic acquisition.

Field-portable quantitative lensless microscopy based on translated speckle illumination and sub-sampled ptychographic phase retrieval

We report a compact, cost-effective, and field-portable lensless imaging platform for quantitative microscopy. In this platform, the object is placed on top of an image sensor chip without using a lens. We use a low-cost galvo scanner to rapidly scan an unknown laser speckle pattern on the object. To address the positioning repeatability and accuracy issues, we directly recover the positional shifts of the speckle pattern based on the phase correlation of the captured images. To bypass the resolution limit set by the imager pixel size, we employ a sub-sampled ptychographic phase retrieval process to recover the complex object. We validate our approach using a resolution target, phase target, and biological sample. Our results show that accurate, high-quality complex images can be obtained from a lensless dataset with as few as ∼10 images. We also demonstrate the reported approach to achieve a 6.4-mm by 4.6-mm field of view and a half-pitch resolution of 1 μm. The reported approach may provide a quantitative lensless imaging strategy for addressing point-of-care-, global-health-, and telemedicine-related challenges.

Fourier ptychographic microscopy reconstruction with multiscale deep residual network

Fourier ptychographic microscopy (FPM) is a newly developed microscopic technique for large field of view, high-resolution and quantitative phase imaging by combining the techniques from ptychographic imaging, aperture synthesizing and phase retrieval. In FPM, an LED array is utilized to illuminate the specimen from different angles and the corresponding intensity images are synthesized to reconstruct a high-resolution complex field. As a flexible and low-cost approach to achieve high-resolution, wide-field and quantitative phase imaging, FPM is of enormous potential in biomedical applications such as hematology and pathology. Conventionally, the FPM reconstruction problem is solved by using a phase retrieval method, termed Alternate Projection. By iteratively updating the Fourier spectrum with low-resolution-intensity images, the result converges to a high-resolution complex field. Here we propose a new FPM reconstruction framework with deep learning methods and design a multiscale, deep residual neural network for FPM reconstruction. We employ the widely used open-source deep learning library PyTorch to train and test our model and carefully choose the hyperparameters of our model. To train and analyze our model, we build a large-scale simulation dataset with an FPM imaging model and an actual dataset captured with an FPM system. The simulation dataset and actual dataset are separated as training datasets and test datasets, respectively. Our model is trained with the simulation training dataset and fine tuned with the fine-tune dataset, which contains actual training data. Both our model and the conventional method are tested on the simulation test dataset and the actual test dataset to evaluate the performances. We also show the reconstruction result of another neural network-based method for comparison. The experiments demonstrate that our model achieves better reconstruction results and consumes much less time than conventional methods. The results also point out that our model is more robust under system aberrations such as noise and blurring (fewer intensity images) compared with conventional methods.

Near-field Fourier ptychography: super-resolution phase retrieval via speckle illumination

High spatial resolution is the goal of many imaging systems. While designing a high-resolution lens with diffraction-limited performance over a large field of view remains a difficult task, creating a complex speckle pattern with wavelength-limited spatial features is easily accomplished with a simple random diffuser. With this observation and the concept of near-field ptychography, we report a new imaging modality, termed near-field Fourier ptychography, to address high-resolution imaging challenges in both microscopic and macroscopic imaging settings. 'Near-field' refers to placing the object at a short defocus distance with a large Fresnel number. We project a speckle pattern with fine spatial features on the object instead of directly resolving the spatial features via a high-resolution lens. We then translate the object (or speckle) to different positions and acquire the corresponding images by using a low-resolution lens. A ptychographic phase retrieval process is used to recover the complex object, the unknown speckle pattern, and the coherent transfer function at the same time. In a microscopic imaging setup, we use a 0.12 numerical aperture (NA) lens to achieve an NA of 0.85 in the reconstruction process. In a macroscale photographic imaging setup, we achieve ~7-fold resolution gain by using a photographic lens. The collection optics do not determine the final achievable resolution; rather, the speckle pattern's feature size does. This is similar to our recent demonstration in fluorescence imaging settings (Guo et al., Biomed. Opt. Express, 9(1), 2018). The reported imaging modality can be employed in light, coherent X-ray, and transmission electron imaging systems to increase resolution and provide quantitative absorption and object phase contrast.

X-ray Fourier ptychography

To a large extent, the performance of imaging systems is determined by their objectives, which affect properties as varied as collection efficiency, resolving power, and image distortions. Such limitations can be addressed by so-called aperture synthesis, a technique used, for instance, in radar, astronomy, and, increasingly, microscopy. Here, we apply such techniques to x-ray imaging and demonstrate how Fourier ptychography can be used at transmission x-ray microscopes to increase resolution, provide quantitative absorption and phase contrast, and allow for corrections of lens aberrations. We anticipate that such methods will find common and frequent applications, alleviating a number of limitations imposed by x-ray optical elements, offering an alternative approach to phase contrast imaging, and providing novel opportunities to mitigate radiation damage.

Scanning diffracted-light photography using white-light and thermal radiation sources

A 4-f imaging arrangement of lenses with a camera and a rotating slit placed at the Fourier plane of the system was used to obtain the optical disturbance produced by a macroscopic sample. The sample was illuminated by collimated beams from white-light and thermal radiation sources. The agreement between simulated and experimental results, obtained by processing the captured images using a Fourier ptychographic algorithm, demonstrates that scanning with the slit the direction of the light diffracted by the sample permits achieving the image diversity necessary for successful implementation of the scanning diffracted-light imaging technique.

Single-shot aperture-scanning Fourier ptychography

Aperture-scanning Fourier ptychography [Opt. Express22, 13586 (2014)] is a promising non-interferometric wavefront measurement technique. It eliminates the thin-sample requirement in typical Fourier ptychography employing angle-varying illumination. However, as aperture-scanning Fourier ptychography is based on step-by-step scanning, it requires long data acquisition time and a high-stability optical system. In this paper, we propose a single-shot aperture-scanning Fourier ptychography method. In our method, multiple low-resolution images are collected in a single shot by inserting a Dammann grating at a certain distance before the aperture, and the images are subsequently converted to a high-resolution complex wavefront. Compared with scanning-based aperture-scanning Fourier ptychography, the total acquisition time of the proposed method is dramatically reduced. The feasibility of our proposed method is demonstrated by proof-of-concept experiments.

Sub-Diffraction Visible Imaging Using Macroscopic Fourier Ptychography and Regularization by Denoising

Imaging past the diffraction limit is of significance to an optical system. Fourier ptychography (FP) is a novel coherent imaging technique that can achieve this goal and it is widely used in microscopic imaging. Most phase retrieval algorithms for FP reconstruction are based on Gaussian measurements which cannot extend straightforwardly to long range, sub-diffraction imaging setup because of laser speckle noise corruption. In this work, a new FP reconstruction framework is proposed for macroscopic visible imaging. When compared with existing research, the reweighted amplitude flow algorithm is adopted for better signal modeling, and the Regularization by Denoising (RED) scheme is introduced to reduce the effects of speckle. Experiments demonstrate that the proposed method can obtain state-of-the-art recovered results on both visual and quantitative metrics without increasing computation cost, and it is flexible for real imaging applications.

Scanning diffracted-light microscopy

Scanning the direction of the light that is diffracted by a sample permits the achievement of image diversity, which is necessary for implementing the Fourier ptychographic microscopy technique (FPM) using only perpendicular illumination. We also demonstrated that the same method allows for implementation of the illumination-direction-multiplexing FPM technique when the sample is illuminated using a ring-shaped condenser.

Dual-wavelength Fourier ptychography using a single LED

We propose dual-wavelength Fourier ptychography for topographic measurement. To extend the axial measurement range, a single light-emitting diode (LED) and two appropriate bandpass filters are employed. This provides a speckle-free phase image, and reduces the possibility of a systematic error, which yields a high-quality topographic image. The proposed system can measure the surface topography in the range of nano- to micro-structures. The performance of the system is experimentally verified.

Through-focus or volumetric type of optical imaging methods: a review

In recent years, the use of through-focus (TF) or volumetric type of optical imaging has gained momentum in several areas such as biological imaging, microscopy, adaptive optics, material processing, optical data storage, and optical inspection. We provide a review of basic TF optical methods highlighting their design, major unique characteristics, and application space.

Dual-wavelength off-axis digital holography using a single light-emitting diode

We propose a new low-coherence interferometry system for dual-wavelength off-axis digital holography. By utilizing diffraction gratings, two beams with narrower bandwidths and different center wavelengths could be filtered in a single light-emitting diode. The characteristics of the system are analytically determined to extend the coherence length and field-of-view enough for off-axis configuration. The proposed system enables the fast and accurate measurement of the surface profile with more than a micrometer step height and less noise. The performance of the system is verified by the experimental results of a standard height sample.

Fourier ptychographic microscopy using wavelength multiplexing

Fourier ptychographic microscopy (FPM) is a recently developed technique stitching low-resolution images in Fourier domain to realize wide-field high-resolution imaging. However, the time-consuming process of image acquisition greatly narrows its applications in dynamic imaging. We report a wavelength multiplexing strategy to speed up the acquisition process of FPM several folds. A proof-of-concept system is built to verify its feasibility. Distinguished from many current multiplexing methods in Fourier domain, we explore the potential of high-speed FPM in spectral domain. Compatible with most existing FPM methods, our strategy provides an approach to high-speed gigapixel microscopy. Several experimental results are also presented to validate the strategy.

SAVI: Synthetic apertures for long-range, subdiffraction-limited visible imaging using Fourier ptychography

Synthetic aperture radar is a well-known technique for improving resolution in radio imaging. Extending these synthetic aperture techniques to the visible light domain is not straightforward because optical receivers cannot measure phase information. We propose to use macroscopic Fourier ptychography (FP) as a practical means of creating a synthetic aperture for visible imaging to achieve subdiffraction-limited resolution. We demonstrate the first working prototype for macroscopic FP in a reflection imaging geometry that is capable of imaging optically rough objects. In addition, a novel image space denoising regularization is introduced during phase retrieval to reduce the effects of speckle and improve perceptual quality of the recovered high-resolution image. Our approach is validated experimentally where the resolution of various diffuse objects is improved sixfold.