Fourier ptychography multi-parameunter neural network with composite physical priori optimization

Transformed pupil-function misalignment calibration strategy for Fourier ptychographic microscopy

Fourier ptychographic microscopy (FPM) is an enabling quantitative phase imaging technique with both high-resolution (HR) and wide field-of-view (FOV), which can surpass the diffraction limit of the objective lens by employing an LED array to provide angular-varying illumination. The precise illumination angles are critical to ensure exact reconstruction, while it's difficult to separate actual positional parameters in conventional algorithmic self-calibration approaches due to the mixing of multiple systematic error sources. In this paper, we report a pupil-function-based strategy for independently calibrating the position of LED array. We first deduce the relationship between positional deviation and pupil function in the Fourier domain through a common iterative route. Then, we propose a judgment criterion to determine the misalignment situations, which is based on the arrangement of LED array in the spatial domain. By combining the mapping of complex domains, we can accurately solve the spatial positional parameters concerning the LED array through a boundary-finding scheme. Relevant simulations and experiments demonstrate the proposed method is accessible to precisely correct the positional misalignment of LED array. The approach based on the pupil function is expected to provide valuable insights for precise position correction in the field of microscopy.

Fourier Ptychographic Microscopy 10 Years on: A Review

Fourier ptychographic microscopy (FPM) emerged as a prominent imaging technique in 2013, attracting significant interest due to its remarkable features such as precise phase retrieval, expansive field of view (FOV), and superior resolution. Over the past decade, FPM has become an essential tool in microscopy, with applications in metrology, scientific research, biomedicine, and inspection. This achievement arises from its ability to effectively address the persistent challenge of achieving a trade-off between FOV and resolution in imaging systems. It has a wide range of applications, including label-free imaging, drug screening, and digital pathology. In this comprehensive review, we present a concise overview of the fundamental principles of FPM and compare it with similar imaging techniques. In addition, we present a study on achieving colorization of restored photographs and enhancing the speed of FPM. Subsequently, we showcase several FPM applications utilizing the previously described technologies, with a specific focus on digital pathology, drug screening, and three-dimensional imaging. We thoroughly examine the benefits and challenges associated with integrating deep learning and FPM. To summarize, we express our own viewpoints on the technological progress of FPM and explore prospective avenues for its future developments.

Erratum: Fourier ptychography multi-parameter neural network with composite physical priori optimization: publisher's note

[This corrects the article on p. 2739 in vol. 13, PMID: 35774326.].

Refractive index tomography with a physics-based optical neural network

The non-interference three-dimensional refractive index (RI) tomography has attracted extensive attention in the life science field for its simple system implementation and robust imaging performance. However, the complexity inherent in the physical propagation process poses significant challenges when the sample under study deviates from the weak scattering approximation. Such conditions complicate the task of achieving global optimization with conventional algorithms, rendering the reconstruction process both time-consuming and potentially ineffective. To address such limitations, this paper proposes an untrained multi-slice neural network (MSNN) with an optical structure, in which each layer has a clear corresponding physical meaning according to the beam propagation model. The network does not require pre-training and performs good generalization and can be recovered through the optimization of a set of intensity images. Concurrently, MSNN can calibrate the intensity of different illumination by learnable parameters, and the multiple backscattering effects have also been taken into consideration by integrating a "scattering attenuation layer" between adjacent "RI" layers in the MSNN. Both simulations and experiments have been conducted carefully to demonstrate the effectiveness and feasibility of the proposed method. Experimental results reveal that MSNN can enhance clarity with increased efficiency in RI tomography. The implementation of MSNN introduces a novel paradigm for RI tomography.

Unsupervised adaptive coded illumination Fourier ptychographic microscopy based on a physical neural network

Fourier Ptychographic Microscopy (FPM) is a computational technique that achieves a large space-bandwidth product imaging. It addresses the challenge of balancing a large field of view and high resolution by fusing information from multiple images taken with varying illumination angles. Nevertheless, conventional FPM framework always suffers from long acquisition time and a heavy computational burden. In this paper, we propose a novel physical neural network that generates an adaptive illumination mode by incorporating temporally-encoded illumination modes as a distinct layer, aiming to improve the acquisition and calculation efficiency. Both simulations and experiments have been conducted to validate the feasibility and effectiveness of the proposed method. It is worth mentioning that, unlike previous works that obtain the intensity of a multiplexed illumination by post-combination of each sequentially illuminated and obtained low-resolution images, our experimental data is captured directly by turning on multiple LEDs with a coded illumination pattern. Our method has exhibited state-of-the-art performance in terms of both detail fidelity and imaging velocity when assessed through a multitude of evaluative aspects.

Hybrid full-pose parameter calibration of a freeform illuminator for Fourier ptychographic microscopy

As a typical computational method, Fourier ptychographic microscopy (FPM) can realize high spatial resolution and quantitative phase imaging while preserving the large field of view with a low numerical aperture (NA) objective. A programmable light-emitting diode (LED) array is used as a typical illuminator in an FPM system, and the illumination parameters of each LED element are crucial to the success of the FPM reconstruction algorithm. Compared with LED arrays arranged in rectangular arrays, LED arrays with special structures such as domes or rings can effectively improve FPM imaging results and imaging efficiency. As a trade-off, their calibration difficulty is greatly increased due to the lack of geometric constraints of rectangular arrays. In this paper, we propose an effective hybrid full-pose parameter calibration method for freeform LED array illuminators, combining stereoscopic 3D imaging techniques and the geometric constraints of the microscopic platform. First, a stereovision system is used to obtain the accurate 3D position of each LED element of the freeform illuminator and to construct a rigid 3D coordinate LED array system. Then, calibration between the coordinate system of the LED array and that of the optical imaging component is realized according to the geometric features of the brightfield-to-darkfield edges. Finally, we verify the feasibility and effectiveness of the proposed method through full-pose parameter calibration of LED arrays with different arrangement rules.

Optical ptychography for biomedical imaging: recent progress and future directions [Invited]

Ptychography is an enabling microscopy technique for both fundamental and applied sciences. In the past decade, it has become an indispensable imaging tool in most X-ray synchrotrons and national laboratories worldwide. However, ptychography's limited resolution and throughput in the visible light regime have prevented its wide adoption in biomedical research. Recent developments in this technique have resolved these issues and offer turnkey solutions for high-throughput optical imaging with minimum hardware modifications. The demonstrated imaging throughput is now greater than that of a high-end whole slide scanner. In this review, we discuss the basic principle of ptychography and summarize the main milestones of its development. Different ptychographic implementations are categorized into four groups based on their lensless/lens-based configurations and coded-illumination/coded-detection operations. We also highlight the related biomedical applications, including digital pathology, drug screening, urinalysis, blood analysis, cytometric analysis, rare cell screening, cell culture monitoring, cell and tissue imaging in 2D and 3D, polarimetric analysis, among others. Ptychography for high-throughput optical imaging, currently in its early stages, will continue to improve in performance and expand in its applications. We conclude this review article by pointing out several directions for its future development.

Sparse phase retrieval using a physics-informed neural network for Fourier ptychographic microscopy

In this paper, we report a sparse phase retrieval framework for Fourier ptychographic microscopy using the recently proposed principle of physics-informed neural networks. The phase retrieval problem is cast as training bidirectional mappings from the measured image space with random noise and the object space to be reconstructed, in which the image formation physics and convolutional neural network are integrated. Meanwhile, we slightly modify the mean absolute error loss function considering the signal characteristics. Two datasets are used to validate this framework. The results indicate that the proposed framework is able to reconstruct sparsely sampled data using a small aperture overlapping rate without additional data driving whereas conventional methods cannot.

Robust full-pose-parameter estimation for the LED array in Fourier ptychographic microscopy

Fourier ptychographic microscopy (FPM) can achieve quantitative phase imaging with a large space-bandwidth product by synthesizing a set of low-resolution intensity images captured under angularly varying illuminations. Determining accurate illumination angles is critical because the consistency between actual systematic parameters and those used in the recovery algorithm is essential for high-quality imaging. This paper presents a full-pose-parameter and physics-based method for calibrating illumination angles. Using a physics-based model constructed with general knowledge of the employed microscope and the brightfield-to-darkfield boundaries inside captured images, we can solve for the full-pose parameters of misplaced LED array, which consist of the distance between the sample and the LED array, two orthogonal lateral shifts, one in-plane rotation angle, and two tilt angles, to correct illumination angles precisely. The feasibility and effectiveness of the proposed method for recovering random or remarkable pose parameters have been demonstrated by both qualitative and quantitative experiments. Due to the completeness of the pose parameters, the clarity of the physical model, and the high robustness for arbitrary misalignments, our method can significantly facilitate the design, implementation, and application of concise and robust FPM platforms.

Implementation of free-space Fourier Ptychography with near maximum system numerical aperture

Over the past decade, the research field of Fourier Ptychographic Microscopy (FPM) has seen numerous innovative developments that significantly expands its utility. Here, we report a high numerical aperture (NA) FPM implementation that incorporates some of these innovations to achieve a synthetic NA of 1.9 - close to the maximum possible synthetic NA of 2 for a free space FPM system. At this high synthetic NA, we experimentally found that it is vital to homogenize the illumination field in order to achieve the best resolution. Our FPM implementation has a full pitch resolution of 266 nm for 465 nm light, and depth of field of 3.6 µm. In comparison, a standard transmission microscope (incoherent) with close to maximum possible NA of 0.95 has a full pitch resolution of 318 nm for 465 nm light, and depth of field of 0.65 µm. While it is generally assumed that a free-space coherent imaging system and a free-space incoherent imaging system operating at their respective maximum NA should give comparable resolution, we experimentally find that an FPM system significantly outperforms its incoherent standard microscopy counterpart in resolution by a factor of 20%. Coupled with FPM's substantially longer effective depth of field (5.5 times longer), our work indicates that, in the near-maximum NA operation regime, the FPM has significant resolution and depth of field advantages over incoherent standard microscopy.