Overlapped Fourier coding for optical aberration removal

Multi-wavelength interference phase imaging for automatic breast cancer detection and delineation using diffuse reflection imaging

Millions of women globally are impacted by the major health problem of breast cancer (BC). Early detection of BC is critical for successful treatment and improved survival rates. In this study, we provide a progressive approach for BC detection using multi-wavelength interference (MWI) phase imaging based on diffuse reflection hyperspectral (HS) imaging. The proposed findings are based on the measurement of the interference pattern between the blue (446.6 nm) and red (632 nm) wavelengths. We consider implementing a comprehensive image processing and categorization method based on the use of Fast Fourier (FF) transform analysis pertaining to a change in the refractive index between tumor and normal tissue. We observed that cancer growth affects tissue organization dramatically, as seen by persistently increased refractive index variance in tumors compared normal areas. Both malignant and normal tissue had different depth data collected from it that was analyzed. To enhance the categorization of ex-vivo BC tissue, we developed and validated a training classifier algorithm specifically designed for categorizing HS cube data. Following the application of signal normalization with the FF transform algorithm, our methodology achieved a high level of performance with a specificity (Spec) of 94% and a sensitivity (Sen) of 90.9% for the 632 nm acquired image categorization, based on preliminary findings from breast specimens under investigation. Notably, we successfully leveraged unstained tissue samples to create 3D phase-resolved images that effectively highlight the distinctions in diffuse reflectance features between cancerous and healthy tissue. Preliminary data revealed that our imaging method might be able to assist specialists in safely excising malignant areas and assessing the tumor bed following resection automatically at different depths. This preliminary investigation might result in an effective "in-vivo" disease description utilizing optical technology using a typical RGB camera with wavelength-specific operation with our quantitative phase MWI imaging methodology.

Optical ptychography for biomedical imaging: recent progress and future directions [Invited]

Ptychography is an enabling microscopy technique for both fundamental and applied sciences. In the past decade, it has become an indispensable imaging tool in most X-ray synchrotrons and national laboratories worldwide. However, ptychography's limited resolution and throughput in the visible light regime have prevented its wide adoption in biomedical research. Recent developments in this technique have resolved these issues and offer turnkey solutions for high-throughput optical imaging with minimum hardware modifications. The demonstrated imaging throughput is now greater than that of a high-end whole slide scanner. In this review, we discuss the basic principle of ptychography and summarize the main milestones of its development. Different ptychographic implementations are categorized into four groups based on their lensless/lens-based configurations and coded-illumination/coded-detection operations. We also highlight the related biomedical applications, including digital pathology, drug screening, urinalysis, blood analysis, cytometric analysis, rare cell screening, cell culture monitoring, cell and tissue imaging in 2D and 3D, polarimetric analysis, among others. Ptychography for high-throughput optical imaging, currently in its early stages, will continue to improve in performance and expand in its applications. We conclude this review article by pointing out several directions for its future development.

Single-shot blind deconvolution with coded aperture

In this paper, we present a method for single-shot blind deconvolution incorporating a coded aperture (CA). In this method, we utilize the CA, inserted on the pupil plane, as support constraints in blind deconvolution. Not only an object is estimated, but also a point spread function of turbulence from a single captured image by a reconstruction algorithm with CA support. The proposed method is demonstrated by simulation and an experiment in which point sources are recovered under severe turbulence.

Pupil-aberration calibration with controlled illumination for quantitative phase imaging

Quantitative phase imaging (QPI) exploits sample-induced changes in the optical field to analyze biological specimens in a label-free manner. However, the quantitative nature of QPI makes it susceptible to optical aberrations. We propose a method for calibrating pupil aberrations by imaging a sample of interest. The proposed method recovers pupil information by utilizing the cross-spectral density between optical fields at different incident angles and allows both thin and weakly scattering three-dimensional samples for calibration. We experimentally validate the proposed method by imaging various samples, including a resolution target, breast tissue, and a polystyrene bead, and demonstrate aberration-free two- and three-dimensional QPI.

Review of bio-optical imaging systems with a high space-bandwidth product

Optical imaging has served as a primary method to collect information about biosystems across scales-from functionalities of tissues to morphological structures of cells and even at biomolecular levels. However, to adequately characterize a complex biosystem, an imaging system with a number of resolvable points, referred to as a space-bandwidth product (SBP), in excess of one billion is typically needed. Since a gigapixel-scale far exceeds the capacity of current optical imagers, compromises must be made to obtain either a low spatial resolution or a narrow field-of-view (FOV). The problem originates from constituent refractive optics-the larger the aperture, the more challenging the correction of lens aberrations. Therefore, it is impractical for a conventional optical imaging system to achieve an SBP over hundreds of millions. To address this unmet need, a variety of high-SBP imagers have emerged over the past decade, enabling an unprecedented resolution and FOV beyond the limit of conventional optics. We provide a comprehensive survey of high-SBP imaging techniques, exploring their underlying principles and applications in bioimaging.

Fourier ptychography: current applications and future promises

Traditional imaging systems exhibit a well-known trade-off between the resolution and the field of view of their captured images. Typical cameras and microscopes can either "zoom in" and image at high-resolution, or they can "zoom out" to see a larger area at lower resolution, but can rarely achieve both effects simultaneously. In this review, we present details about a relatively new procedure termed Fourier ptychography (FP), which addresses the above trade-off to produce gigapixel-scale images without requiring any moving parts. To accomplish this, FP captures multiple low-resolution, large field-of-view images and computationally combines them in the Fourier domain into a high-resolution, large field-of-view result. Here, we present details about the various implementations of FP and highlight its demonstrated advantages to date, such as aberration recovery, phase imaging, and 3D tomographic reconstruction, to name a few. After providing some basics about FP, we list important details for successful experimental implementation, discuss its relationship with other computational imaging techniques, and point to the latest advances in the field while highlighting persisting challenges.

Computational aberration correction of VIS-NIR multispectral imaging microscopy based on Fourier ptychography

Due to the chromatic dispersion properties inherent in all optical materials, even the best-designed multispectral objective will exhibit residual chromatic aberration. Here, we demonstrate a multispectral microscope with a computational scheme based on the Fourier ptychographic microscopy (FPM) to correct these effects in order to render undistorted, in-focus images. The microscope consists of 4 spectral channels ranging from 405 nm to 1552 nm. After the computational aberration correction, it can achieve isotropic resolution enhancement as verified with the Siemens star sample. We image a flip-chip to show the promise of our system to conduct fault detection on silicon chips. This computational approach provides a cost-efficient strategy for high quality multispectral imaging over a broad spectral range.

Parallel Fourier ptychographic microscopy for high-throughput screening with 96 cameras (96 Eyes)

We report the implementation of a parallel microscopy system (96 Eyes) that is capable of simultaneous imaging of all wells on a 96-well plate. The optical system consists of 96 microscopy units, where each unit is made out of a four element objective, made through a molded injection process, and a low cost CMOS camera chip. By illuminating the sample with angle varying light and applying Fourier Ptychography, we can improve the effective brightfield imaging numerical aperture of the objectives from 0.23 to 0.3, and extend the depth of field from ±5 μm to ±15 μm. The use of Fourier Ptychography additionally allows us to computationally correct the objectives’ aberrations out of the rendered images, and provides us with the ability to render phase images. The 96 Eyes acquires raw data at a rate of 0.7 frame per second (all wells) and the data are processed with 4 cores of graphical processing units (GPUs; GK210, Nvidia Tesla K80, USA). The system is also capable of fluorescence imaging (excitation = 465 nm, emission = 510 nm) at the native resolution of the objectives. We demonstrate the capability of this system by imaging S1P₁-eGFP-Human bone osteosarcoma epithelial (U2OS) cells.

Phase and amplitude imaging with quantum correlations through Fourier Ptychography

Extracting as much information as possible about an object when probing with a limited number of photons is an important goal with applications from biology and security to metrology. Imaging with a few photons is a challenging task as the detector noise and stray light are then predominant, which precludes the use of conventional imaging methods. Quantum correlations between photon pairs has been exploited in a so called ‘heralded imaging scheme’ to eliminate this problem. However these implementations have so-far been limited to intensity imaging and the crucial phase information is lost in these methods. In this work, we propose a novel quantum-correlation enabled Fourier Ptychography technique, to capture high-resolution amplitude and phase images with a few photons. This is enabled by the heralding of single photons combined with Fourier ptychographic reconstruction. We provide experimental validation and discuss the advantages of our technique that include the possibility of reaching a higher signal to noise ratio and non-scanning Fourier Ptychographic acquisition.

Computational aberration compensation by coded-aperture-based correction of aberration obtained from optical Fourier coding and blur estimation

We report a novel generalized optical measurement system and computational approach to determine and correct aberrations in optical systems. The system consists of a computational imaging method capable of reconstructing an optical system's pupil function by adapting overlapped Fourier coding to an incoherent imaging modality. It recovers the high-resolution image latent in an aberrated image via deconvolution. The deconvolution is made robust to noise by using coded apertures to capture images. We term this method coded-aperture-based correction of aberration obtained from overlapped Fourier coding and blur estimation (CACAO-FB). It is well-suited for various imaging scenarios where aberration is present and where providing a spatially coherent illumination is very challenging or impossible. We report the demonstration of CACAO-FB with a variety of samples including an in vivo imaging experiment on the eye of a rhesus macaque to correct for its inherent aberration in the rendered retinal images. CACAO-FB ultimately allows for an aberrated imaging system to achieve diffraction-limited performance over a wide field of view by casting optical design complexity to computational algorithms in post-processing.

Convex optimization-based blind deconvolution for images taken with coherent illumination

A rank-constrained reformulation of the blind deconvolution problem on images taken with coherent illumination is proposed. Since in the reformulation the rank constraint is imposed on a matrix that is affine in the decision variables, we propose a novel convex heuristic for the blind deconvolution problem. The proposed heuristic allows for easy incorporation of prior information on the decision variables and the use of the phase diversity concept. The convex optimization problem can be iteratively re-parameterized to obtain better estimates. The proposed methods are demonstrated on numerically illustrative examples.

Quantitative differential phase contrast (DPC) microscopy with computational aberration correction

Differential Phase Contrast (DPC) microscopy is a practical method for recovering quantitative phase from intensity images captured with different source patterns in an LED array microscope. Being a partially coherent imaging method, DPC does not suffer from speckle artifacts and achieves 2× better resolution than coherent methods. Like all imaging systems, however, DPC is susceptible to aberrations. Here, we propose a method of algorithmic self-calibration for DPC where we simultaneously recover the complex-field of the sample and the spatially-variant aberrations of the system, using 4 images with different illumination source patterns. The resulting phase reconstructions are digitally aberration-corrected.

Single-shot aperture-scanning Fourier ptychography

Aperture-scanning Fourier ptychography [Opt. Express22, 13586 (2014)] is a promising non-interferometric wavefront measurement technique. It eliminates the thin-sample requirement in typical Fourier ptychography employing angle-varying illumination. However, as aperture-scanning Fourier ptychography is based on step-by-step scanning, it requires long data acquisition time and a high-stability optical system. In this paper, we propose a single-shot aperture-scanning Fourier ptychography method. In our method, multiple low-resolution images are collected in a single shot by inserting a Dammann grating at a certain distance before the aperture, and the images are subsequently converted to a high-resolution complex wavefront. Compared with scanning-based aperture-scanning Fourier ptychography, the total acquisition time of the proposed method is dramatically reduced. The feasibility of our proposed method is demonstrated by proof-of-concept experiments.

Efficient illumination angle self-calibration in Fourier ptychography

Fourier ptychography captures intensity images with varying source patterns (illumination angles) in order to computationally reconstruct large space-bandwidth-product images. Accurate knowledge of the illumination angles is necessary for good image quality; hence, calibration methods are crucial, despite often being impractical or slow. Here, we propose a fast, robust, and accurate self-calibration algorithm that uses only experimentally collected data and general knowledge of the illumination setup. First, our algorithm makes a fast direct estimate of the brightfield illumination angles based on image processing. Then, a more computationally intensive spectral correlation method is used inside the iterative solver to further refine the angle estimates of both brightfield and darkfield images. We demonstrate our method for correcting large and small misalignment artifacts in 2D and 3D Fourier ptychography with different source types: an LED array, a galvo-steered laser, and a high-NA quasi-dome LED illuminator.

SAVI: Synthetic apertures for long-range, subdiffraction-limited visible imaging using Fourier ptychography

Synthetic aperture radar is a well-known technique for improving resolution in radio imaging. Extending these synthetic aperture techniques to the visible light domain is not straightforward because optical receivers cannot measure phase information. We propose to use macroscopic Fourier ptychography (FP) as a practical means of creating a synthetic aperture for visible imaging to achieve subdiffraction-limited resolution. We demonstrate the first working prototype for macroscopic FP in a reflection imaging geometry that is capable of imaging optically rough objects. In addition, a novel image space denoising regularization is introduced during phase retrieval to reduce the effects of speckle and improve perceptual quality of the recovered high-resolution image. Our approach is validated experimentally where the resolution of various diffuse objects is improved sixfold.

Wide-field Fourier ptychographic microscopy using laser illumination source

Fourier ptychographic (FP) microscopy is a coherent imaging method that can synthesize an image with a higher bandwidth using multiple low-bandwidth images captured at different spatial frequency regions. The method's demand for multiple images drives the need for a brighter illumination scheme and a high-frame-rate camera for a faster acquisition. We report the use of a guided laser beam as an illumination source for an FP microscope. It uses a mirror array and a 2-dimensional scanning Galvo mirror system to provide a sample with plane-wave illuminations at diverse incidence angles. The use of a laser presents speckles in the image capturing process due to reflections between glass surfaces in the system. They appear as slowly varying background fluctuations in the final reconstructed image. We are able to mitigate these artifacts by including a phase image obtained by differential phase contrast (DPC) deconvolution in the FP algorithm. We use a 1-Watt laser configured to provide a collimated beam with 150 mW of power and beam diameter of 1 cm to allow for the total capturing time of 0.96 seconds for 96 raw FPM input images in our system, with the camera sensor's frame rate being the bottleneck for speed. We demonstrate a factor of 4 resolution improvement using a 0.1 NA objective lens over the full camera field-of-view of 2.7 mm by 1.5 mm.

Quantitative phase imaging and complex field reconstruction by pupil modulation differential phase contrast

Differential phase contrast (DPC) is a non-interferometric quantitative phase imaging method achieved by using an asymmetric imaging procedure. We report a pupil modulation differential phase contrast (PMDPC) imaging method by filtering a sample's Fourier domain with half-circle pupils. A phase gradient image is captured with each half-circle pupil, and a quantitative high resolution phase image is obtained after a deconvolution process with a minimum of two phase gradient images. Here, we introduce PMDPC quantitative phase image reconstruction algorithm and realize it experimentally in a 4f system with an SLM placed at the pupil plane. In our current experimental setup with the numerical aperture of 0.36, we obtain a quantitative phase image with a resolution of 1.73μm after computationally removing system aberrations and refocusing. We also extend the depth of field digitally by 20 times to ±50μm with a resolution of 1.76μm.

Aperture scanning Fourier ptychographic microscopy

Fourier ptychographic microscopy (FPM) is implemented through aperture scanning by an LCOS spatial light modulator at the back focal plane of the objective lens. This FPM configuration enables the capturing of the complex scattered field for a 3D sample both in the transmissive mode and the reflective mode. We further show that by combining with the compressive sensing theory, the reconstructed 2D complex scattered field can be used to recover the 3D sample scattering density. This implementation expands the scope of application for FPM and can be beneficial for areas such as tissue imaging and wafer inspection.

Ptychography for optical metrology with limited translation knowledge

We introduce unknown-transverse translation diversity phase retrieval: a ptychographic algorithm for optical metrology when a subaperture is translating through a plane conjugate to the exit pupil in a very poorly known fashion. The algorithm estimates the direction of translation and the distance traveled by the subaperture from one point spread function (PSF) to the next. It also estimates unknown point target motion and rotations of the subaperture between PSF acquisitions from the PSF data.

Experimental robustness of Fourier ptychography phase retrieval algorithms

Fourier ptychography is a new computational microscopy technique that provides gigapixel-scale intensity and phase images with both wide field-of-view and high resolution. By capturing a stack of low-resolution images under different illumination angles, an inverse algorithm can be used to computationally reconstruct the high-resolution complex field. Here, we compare and classify multiple proposed inverse algorithms in terms of experimental robustness. We find that the main sources of error are noise, aberrations and mis-calibration (i.e. model mis-match). Using simulations and experiments, we demonstrate that the choice of cost function plays a critical role, with amplitude-based cost functions performing better than intensity-based ones. The reason for this is that Fourier ptychography datasets consist of images from both brightfield and darkfield illumination, representing a large range of measured intensities. Both noise (e.g. Poisson noise) and model mis-match errors are shown to scale with intensity. Hence, algorithms that use an appropriate cost function will be more tolerant to both noise and model mis-match. Given these insights, we propose a global Newton's method algorithm which is robust and accurate. Finally, we discuss the impact of procedures for algorithmic correction of aberrations and mis-calibration.

Solving ptychography with a convex relaxation

Ptychography is a powerful computational imaging technique that transforms a collection of low-resolution images into a high-resolution sample reconstruction. Unfortunately, algorithms that currently solve this reconstruction problem lack stability, robustness, and theoretical guarantees. Recently, convex optimization algorithms have improved the accuracy and reliability of several related reconstruction efforts. This paper proposes a convex formulation of the ptychography problem. This formulation has no local minima, it can be solved using a wide range of algorithms, it can incorporate appropriate noise models, and it can include multiple a priori constraints. The paper considers a specific algorithm, based on low-rank factorization, whose runtime and memory usage are near-linear in the size of the output image. Experiments demonstrate that this approach offers a 25% lower background variance on average than alternating projections, the ptychographic reconstruction algorithm that is currently in widespread use.

Microscopy illumination engineering using a low-cost liquid crystal display

Illumination engineering is critical for obtaining high-resolution, high-quality images in microscope settings. In a typical microscope, the condenser lens provides sample illumination that is uniform and free from glare. The associated condenser diaphragm can be manually adjusted to obtain the optimal illumination numerical aperture. In this paper, we report a programmable condenser lens for active illumination control. In our prototype setup, we used a $15 liquid crystal display as a transparent spatial light modulator and placed it at the back focal plane of the condenser lens. By setting different binary patterns on the display, we can actively control the illumination and the spatial coherence of the microscope platform. We demonstrated the use of such a simple scheme for multimodal imaging, including bright-field microscopy, darkfield microscopy, phase-contrast microscopy, polarization microscopy, 3D tomographic imaging, and super-resolution Fourier ptychographic imaging. The reported illumination engineering scheme is cost-effective and compatible with most existing platforms. It enables a turnkey solution with high flexibility for researchers in various communities. From the engineering point-of-view, the reported illumination scheme may also provide new insights for the development of multimodal microscopy and Fourier ptychographic imaging.