Lensless phase microscopy using phase retrieval with multiple illumination wavelengths

Off-axis digital lensless holographic microscopy based on spatially multiplexed interferometry

Significance

Digital holographic microscopy (DHM) is a label-free microscopy technique that provides time-resolved quantitative phase imaging (QPI) by measuring the optical path delay of light induced by transparent biological samples. DHM has been utilized for various biomedical applications, such as cancer research and sperm cell assessment, as well as for in vitro drug or toxicity testing. Its lensless version, digital lensless holographic microscopy (DLHM), is an emerging technology that offers size-reduced, lightweight, and cost-effective imaging systems. These features make DLHM applicable, for example, in limited resource laboratories, remote areas, and point-of-care applications.

Aim

In addition to the abovementioned advantages, in-line arrangements for DLHM also include the limitation of the twin-image presence, which can restrict accurate QPI. We therefore propose a compact lensless common-path interferometric off-axis approach that is capable of quantitative imaging of fast-moving biological specimens, such as living cells in flow.

Approach

We suggest lensless spatially multiplexed interferometric microscopy (LESSMIM) as a lens-free variant of the previously reported spatially multiplexed interferometric microscopy (SMIM) concept. LESSMIM comprises a common-path interferometric architecture that is based on a single diffraction grating to achieve digital off-axis holography. From a series of single-shot off-axis holograms, twin-image free and time-resolved QPI is achieved by commonly used methods for Fourier filtering-based reconstruction, aberration compensation, and numerical propagation.

Results

Initially, the LESSMIM concept is experimentally demonstrated by results from a resolution test chart and investigations on temporal stability. Then, the accuracy of QPI and capabilities for imaging of living adherent cell cultures is characterized. Finally, utilizing a microfluidic channel, the cytometry of suspended cells in flow is evaluated.

Conclusions

LESSMIM overcomes several limitations of in-line DLHM and provides fast time-resolved QPI in a compact optical arrangement. In summary, LESSMIM represents a promising technique with potential biomedical applications for fast imaging such as in imaging flow cytometry or sperm cell analysis.

Complex amplitude field recovery of a scattering media obstructed object with multi-captured images

An iterative-based method for recovering the complex amplitude field behind scattering media is presented in this Letter. This method compensates the random phase modulation of scattering media by using multiple captured scattered light fields. Complex amplitude reconstruction with local iterative averaging of scattered light fields, and double weighted feedback is efficiently applied. Two feasible types of system setups, with varying detector positions and wavelength, are proposed. Simulations and proof-of-concept experiments are employed to demonstrate the effectiveness of the proposed method in reconstructing complex amplitude of a hidden target.

Multi-Illumination Single-Holographic-Exposure Lensless Fresnel (MISHELF) Microscopy: Principles and Biomedical Applications

Lensless holographic microscopy (LHM) comes out as a promising label-free technique since it supplies high-quality imaging and adaptive magnification in a lens-free, compact and cost-effective way. Compact sizes and reduced prices of LHMs make them a perfect instrument for point-of-care diagnosis and increase their usability in limited-resource laboratories, remote areas, and poor countries. LHM can provide excellent intensity and phase imaging when the twin image is removed. In that sense, multi-illumination single-holographic-exposure lensless Fresnel (MISHELF) microscopy appears as a single-shot and phase-retrieved imaging technique employing multiple illumination/detection channels and a fast-iterative phase-retrieval algorithm. In this contribution, we review MISHELF microscopy through the description of the principles, the analysis of the performance, the presentation of the microscope prototypes and the inclusion of the main biomedical applications reported so far.

Physics-enhanced neural network for phase retrieval from two diffraction patterns

In this work, we propose a physics-enhanced two-to-one Y-neural network (two inputs and one output) for phase retrieval of complex wavefronts from two diffraction patterns. The learnable parameters of the Y-net are optimized by minimizing a hybrid loss function, which evaluates the root-mean-square error and normalized Pearson correlated coefficient on the two diffraction planes. An angular spectrum method network is designed for self-supervised training on the Y-net. Amplitudes and phases of wavefronts diffracted by a USAF-1951 resolution target, a phase grating of 200 lp/mm, and a skeletal muscle cell were retrieved using a Y-net with 100 learning iterations. Fast reconstructions could be realized without constraints or a priori knowledge of the samples.

Quantitative reconstruction of the complex-valued object based on complementary phase modulations

In the coherent diffraction imaging (CDI) techniques, a key point is to reconstruct the complex-valued object from the far-field intensity measurements, i.e., solving the phase retrieval problem. However, due to this ill-posed problem, traditional phase retrieval algorithms often encounter some problems associated with the iteration convergence. In this work, complementary phase modulations (CPM) are introduced to generate different far-field intensity measurements. The namely CPM-based method aims to find out the global optimal solution by imposing multi-dimensional constraints, including the diverse intensity images at the Fourier plane and the CPM at the object plane. It is proved by the numerical simulations and the optical experiments that the convergence speed and the recovery accuracy could be greatly improved. Furthermore, the shifting complementary phase modulations (SCPM)-based method is proposed by introducing more CPMs. The reconstruction performance is further improved even when the phase range is larger, and the support constraints are not required. In addition, the SCPM-based method is more robust to the Poisson noise. With the outstanding reconstruction performance, the CPM-based methods may be helpful to phase imaging in the application of visible-light microscopy and X-ray imaging.

Holographic optical field recovery using a regularized untrained deep decoder network

Image reconstruction using minimal measured information has been a long-standing open problem in many computational imaging approaches, in particular in-line holography. Many solutions are devised based on compressive sensing (CS) techniques with handcrafted image priors or supervised deep neural networks (DNN). However, the limited performance of CS methods due to lack of information about the image priors and the requirement of an enormous amount of per-sample-type training resources for DNNs has posed new challenges over the primary problem. In this study, we propose a single-shot lensless in-line holographic reconstruction method using an untrained deep neural network which is incorporated with a physical image formation algorithm. We demonstrate that by modifying a deep decoder network with simple regularizers, a Gabor hologram can be inversely reconstructed via a minimization process that is constrained by a deep image prior. The outcoming model allows to accurately recover the phase and amplitude images without any training dataset, excess measurements, or specific assumptions about the object's or the measurement's characteristics.

High-resolution multi-wavelength lensless diffraction imaging with adaptive dispersion correction

Multi-wavelength imaging diffraction system is a promising phase imaging technology due to its advantages of no mechanical movement and low complexity. In a multi-wavelength focused system, spectral bandwidth and dispersion correction are critical for high resolution reconstruction. Here, an optical setup for the multi-wavelength lensless diffraction imaging system with adaptive dispersion correction is proposed. Three beams with different wavelengths are adopted to illuminate the test object, and then the diffraction patterns are recorded by a image sensor. The chromatic correction is successfully realized by a robust refocusing technique. High-resolution images can be finally retrieved through phase retrieval algorithm. The effectiveness and reliability of our method is demonstrated in numerical simulation and experiments. The proposed method has the potential to be an alternative technology for quantitative biological imaging.

Lensless hyperspectral phase imaging in a self-reference setup based on Fourier transform spectroscopy and noise suppression

A novel phase retrieval algorithm for broadband hyperspectral phase imaging from noisy intensity observations is proposed. It utilizes advantages of the Fourier transform spectroscopy in the self-referencing optical setup and provides additional, beyond spectral intensity distribution, reconstruction of the investigated object's phase. The noise amplification Fellgett's disadvantage is relaxed by the application of a sparse wavefront noise filtering embedded in the proposed algorithm. The algorithm reliability is proved by simulation tests and by results of physical experiments for transparent objects. These tests demonstrate precise phase imaging and object depth (profile) reconstruction.

Pixel super-resolved lens-free on-chip microscopy based on dual laterally shifting modulation

Achieving high spatial resolution over a wide field of view (FOV) is the goal of many imaging systems. In a traditional lens-based microscope, designing a complex objective with high numerical aperture (NA) to achieve this goal is a tough and challenging task. The lens-free wide-field imaging method based on phase retrieval provides a new way to bypass the trade-off between the spatial resolution and FOV of conventional microscopy. However, the typical lens-free microscopy usually requires mechanical devices with high precision and repeatability. In this paper, we report a robust and cost-effective pixel super-resolved lens-free imaging method based on dual laterally shifting modulation. A thin diffuser is inserted between the object and the image sensor to be used as the modulator. The diffuser and the object are transversely scanned at the same time to add diversities for phase retrieval and pixel super-resolution, respectively. In this way, the positional shifts of the diffuser and the object can be directly recovered with the registration algorithm, thus addressing the low stability and inaccuracy issues of translation stages. We also propose a pixel super-resolution phase-retrieval algorithm to recover the object and the unknown diffuser. We first use numerical simulations to evaluate the proposed scheme. Then, we validate this approach by imaging a resolution target and a pollen sample, thus achieving an FOV of ${\sim}{30}\;{\text{mm}^2}$∼30mm² and a half-pitch resolution of 0.78 µm, which surpasses 2.14 times the theoretical Nyquist-Shannon sampling resolution limit. Finally, the 3D refocusing ability is also verified by imaging a thick mosquito sample.

New methods of removing debris and high-throughput counting of cyst nematode eggs extracted from field soil

The soybean cyst nematode (SCN), Heterodera glycines, is the most damaging pathogen of soybeans in the United States. To assess the severity of nematode infestations in the field, SCN egg population densities are determined. Cysts (dead females) of the nematode must be extracted from soil samples and then ground to extract the eggs within. Sucrose centrifugation commonly is used to separate debris from suspensions of extracted nematode eggs. We present a method using OptiPrep as a density gradient medium with improved separation and recovery of extracted eggs compared to the sucrose centrifugation technique. Also, computerized methods were developed to automate the identification and counting of nematode eggs from the processed samples. In one approach, a high-resolution scanner was used to take static images of extracted eggs and debris on filter papers, and a deep learning network was trained to identify and count the eggs among the debris. In the second approach, a lensless imaging setup was developed using off-the-shelf components, and the processed egg samples were passed through a microfluidic flow chip made from double-sided adhesive tape. Holographic videos were recorded of the passing eggs and debris, and the videos were reconstructed and processed by custom software program to obtain egg counts. The performance of the software programs for egg counting was characterized with SCN-infested soil collected from two farms, and the results using these methods were compared with those obtained through manual counting.

Robust autofocusing method for multi-wavelength lensless imaging

Lensless imaging based on multi-wavelength phase retrieval becomes a promising technology widely used as it has simple acquisition, miniaturized size and low-cost setup. However, measuring the sample-to-sensor distance with high accuracy, which is the key for high-resolution reconstruction, is still a challenge. In this work, we propose a multi-wavelength criterion to realize autofocusing modulation, i.e., achieving much higher accuracy in determining the sample-to-sensor distance, compared to the conventional methods. Three beams in different spectrums are adopted to illuminate the sample, and the resulting holograms are recorded by a CCD camera. The patterns calculated by performing back propagation of the recorded holograms, with exhaustively searched sample-to-sensor distance value, are adopted to access the criterion. Image sharpness can be accessed and the optimal sample-to-sensor distance can be finely determined by targeting the valley of the curve given by the criterion. Through our novel multi-wavelength based autofocusing strategy and executing further phase retrieval process, high-resolution images can be finally retrieved. The applicability and robustness of our method is validated both in simulations and experiments. Our technique provides a useful tool for multi-wavelength lensless imaging under limited experimental conditions.

Object plane detection and phase retrieval from single-shot holograms using multi-wavelength in-line holography

Phase retrieval and the twin-image problem in digital in-line holographic microscopy can be resolved by iterative reconstruction routines. However, recovering the phase properties of an object in a hologram requires an object plane to be chosen correctly for reconstruction. In this work, we present a novel multi-wavelength iterative algorithm to determine the object plane using single-shot holograms recorded at multiple wavelengths in an in-line holographic microscope. Using micro-sized objects, we verify the object positioning capabilities of the method for various shapes and derive the phase information using synthetic and experimental data. Experimentally, we built a compact digital in-line holographic microscopy setup around a standard optical microscope with a regular RGB-CCD camera and acquired holograms of micro-spheres, E. coli, and red blood cells, which are illuminated using three lasers operating at 491 nm, 532 nm, and 633 nm, respectively. We demonstrate that our method provides accurate object plane detection and phase retrieval under noisy conditions, e.g., using low-contrast holograms with an inhomogeneous background. This method allows for automatic positioning and phase retrieval suitable for holographic particle velocimetry, and object tracking in biophysical or colloidal research.

Lensless imaging through multiple phase patterns illumination

A stable optical system is required to acquire a high-quality image. A motionless lensless setup is designated to obtain high-resolution and large field of view images. The sample is sequentially illuminated with multiple random phase patterns, and the recorded images are subtracted from the system calibration images correspondingly. The resultant images are propagated to the sample plane. The summation of all images yields a final image with resolution of ∼4  μm, field of view of ∼15  mm2, and better signal-to-noise ratio. This technique provides a compact, stable, and cost-effective optical system.

Scatter-plate microscope for lensless microscopy with diffraction limited resolution

Scattering media have always been looked upon as an obstacle in imaging. Various methods, ranging from holography to phase compensation as well as to correlation techniques, have been proposed to cope with this obstacle. We, on the other hand, have a different understanding about the role of the diffusing media. In this paper we propose and demonstrate a ‘scatter-plate microscope’ that utilizes the diffusing property of the random medium for imaging micro structures with diffraction-limited resolution. The ubiquitous property of the speckle patterns permits to exploit the scattering medium as an ultra-thin lensless microscope objective with a variable focal length and a large working distance. The method provides a light, flexible and cost effective imaging device as an alternative to conventional microscope objectives. In principle, the technique is also applicable to lensless imaging in UV and X-ray microscopy. Experiments were performed with visible light to demonstrate the microscopic imaging of USAF resolution test target and a biological sample with varying numerical aperture (NA) and magnifications.

Real-time acquisition of complex optical fields by binary amplitude modulation

We describe, through simulations and experiments, a real-time wavefront acquisition technique using random binary amplitude masks and an iterative phase retrieval algorithm based on the Fresnel propagator. By using a digital micromirror device, it is possible to recover an unknown complex object by illuminating with this set of masks and simultaneously recording the resulting intensity patterns with a high-speed camera, making this technique suitable for dynamic applications.

Compact, cost-effective and field-portable microscope prototype based on MISHELF microscopy

We report on a reduced cost, portable and compact prototype design of lensless holographic microscope with an illumination/detection scheme based on wavelength multiplexing, working with single hologram acquisition and using a fast convergence algorithm for image processing. All together, MISHELF (initials coming from Multi-Illumination Single-Holographic-Exposure Lensless Fresnel) microscopy allows the recording of three Fresnel domain diffraction patterns in a single camera snap-shot incoming from illuminating the sample with three coherent lights at once. Previous implementations have proposed an illumination/detection procedure based on a tuned (illumination wavelengths centered at the maximum sensitivity of the camera detection channels) configuration but here we report on a detuned (non-centered ones) scheme resulting in prototype miniaturization and cost reduction. Thus, MISHELF microscopy in combination with a novel and fast iterative algorithm allows high-resolution (μm range) phase-retrieved (twin image elimination) quantitative phase imaging of dynamic events (video rate recording speed). The performance of this microscope prototype is validated through experiments using both amplitude (USAF resolution test) and complex (live swine sperm cells and flowing microbeads) samples. The proposed method becomes in an alternative instrument improving some capabilities of existing lensless microscopes.

Sparsity-based multi-height phase recovery in holographic microscopy

High-resolution imaging of densely connected samples such as pathology slides using digital in-line holographic microscopy requires the acquisition of several holograms, e.g., at >6–8 different sample-to-sensor distances, to achieve robust phase recovery and coherent imaging of specimen. Reducing the number of these holographic measurements would normally result in reconstruction artifacts and loss of image quality, which would be detrimental especially for biomedical and diagnostics-related applications. Inspired by the fact that most natural images are sparse in some domain, here we introduce a sparsity-based phase reconstruction technique implemented in wavelet domain to achieve at least 2-fold reduction in the number of holographic measurements for coherent imaging of densely connected samples with minimal impact on the reconstructed image quality, quantified using a structural similarity index. We demonstrated the success of this approach by imaging Papanicolaou smears and breast cancer tissue slides over a large field-of-view of ~20 mm² using 2 in-line holograms that are acquired at different sample-to-sensor distances and processed using sparsity-based multi-height phase recovery. This new phase recovery approach that makes use of sparsity can also be extended to other coherent imaging schemes, involving e.g., multiple illumination angles or wavelengths to increase the throughput and speed of coherent imaging.

Pixel super-resolution using wavelength scanning

Undersampling and pixelation affect a number of imaging systems, limiting the resolution of the acquired images, which becomes particularly significant for wide-field microscopy applications. Various super-resolution techniques have been implemented to mitigate this resolution loss by utilizing sub-pixel displacements in the imaging system, achieved, for example, by shifting the illumination source, the sensor array and/or the sample, followed by digital synthesis of a smaller effective pixel by merging these sub-pixel-shifted low-resolution images. Herein, we introduce a new pixel super-resolution method that is based on wavelength scanning and demonstrate that as an alternative to physical shifting/displacements, wavelength diversity can be used to boost the resolution of a wide-field imaging system and significantly increase its space-bandwidth product. We confirmed the effectiveness of this new technique by improving the resolution of lens-free as well as lens-based microscopy systems and developed an iterative algorithm to generate high-resolution reconstructions of a specimen using undersampled diffraction patterns recorded at a few wavelengths covering a narrow spectrum (10-30 nm). When combined with a synthetic-aperture-based diffraction imaging technique, this wavelength-scanning super-resolution approach can achieve a half-pitch resolution of 250 nm, corresponding to a numerical aperture of ~1.0, across a large field of view (>20 mm²). We also demonstrated the effectiveness of this approach by imaging various biological samples, including blood and Papanicolaou smears. Compared with displacement-based super-resolution techniques, wavelength scanning brings uniform resolution improvement in all directions across a sensor array and requires significantly fewer measurements. This technique would broadly benefit wide-field imaging applications that demand larger space-bandwidth products.

Propagation phasor approach for holographic image reconstruction

To achieve high-resolution and wide field-of-view, digital holographic imaging techniques need to tackle two major challenges: phase recovery and spatial undersampling. Previously, these challenges were separately addressed using phase retrieval and pixel super-resolution algorithms, which utilize the diversity of different imaging parameters. Although existing holographic imaging methods can achieve large space-bandwidth-products by performing pixel super-resolution and phase retrieval sequentially, they require large amounts of data, which might be a limitation in high-speed or cost-effective imaging applications. Here we report a propagation phasor approach, which for the first time combines phase retrieval and pixel super-resolution into a unified mathematical framework and enables the synthesis of new holographic image reconstruction methods with significantly improved data efficiency. In this approach, twin image and spatial aliasing signals, along with other digital artifacts, are interpreted as noise terms that are modulated by phasors that analytically depend on the lateral displacement between hologram and sensor planes, sample-to-sensor distance, wavelength, and the illumination angle. Compared to previous holographic reconstruction techniques, this new framework results in five- to seven-fold reduced number of raw measurements, while still achieving a competitive resolution and space-bandwidth-product. We also demonstrated the success of this approach by imaging biological specimens including Papanicolaou and blood smears.

Complex wavefront reconstruction from multiple-image planes produced by a focus tunable lens

We propose, through simulations and experiments, a wavefront reconstruction technique using a focus-tunable lens and a phase-retrieval technique. A collimated beam illuminates a complex object (amplitude and phase), and a diffuser then modulates the outgoing wavefront. Finally the diffracted complex field reaches the focus-tunable lens, and a CMOS camera positioned at a fixed plane registers the subjective speckle distribution produced by the lens (one pattern for each focal length). We have demonstrated that a tunable lens can replace the translation stage used in the conventional single-beam, multiple-intensity reconstruction algorithm. In other words, through iterations with a modified version of this algorithm, the speckle images produced by different focal lengths can be successfully employed to recover the initial complex object. With no movable elements, (speckle) image sampling can be performed at high frame rates, which is suitable for dynamical reconstruction applications.

Improved quantitative phase imaging in lensless microscopy by single-shot multi-wavelength illumination using a fast convergence algorithm

We report on a novel algorithm for high-resolution quantitative phase imaging in a new concept of lensless holographic microscope based on single-shot multi-wavelength illumination. This new microscope layout, reported by Noom et al. along the past year and named by us as MISHELF (initials incoming from Multi-Illumination Single-Holographic-Exposure Lensless Fresnel) microscopy, rises from the simultaneous illumination and recording of multiple diffraction patterns in the Fresnel domain. In combination with a novel and fast iterative phase retrieval algorithm, MISHELF microscopy is capable of high-resolution (micron range) phase-retrieved (twin image elimination) biological imaging of dynamic events. In this contribution, MISHELF microscopy is demonstrated through qualitative concept description, algorithm implementation, and experimental validation using both a synthetic object (resolution test target) and a biological sample (swine sperm sample) for the case of three (RGB) illumination wavelengths. The proposed method becomes in an alternative instrument improving the capabilities of existing lensless microscopes.

Exploring neural cell dynamics with digital holographic microscopy

In this review, we summarize how the new concept of digital optics applied to the field of holographic microscopy has allowed the development of a reliable and flexible digital holographic quantitative phase microscopy (DH-QPM) technique at the nanoscale particularly suitable for cell imaging. Particular emphasis is placed on the original biological information provided by the quantitative phase signal. We present the most relevant DH-QPM applications in the field of cell biology, including automated cell counts, recognition, classification, three-dimensional tracking, discrimination between physiological and pathophysiological states, and the study of cell membrane fluctuations at the nanoscale. In the last part, original results show how DH-QPM can address two important issues in the field of neurobiology, namely, multiple-site optical recording of neuronal activity and noninvasive visualization of dendritic spine dynamics resulting from a full digital holographic microscopy tomographic approach.