Computational structured illumination for high-content fluorescence and phase microscopy

Polarization differential interference contrast microscopy with physics-inspired plug-and-play denoiser for single-shot high-performance quantitative phase imaging

We present single-shot high-performance quantitative phase imaging with a physics-inspired plug-and-play denoiser for polarization differential interference contrast (PDIC) microscopy. The quantitative phase is recovered by the alternating direction method of multipliers (ADMM), balancing total variance regularization and a pre-trained dense residual U-net (DRUNet) denoiser. The custom DRUNet uses the Tanh activation function to guarantee the symmetry requirement for phase retrieval. In addition, we introduce an adaptive strategy accelerating convergence and explicitly incorporating measurement noise. After validating this deep denoiser-enhanced PDIC microscopy on simulated data and phantom experiments, we demonstrated high-performance phase imaging of histological tissue sections. The phase retrieval by the denoiser-enhanced PDIC microscopy achieves significantly higher quality and accuracy than the solution based on Fourier transforms or the iterative solution with total variance regularization alone.

Spatial- and Fourier-domain ptychography for high-throughput bio-imaging

First envisioned for determining crystalline structures, ptychography has become a useful imaging tool for microscopists. However, ptychography remains underused by biomedical researchers due to its limited resolution and throughput in the visible light regime. Recent developments of spatial- and Fourier-domain ptychography have successfully addressed these issues and now offer the potential for high-resolution, high-throughput optical imaging with minimal hardware modifications to existing microscopy setups, often providing an excellent trade-off between resolution and field of view inherent to conventional imaging systems, giving biomedical researchers the best of both worlds. Here, we provide extensive information to enable the implementation of ptychography by biomedical researchers in the visible light regime. We first discuss the intrinsic connections between spatial-domain coded ptychography and Fourier ptychography. A step-by-step guide then provides the user instructions for developing both systems with practical examples. In the spatial-domain implementation, we explain how a large-scale, high-performance blood-cell lens can be made at negligible expense. In the Fourier-domain implementation, we explain how adding a low-cost light source to a regular microscope can improve the resolution beyond the limit of the objective lens. The turnkey operation of these setups is suitable for use by professional research laboratories, as well as citizen scientists. Users with basic experience in optics and programming can build the setups within a week. The do-it-yourself nature of the setups also allows these procedures to be implemented in laboratory courses related to Fourier optics, biomedical instrumentation, digital image processing, robotics and capstone projects.

Optical ptychography for biomedical imaging: recent progress and future directions [Invited]

Ptychography is an enabling microscopy technique for both fundamental and applied sciences. In the past decade, it has become an indispensable imaging tool in most X-ray synchrotrons and national laboratories worldwide. However, ptychography's limited resolution and throughput in the visible light regime have prevented its wide adoption in biomedical research. Recent developments in this technique have resolved these issues and offer turnkey solutions for high-throughput optical imaging with minimum hardware modifications. The demonstrated imaging throughput is now greater than that of a high-end whole slide scanner. In this review, we discuss the basic principle of ptychography and summarize the main milestones of its development. Different ptychographic implementations are categorized into four groups based on their lensless/lens-based configurations and coded-illumination/coded-detection operations. We also highlight the related biomedical applications, including digital pathology, drug screening, urinalysis, blood analysis, cytometric analysis, rare cell screening, cell culture monitoring, cell and tissue imaging in 2D and 3D, polarimetric analysis, among others. Ptychography for high-throughput optical imaging, currently in its early stages, will continue to improve in performance and expand in its applications. We conclude this review article by pointing out several directions for its future development.

Three-dimensional bi-functional refractive index and fluorescence microscopy (BRIEF)

Fluorescence microscopy is a powerful tool for imaging biological samples with molecular specificity. In contrast, phase microscopy provides label-free measurement of the sample's refractive index (RI), which is an intrinsic optical property that quantitatively relates to cell morphology, mass, and stiffness. Conventional imaging techniques measure either the labeled fluorescence (functional) information or the label-free RI (structural) information, though it may be valuable to have both. For example, biological tissues have heterogeneous RI distributions, causing sample-induced scattering that degrades the fluorescence image quality. When both fluorescence and 3D RI are measured, one can use the RI information to digitally correct multiple-scattering effects in the fluorescence image. Here, we develop a new computational multi-modal imaging method based on epi-mode microscopy that reconstructs both 3D fluorescence and 3D RI from a single dataset. We acquire dozens of fluorescence images, each 'illuminated' by a single fluorophore, then solve an inverse problem with a multiple-scattering forward model. We experimentally demonstrate our method for epi-mode 3D RI imaging and digital correction of multiple-scattering effects in fluorescence images.

Large-field lattice structured illumination microscopy

In this paper, we present large-field, five-step lattice structured illumination microscopy (Lattice SIM). This method utilizes a 2D grating for lattice projection and a spatial light modulator (SLM) for phase shifting. Five phase-shifted intensity images are recorded to reconstruct a super-resolution image, enhancing the imaging speed and reducing the photo-bleaching both by 17%, compared to conventional two-direction and three-shift SIM. Furthermore, lattice SIM has a three-fold spatial bandwidth product (SBP) enhancement compared to SLM/DMD-based SIM, of which the fringe number is limited by the SLM/DMD pixel number. We believe that the proposed technique will be further developed and widely applied in many fields.

Large-field structured illumination microscopy based on 2D grating and a spatial light modulator

Structured illumination microscopy (SIM) has been widely used in biological research due to its merits of fast imaging speed, minimal invasiveness, super-resolution, and optical sectioning imaging capability. However, the conventional SIM that uses a spatial light modulator (SLM) for fringe projection often has a limited imaging field of view. Herein, we report a large-field SIM technique that combines a 2D grating for fringe pattern projection and an SLM for selecting fringe orientation and performing phase shifting digitally. The proposed SIM technique breaks the bottleneck of fringe number limited by the digital projection devices, while maintaining the advantage of high-speed (digital) phase shifting of conventional SIM. The method avoids the pixilation and dispersion effects of the SLMs. Finally, a 1.8-fold resolution enhancement in a large field of 690 × 517 µm² under a 20×/NA0.75 objective is experimentally demonstrated. The proposed technique can be widely applied to biology, chemistry, and industry.

High-Throughput, Label-Free and Slide-Free Histological Imaging by Computational Microscopy and Unsupervised Learning

Rapid and high-resolution histological imaging with minimal tissue preparation has long been a challenging and yet captivating medical pursuit. Here, the authors propose a promising and transformative histological imaging method, termed computational high-throughput autofluorescence microscopy by pattern illumination (CHAMP). With the assistance of computational microscopy, CHAMP enables high-throughput and label-free imaging of thick and unprocessed tissues with large surface irregularity at an acquisition speed of 10 mm2 /10 s with 1.1-µm lateral resolution. Moreover, the CHAMP image can be transformed into a virtually stained histological image (Deep-CHAMP) through unsupervised learning within 15 s, where significant cellular features are quantitatively extracted with high accuracy. The versatility of CHAMP is experimentally demonstrated using mouse brain/kidney and human lung tissues prepared with various clinical protocols, which enables a rapid and accurate intraoperative/postoperative pathological examination without tissue processing or staining, demonstrating its great potential as an assistive imaging platform for surgeons and pathologists to provide optimal adjuvant treatment.

Review of bio-optical imaging systems with a high space-bandwidth product

Optical imaging has served as a primary method to collect information about biosystems across scales-from functionalities of tissues to morphological structures of cells and even at biomolecular levels. However, to adequately characterize a complex biosystem, an imaging system with a number of resolvable points, referred to as a space-bandwidth product (SBP), in excess of one billion is typically needed. Since a gigapixel-scale far exceeds the capacity of current optical imagers, compromises must be made to obtain either a low spatial resolution or a narrow field-of-view (FOV). The problem originates from constituent refractive optics-the larger the aperture, the more challenging the correction of lens aberrations. Therefore, it is impractical for a conventional optical imaging system to achieve an SBP over hundreds of millions. To address this unmet need, a variety of high-SBP imagers have emerged over the past decade, enabling an unprecedented resolution and FOV beyond the limit of conventional optics. We provide a comprehensive survey of high-SBP imaging techniques, exploring their underlying principles and applications in bioimaging.

Super-resolution structured illumination microscopy: past, present and future

Structured illumination microscopy (SIM) has emerged as an essential technique for three-dimensional (3D) and live-cell super-resolution imaging. However, to date, there has not been a dedicated workshop or journal issue covering the various aspects of SIM, from bespoke hardware and software development and the use of commercial instruments to biological applications. This special issue aims to recap recent developments as well as outline future trends. In addition to SIM, we cover related topics such as complementary super-resolution microscopy techniques, computational imaging, visualization and image processing methods. This article is part of the Theo Murphy meeting issue 'Super-resolution structured illumination microscopy (part 1)'.

Incoherent color digital holography with computational coherent superposition for fluorescence imaging [Invited]

We present color fluorescence imaging using an incoherent digital holographic technique in which holographic multiplexing of multiple wavelengths is exploited. Self-interference incoherent digital holography with a single-path in-line configuration and the computational coherent superposition scheme are adopted to obtain color holographic three-dimensional information of self-luminous objects with a monochrome image sensor and no mechanical scanning. We perform not only simultaneous color three-dimensional sensing of multiple self-luminous objects but also color fluorescence imaging of stained biological samples. Color fluorescence imaging with an improved point spread function is also demonstrated experimentally by adopting a Fresnel incoherent correlation holography system.

Object detection neural network improves Fourier ptychography reconstruction

High resolution microscopy is heavily dependent on superb optical elements and superresolution microscopy even more so. Correcting unavoidable optical aberrations during post-processing is an elegant method to reduce the optical system's complexity. A prime method that promises superresolution, aberration correction, and quantitative phase imaging is Fourier ptychography. This microscopy technique combines many images of the sample, recorded at differing illumination angles akin to computed tomography and uses error minimisation between the recorded images with those generated by a forward model. The more precise knowledge of those illumination angles is available for the image formation forward model, the better the result. Therefore, illumination estimation from the raw data is an important step and supports correct phase recovery and aberration correction. Here, we derive how illumination estimation can be cast as an object detection problem that permits the use of a fast convolutional neural network (CNN) for this task. We find that faster-RCNN delivers highly robust results and outperforms classical approaches by far with an up to 3-fold reduction in estimation errors. Intriguingly, we find that conventionally beneficial smoothing and filtering of raw data is counterproductive in this type of application. We present a detailed analysis of the network's performance and provide all our developed software openly.

Metal-enhanced bright-field microscopy

Transparent samples pose serious challenges in bright-field microscopic imaging due to their low optical contrast, which also affects the imaging speed. While the role of the substrate in enhancing optical contrast has been investigated for imaging modalities such as fluorescence, a similar analysis for bright-field imaging has not been conducted. Here, we explore the effect of the substrate on the optical contrast and imaging speed in bright-field microscopic imaging. We develop a simulation model to explain the significant improvement in these imaging parameters by the introduction of reflective back substrates. We explain the effects using energy density distribution plots, power coupling, and scattering. Further, we explore the notion of optical contrast in the case of microfeatures and a procedure for selecting an appropriate metric to quantify the same. This work concludes with experimental results showing a twofold improvement in contrast and a fivefold improvement in imaging rate by an appropriate choice of back substrate.

High-resolution and large field-of-view Fourier ptychographic microscopy and its applications in biomedicine

Fourier ptychographic microscopy (FPM) is a promising and fast-growing computational imaging technique with high resolution, wide field-of-view (FOV) and quantitative phase recovery, which effectively tackles the problems of phase loss, aberration-introduced artifacts, narrow depth-of-field and the trade-off between resolution and FOV in conventional microscopy simultaneously. In this review, we provide a comprehensive roadmap of microscopy, the fundamental principles, advantages, and drawbacks of existing imaging techniques, and the significant roles that FPM plays in the development of science. Since FPM is an optimization problem in nature, we discuss the framework and related work. We also reveal the connection of Euler's formula between FPM and structured illumination microscopy. We review recent advances in FPM, including the implementation of high-precision quantitative phase imaging, high-throughput imaging, high-speed imaging, three-dimensional imaging, mixed-state decoupling, and introduce the prosperous biomedical applications. We conclude by discussing the challenging problems and future applications. FPM can be extended to a kind of framework to tackle the phase loss and system limits in the imaging system. This insight can be used easily in speckle imaging, incoherent imaging for retina imaging, large-FOV fluorescence imaging, etc.

Fourier ptychography: current applications and future promises

Traditional imaging systems exhibit a well-known trade-off between the resolution and the field of view of their captured images. Typical cameras and microscopes can either "zoom in" and image at high-resolution, or they can "zoom out" to see a larger area at lower resolution, but can rarely achieve both effects simultaneously. In this review, we present details about a relatively new procedure termed Fourier ptychography (FP), which addresses the above trade-off to produce gigapixel-scale images without requiring any moving parts. To accomplish this, FP captures multiple low-resolution, large field-of-view images and computationally combines them in the Fourier domain into a high-resolution, large field-of-view result. Here, we present details about the various implementations of FP and highlight its demonstrated advantages to date, such as aberration recovery, phase imaging, and 3D tomographic reconstruction, to name a few. After providing some basics about FP, we list important details for successful experimental implementation, discuss its relationship with other computational imaging techniques, and point to the latest advances in the field while highlighting persisting challenges.

Speckle-structured illumination for 3D phase and fluorescence computational microscopy

High-content biological microscopy targets high-resolution imaging across large fields-of-view, often achieved by computational imaging approaches. Previously, we demonstrated 2D multimodal high-content microscopy via structured illumination microscopy (SIM) with resolution > 2 × the diffraction limit, using speckle illumination from Scotch tape. In this work, we extend the method to 3D by leveraging the fact that the speckle illumination is in fact a 3D structured pattern. We use both a coherent and an incoherent imaging model to develop algorithms for joint retrieval of the 3D super-resolved fluorescent and complex-field distributions of the sample. Our reconstructed images resolve features beyond the physical diffraction-limit set by the system's objective and demonstrate 3D multimodal imaging with ∼ 0.6 × 0.6 × 6   μ m³ resolution over a volume of ∼ 314 × 500 × 24   μ m³.