Tomographic phase microscopy: principles and applications in bioimaging [Invited]

Perspective on quantitative phase imaging to improve precision cancer medicine

Significance

Quantitative phase imaging (QPI) offers a label-free approach to non-invasively characterize cellular processes by exploiting their refractive index based intrinsic contrast. QPI captures this contrast by translating refractive index associated phase shifts into intensity-based quantifiable data with nanoscale sensitivity. It holds significant potential for advancing precision cancer medicine by providing quantitative characterization of the biophysical properties of cells and tissue in their natural states.

Aim

This perspective aims to discuss the potential of QPI to increase our understanding of cancer development and its response to therapeutics. It also explores new developments in QPI methods towards advancing personalized cancer therapy and early detection.

Approach

We begin by detailing the technical advancements of QPI, examining its implementations across transmission and reflection geometries and phase retrieval methods, both interferometric and non-interferometric. The focus then shifts to QPI's applications in cancer research, including dynamic cell mass imaging for drug response assessment, cancer risk stratification, and in-vivo tissue imaging.

Results

QPI has emerged as a crucial tool in precision cancer medicine, offering insights into tumor biology and treatment efficacy. Its sensitivity to detecting nanoscale changes holds promise for enhancing cancer diagnostics, risk assessment, and prognostication. The future of QPI is envisioned in its integration with artificial intelligence, morpho-dynamics, and spatial biology, broadening its impact in cancer research.

Conclusions

QPI presents significant potential in advancing precision cancer medicine and redefining our approach to cancer diagnosis, monitoring, and treatment. Future directions include harnessing high-throughput dynamic imaging, 3D QPI for realistic tumor models, and combining artificial intelligence with multi-omics data to extend QPI's capabilities. As a result, QPI stands at the forefront of cancer research and clinical application in cancer care.

Fourier space aberration correction for high resolution refractive index imaging using incoherent light

An aberration correction method is introduced for 3D phase deconvolution microscopy. Our technique capitalizes on multiple illumination patterns to iteratively extract Fourier space aberrations, utilizing the overlapping information inherent in these patterns. By refining the point spread function based on the retrieved aberration data, we significantly improve the precision of refractive index deconvolution. We validate the effectiveness of our method on both synthetic and biological three-dimensional samples, achieving notable enhancements in resolution and measurement accuracy. The method's reliability in aberration retrieval is further confirmed through controlled experiments with intentionally induced spherical aberrations, underscoring its potential for wide-ranging applications in microscopy and biomedicine.

Quantitative phase imaging by gradient retardance optical microscopy

Quantitative phase imaging (QPI) has become a vital tool in bioimaging, offering precise measurements of wavefront distortion and, thus, of key cellular metabolism metrics, such as dry mass and density. However, only a few QPI applications have been demonstrated in optically thick specimens, where scattering increases background and reduces contrast. Building upon the concept of structured illumination interferometry, we introduce Gradient Retardance Optical Microscopy (GROM) for QPI of both thin and thick samples. GROM transforms any standard Differential Interference Contrast (DIC) microscope into a QPI platform by incorporating a liquid crystal retarder into the illumination path, enabling independent phase-shifting of the DIC microscope's sheared beams. GROM greatly simplifies related configurations, reduces costs, and eradicates energy losses in parallel imaging modalities, such as fluorescence. We successfully tested GROM on a diverse range of specimens, from microbes and red blood cells to optically thick (~ 300 μm) plant roots without fixation or clearing.

Recent Advances and Current Trends in Transmission Tomographic Diffraction Microscopy

Optical microscopy techniques are among the most used methods in biomedical sample characterization. In their more advanced realization, optical microscopes demonstrate resolution down to the nanometric scale. These methods rely on the use of fluorescent sample labeling in order to break the diffraction limit. However, fluorescent molecules' phototoxicity or photobleaching is not always compatible with the investigated samples. To overcome this limitation, quantitative phase imaging techniques have been proposed. Among these, holographic imaging has demonstrated its ability to image living microscopic samples without staining. However, for a 3D assessment of samples, tomographic acquisitions are needed. Tomographic Diffraction Microscopy (TDM) combines holographic acquisitions with tomographic reconstructions. Relying on a 3D synthetic aperture process, TDM allows for 3D quantitative measurements of the complex refractive index of the investigated sample. Since its initial proposition by Emil Wolf in 1969, the concept of TDM has found a lot of applications and has become one of the hot topics in biomedical imaging. This review focuses on recent achievements in TDM development. Current trends and perspectives of the technique are also discussed.

Beyond fluorescence: advances in computational label-free full specificity in 3D quantitative phase microscopy

Despite remarkable progresses in quantitative phase imaging (QPI) microscopes, their wide acceptance is limited due to the lack of specificity compared with the well-established fluorescence microscopy. In fact, the absence of fluorescent tag prevents to identify subcellular structures in single cells, making challenging the interpretation of label-free 2D and 3D phase-contrast data. Great effort has been made by many groups worldwide to address and overcome such limitation. Different computational methods have been proposed and many more are currently under investigation to achieve label-free microscopic imaging at single-cell level to recognize and quantify different subcellular compartments. This route promises to bridge the gap between QPI and FM for real-world applications.

Optical diffraction tomography for assessing single cell models in angular light scattering

Angularly resolved light scattering (ALS) has become a useful tool for assessing the size and refractive index of biological scatterers at cellular and organelle length scales. Sizing organelle populations with ALS relies on Mie scattering theory models, which require significant assumptions about the object, including spherical scatterers and a homogeneous medium. These assumptions may incur greater error at the single cell level, where there are fewer scatterers to be averaged over. We investigate the validity of these assumptions using 3D refractive index (RI) tomograms measured via optical diffraction tomography (ODT). We compute the angular scattering on digitally manipulated tomograms with increasingly strong model assumptions, including RI-matched immersion media, homogeneous cytosol, and spherical organelles. We also compare the tomogram-computed angular scattering to experimental measurements of angular scattering from the same cells to ensure that the ODT-based approach accurately models angular scattering. We show that enforced RI-matching with the immersion medium and a homogeneous cytosol significantly affects the angular scattering intensity shape, suggesting that these assumptions can reduce the accuracy of size distribution estimates.

AI-driven projection tomography with multicore fibre-optic cell rotation

Optical tomography has emerged as a non-invasive imaging method, providing three-dimensional insights into subcellular structures and thereby enabling a deeper understanding of cellular functions, interactions, and processes. Conventional optical tomography methods are constrained by a limited illumination scanning range, leading to anisotropic resolution and incomplete imaging of cellular structures. To overcome this problem, we employ a compact multi-core fibre-optic cell rotator system that facilitates precise optical manipulation of cells within a microfluidic chip, achieving full-angle projection tomography with isotropic resolution. Moreover, we demonstrate an AI-driven tomographic reconstruction workflow, which can be a paradigm shift from conventional computational methods, often demanding manual processing, to a fully autonomous process. The performance of the proposed cell rotation tomography approach is validated through the three-dimensional reconstruction of cell phantoms and HL60 human cancer cells. The versatility of this learning-based tomographic reconstruction workflow paves the way for its broad application across diverse tomographic imaging modalities, including but not limited to flow cytometry tomography and acoustic rotation tomography. Therefore, this AI-driven approach can propel advancements in cell biology, aiding in the inception of pioneering therapeutics, and augmenting early-stage cancer diagnostics.

Enhanced functionalities of immune cells separated by a microfluidic lattice: assessment based on holotomography

The isolation of white blood cells (WBCs) from whole blood constitutes a pivotal process for immunological studies, diagnosis of hematologic disorders, and the facilitation of immunotherapy. Despite the ubiquity of density gradient centrifugation in WBC isolation, its influence on WBC functionality remains inadequately understood. This research employs holotomography to explore the effects of two distinct WBC separation techniques, namely conventional centrifugation and microfluidic separation, on the functionality of the isolated cells. We utilize three-dimensional refractive index distribution and time-lapse dynamics to analyze individual WBCs in-depth, focusing on their morphology, motility, and phagocytic capabilities. Our observations highlight that centrifugal processes negatively impact WBC motility and phagocytic capacity, whereas microfluidic separation yields a more favorable outcome in preserving WBC functionality. These findings emphasize the potential of microfluidic separation techniques as a viable alternative to traditional centrifugation for WBC isolation, potentially enabling more precise analyses in immunology research and improving the accuracy of hematologic disorder diagnoses.

Label-free biomedical optical imaging

Label-free optical imaging employs natural and nondestructive approaches for the visualisation of biomedical samples for both biological assays and clinical diagnosis. Currently, this field revolves around multiple broad technology-oriented communities, each with a specific focus on a particular modality despite the existence of shared challenges and applications. As a result, biologists or clinical researchers who require label-free imaging are often not aware of the most appropriate modality to use. This manuscript presents a comprehensive review of and comparison among different label-free imaging modalities and discusses common challenges and applications. We expect this review to facilitate collaborative interactions between imaging communities, push the field forward and foster technological advancements, biophysical discoveries, as well as clinical detection, diagnosis, and monitoring of disease.

Roadmap on Label-Free Super-Resolution Imaging

Label-free super-resolution (LFSR) imaging relies on light-scattering processes in nanoscale objects without a need for fluorescent (FL) staining required in super-resolved FL microscopy. The objectives of this Roadmap are to present a comprehensive vision of the developments, the state-of-the-art in this field, and to discuss the resolution boundaries and hurdles which need to be overcome to break the classical diffraction limit of the LFSR imaging. The scope of this Roadmap spans from the advanced interference detection techniques, where the diffraction-limited lateral resolution is combined with unsurpassed axial and temporal resolution, to techniques with true lateral super-resolution capability which are based on understanding resolution as an information science problem, on using novel structured illumination, near-field scanning, and nonlinear optics approaches, and on designing superlenses based on nanoplasmonics, metamaterials, transformation optics, and microsphere-assisted approaches. To this end, this Roadmap brings under the same umbrella researchers from the physics and biomedical optics communities in which such studies have often been developing separately. The ultimate intent of this paper is to create a vision for the current and future developments of LFSR imaging based on its physical mechanisms and to create a great opening for the series of articles in this field.

Label-Free Intracellular Multi-Specificity in Yeast Cells by Phase-Contrast Tomographic Flow Cytometry

In-flow phase-contrast tomography provides a 3D refractive index of label-free cells in cytometry systems. Its major limitation, as with any quantitative phase imaging approach, is the lack of specificity compared to fluorescence microscopy, thus restraining its huge potentialities in single-cell analysis and diagnostics. Remarkable results in introducing specificity are obtained through artificial intelligence (AI), but only for adherent cells. However, accessing the 3D fluorescence ground truth and obtaining accurate voxel-level co-registration of image pairs for AI training is not viable for high-throughput cytometry. The recent statistical inference approach is a significant step forward for label-free specificity but remains limited to cells' nuclei. Here, a generalized computational strategy based on a self-consistent statistical inference to achieve intracellular multi-specificity is shown. Various subcellular compartments (i.e., nuclei, cytoplasmic vacuoles, the peri-vacuolar membrane area, cytoplasm, vacuole-nucleus contact site) can be identified and characterized quantitatively at different phases of the cells life cycle by using yeast cells as a biological model. Moreover, for the first time, virtual reality is introduced for handling the information content of multi-specificity in single cells. Full fruition is proofed for exploring and interacting with 3D quantitative biophysical parameters of the identified compartments on demand, thus opening the route to a metaverse for 3D microscopy.

Volumetric segmentation of biological cells and subcellular structures for optical diffraction tomography images

Three-dimensional, quantitative imaging of biological cells and their internal structures performed by optical diffraction tomography (ODT) is an important part of biomedical research. However, conducting quantitative analysis of ODT images requires performing 3D segmentation with high accuracy, often unattainable with available segmentation methods. Therefore, in this work, we present a new semi-automatic method, called ODT-SAS, which combines several non-machine-learning techniques to segment cells and 2 types of their organelles: nucleoli and lipid structures (LS). ODT-SAS has been compared with Cellpose and slice-by-slice manual segmentation, respectively, in cell segmentation and organelles segmentation. The comparison shows superiority of ODT-SAS over Cellpose and reveals the potential of our technique in detecting cells, nucleoli and LS.

Mapping nanoscale topographic features in thick tissues with speckle diffraction tomography

Resolving three-dimensional morphological features in thick specimens remains a significant challenge for label-free imaging. We report a new speckle diffraction tomography (SDT) approach that can image thick biological specimens with ~500 nm lateral resolution and ~1 μm axial resolution in a reflection geometry. In SDT, multiple-scattering background is rejected through spatiotemporal gating provided by dynamic speckle-field interferometry, while depth-resolved refractive index maps are reconstructed by developing a comprehensive inverse-scattering model that also considers specimen-induced aberrations. Benefiting from the high-resolution and full-field quantitative imaging capabilities of SDT, we successfully imaged red blood cells and quantified their membrane fluctuations behind a turbid medium with a thickness of 2.8 scattering mean-free paths. Most importantly, we performed volumetric imaging of cornea inside an ex vivo rat eye and quantified its optical properties, including the mapping of nanoscale topographic features of Dua's and Descemet's membranes that had not been previously visualized.

Three-dimensional refractive index estimation based on deep-inverse non-interferometric optical diffraction tomography (ODT-Deep)

Optical diffraction tomography (ODT) solves an inverse scattering problem to obtain label-free, 3D refractive index (RI) estimation of biological specimens. This work demonstrates 3D RI retrieval methods suitable for partially-coherent ODT systems supported by intensity-only measurements consisting of axial and angular illumination scanning. This framework allows for access to 3D quantitative RI contrast using a simplified non-interferometric technique. We consider a traditional iterative tomographic solver based on a multiple in-plane representation of the optical scattering process and gradient descent optimization adapted for focus-scanning systems, as well as an approach that relies solely on 3D convolutional neural networks (CNNs) to invert the scattering process. The approaches are validated using simulations of the 3D scattering potential for weak phase 3D biological samples.

Phase unwrapping using deep learning in holographic tomography

Holographic tomography (HT) is a measurement technique that generates phase images, often containing high noise levels and irregularities. Due to the nature of phase retrieval algorithms within the HT data processing, the phase has to be unwrapped before tomographic reconstruction. Conventional algorithms lack noise robustness, reliability, speed, and possible automation. In order to address these problems, this work proposes a convolutional neural network based pipeline consisting of two steps: denoising and unwrapping. Both steps are carried out under the umbrella of a U-Net architecture; however, unwrapping is aided by introducing Attention Gates (AG) and Residual Blocks (RB) to the architecture. Through the experiments, the proposed pipeline makes possible the phase unwrapping of highly irregular, noisy, and complex experimental phase images captured in HT. This work proposes phase unwrapping carried out by segmentation with a U-Net network, that is aided by a pre-processing denoising step. It also discusses the implementation of the AGs and RBs in an ablation study. What is more, this is the first deep learning based solution that is trained solely on real images acquired with HT.

Efficient and accurate intensity diffraction tomography of multiple-scattering samples

Optical Diffraction Tomography (ODT) is a label-free method to quantitatively estimate the 3D refractive index (RI) distributions of microscopic samples. Recently, significant efforts were directed towards methods to model multiple-scattering objects. The fidelity of reconstructions rely on accurately modelling light-matter interactions, but the efficient simulation of light propagation through high-RI structures over a large range of illumination angles is still challenging. Here we present a solution dealing with these problems, proposing a method that allows one to efficiently model the tomographic image formation for strongly scattering objects illuminated over a wide range of angles. Instead of propagating tilted plane waves we apply rotations on the illuminated object and optical field and formulate a new and robust multi-slice model suitable for high-RI contrast structures. We test reconstructions made by our approach against simulations and experiments, using rigorous solutions to Maxwell's equations as ground truth. We find the proposed method to produce reconstructions of higher fidelity compared to conventional multi-slice methods, especially for the challenging case of strongly scattering samples where conventional reconstruction methods fail.

Label-free liquid biopsy through the identification of tumor cells by machine learning-powered tomographic phase imaging flow cytometry

Image-based identification of circulating tumor cells in microfluidic cytometry condition is one of the most challenging perspectives in the Liquid Biopsy scenario. Here we show a machine learning-powered tomographic phase imaging flow cytometry system capable to provide high-throughput 3D phase-contrast tomograms of each single cell. In fact, we show that discrimination of tumor cells against white blood cells is potentially achievable with the aid of artificial intelligence in a label-free flow-cyto-tomography method. We propose a hierarchical machine learning decision-maker, working on a set of features calculated from the 3D tomograms of the cells' refractive index. We prove that 3D morphological features are adequately distinctive to identify tumor cells versus the white blood cell background in the first stage and, moreover, in recognizing the tumor type at the second decision step. Proof-of-concept experiments are shown, in which two different tumor cell lines, namely neuroblastoma cancer cells and ovarian cancer cells, are used against monocytes. The reported results allow claiming the identification of tumor cells with a success rate higher than 97% and with an accuracy over 97% in discriminating between the two cancer cell types, thus opening in a near future the route to a new Liquid Biopsy tool for detecting and classifying circulating tumor cells in blood by stain-free method.

Partially coherent broadband 3D optical transfer functions with arbitrary temporal and angular power spectra

Optical diffraction tomography is a powerful technique to produce 3D volumetric images of biological samples using contrast produced by variations in the index of refraction in an unlabeled specimen. While this is typically performed with coherent illumination from a variety of angles, interest has grown in partially coherent methods due to the simplicity of the illumination and the computation-free axial sectioning provided by the coherence window of the source. However, such methods rely on the symmetry or discretization of a source to facilitate quantitative analysis and are unable to efficiently handle arbitrary illumination that may vary asymmetrically in angle and continuously in the spectrum, such as diffusely scattered or thermal sources. A general broadband theory may expand the scope of illumination methods available for quantitative analysis, as partially coherent sources are commonly available and may benefit from the effects of spatial and temporal incoherence. In this work, we investigate partially coherent tomographic phase microscopy from arbitrary sources regardless of angular distribution and spectrum by unifying the effects of spatial and temporal coherence into a single formulation. This approach further yields a method for efficient computation of the overall systems' optical transfer function, which scales with O(n ³), down from O(mn ⁴) for existing convolutional methods, where n ³ is the number of spatial voxels in 3D space and m is the number of discrete wavelengths in the illumination spectrum. This work has important implications for enabling partially coherent 3D quantitative phase microscopy and refractive index tomography in virtually any transmission or epi-illumination microscope.

3D reconstruction of unstained weakly scattering cells from a single defocused hologram

We investigate the problem of 3D complex field reconstruction corresponding to unstained red blood cells (RBCs) with a single defocused off-axis digital hologram. The main challenge in this problem is the localization of cells to the correct axial range. While investigating the volume recovery problem for a continuous phase object like the RBC, we observe an interesting feature of the backpropagated field that it does not show a clear focusing effect. Therefore, sparsity enforcement within the iterative optimization framework using a single hologram data frame cannot effectively restrict the reconstruction to the true object volume. For phase objects, it is known that the amplitude contrast of the backpropagated object field at the focus plane is minimum. We use this information available in the recovered object field in the hologram plane to device depth-dependent weights that are proportional to the inverse of amplitude contrast. This weight function is employed in the iterative steps of the optimization algorithm to assist the object volume localization. The overall reconstruction process is performed using the mean gradient descent (MGD) framework. Experimental illustrations of 3D volume reconstruction of the healthy as well as malaria-infected RBCs are presented. A test sample of polystyrene microsphere bead is also used to validate the axial localization capability of the proposed iterative technique. The proposed methodology is simple to implement experimentally and provides an approximate tomographic solution, which is axially restricted and consistent with the object field data.

Jones tomographic diffractive microscopy with a polarized array sensor

Tomographic diffractive microscopy (TDM) based on scalar light-field approximation is widely implemented. Samples exhibiting anisotropic structures, however, necessitate accounting for the vectorial nature of light, leading to 3-D quantitative polarimetric imaging. In this work, we have developed a high-numerical aperture (at both illumination and detection) Jones TDM system, with detection multiplexing via a polarized array sensor (PAS), for imaging optically birefringent samples at high resolution. The method is first studied through image simulations. To validate our setup, an experiment using a sample containing both birefringent and non-birefringent objects is performed. Araneus diadematus spider silk fiber and Pinna nobilis oyster shell crystals are finally studied, allowing us to assess both birefringence and fast-axis orientation maps.

3D differential interference contrast microscopy using polarisation-sensitive tomographic diffraction microscopy

Tomographic diffraction microscopy (TDM) is a generalisation of digital holographic microscopy (DHM), for which the illumination angle onto the sample is fully controlled, which has become a tool of choice for 3D, high-resolution imaging of unlabelled samples. TDM makes it possible to obtain the optical field in both amplitude and phase for each illumination angle. Proper information reallocation eventually allows for 3D reconstruction of the complex refractive index map. On the other hand, polarisation array sensors (PAS) paves new way for TDM, as vectorial information assessment about the investigated sample. In this contribution, we show an alternative use of this polarisation information based on the phase sensitive nature of TDM. Here, we demonstrated that TDM coupled with PAS can lead to a 3D differential interference contrast (DIC) microscope with almost no experimental configuration modification.

Multiple-scattering simulator-trained neural network for intensity diffraction tomography

Recovering 3D phase features of complex biological samples traditionally sacrifices computational efficiency and processing time for physical model accuracy and reconstruction quality. Here, we overcome this challenge using an approximant-guided deep learning framework in a high-speed intensity diffraction tomography system. Applying a physics model simulator-based learning strategy trained entirely on natural image datasets, we show our network can robustly reconstruct complex 3D biological samples. To achieve highly efficient training and prediction, we implement a lightweight 2D network structure that utilizes a multi-channel input for encoding the axial information. We demonstrate this framework on experimental measurements of weakly scattering epithelial buccal cells and strongly scattering C. elegans worms. We benchmark the network's performance against a state-of-the-art multiple-scattering model-based iterative reconstruction algorithm. We highlight the network's robustness by reconstructing dynamic samples from a living worm video. We further emphasize the network's generalization capabilities by recovering algae samples imaged from different experimental setups. To assess the prediction quality, we develop a quantitative evaluation metric to show that our predictions are consistent with both multiple-scattering physics and experimental measurements.

Standardizing image assessment in optical diffraction tomography

Optical diffraction tomography (ODT) has gradually become a popular label-free imaging technique that offers diffraction-limited resolution by mapping an object's three-dimensional (3D) refractive index (RI) distribution. However, there is a lack of comprehensive quantitative image assessment metrics in ODT for studying how various experimental conditions influence image quality, and subsequently optimizing the experimental conditions. In this Letter, we propose to standardize the image assessment in ODT by proposing a set of metrics, including signal-to-noise ratio (SNR), contrast-to-noise ratio (CNR), and structural distinguishability (SD). To test the feasibility of the metrics, we performed experiments on angle-scanning ODT by varying the number of illumination angles, RI contrast of samples, sample feature sizes, and sample types (e.g., standard polystyrene beads and 3D printed structures) and evaluated the RI tomograms with SNR, CNR, and SD. We further quantitatively studied how image quality can be improved, and tested the image assessment metrics on subcellular structures of living cells. We envision the proposed image assessment metrics may greatly benefit end-users for assessing the RI tomograms, as well as experimentalists for optimizing ODT instruments.

High-fidelity optical diffraction tomography of live organisms using iodixanol refractive index matching

Optical diffraction tomography (ODT) enables the three-dimensional (3D) refractive index (RI) reconstruction. However, when the RI difference between a sample and a medium increases, the effects of light scattering become significant, preventing the acquisition of high-quality and accurate RI reconstructions. Herein, we present a method for high-fidelity ODT by introducing non-toxic RI matching media. Optimally reducing the RI contrast enhances the fidelity and accuracy of 3D RI reconstruction, enabling visualization of the morphology and intra-organization of live biological samples without producing toxic effects. We validate our method using various biological organisms, including C. albicans and C. elegans.

Stain-free identification of cell nuclei using tomographic phase microscopy in flow cytometry

Quantitative Phase Imaging (QPI) has gained popularity in bioimaging because it can avoid the need for cell staining, which in some cases is difficult or impossible. However, as a result, QPI does not provide labelling of various specific intracellular structures. Here we show a novel computational segmentation method based on statistical inference that makes it possible for QPI techniques to identify the cell nucleus. We demonstrate the approach with refractive index tomograms of stain-free cells reconstructed through the tomographic phase microscopy in flow cytometry mode. In particular, by means of numerical simulations and two cancer cell lines, we demonstrate that the nucleus can be accurately distinguished within the stain-free tomograms. We show that our experimental results are consistent with confocal fluorescence microscopy (FM) data and microfluidic cytofluorimeter outputs. This is a significant step towards extracting specific three-dimensional intracellular structures directly from the phase-contrast data in a typical flow cytometry configuration.

White blood cell detection, classification and analysis using phase imaging with computational specificity (PICS)

Treatment of blood smears with Wright's stain is one of the most helpful tools in detecting white blood cell abnormalities. However, to diagnose leukocyte disorders, a clinical pathologist must perform a tedious, manual process of locating and identifying individual cells. Furthermore, the staining procedure requires considerable preparation time and clinical infrastructure, which is incompatible with point-of-care diagnosis. Thus, rapid and automated evaluations of unlabeled blood smears are highly desirable. In this study, we used color spatial light interference microcopy (cSLIM), a highly sensitive quantitative phase imaging (QPI) technique, coupled with deep learning tools, to localize, classify and segment white blood cells (WBCs) in blood smears. The concept of combining QPI label-free data with AI for the purpose of extracting cellular specificity has recently been introduced in the context of fluorescence imaging as phase imaging with computational specificity (PICS). We employed AI models to first translate SLIM images into brightfield micrographs, then ran parallel tasks of locating and labelling cells using EfficientNet, which is an object detection model. Next, WBC binary masks were created using U-net, a convolutional neural network that performs precise segmentation. After training on digitally stained brightfield images of blood smears with WBCs, we achieved a mean average precision of 75% for localizing and classifying neutrophils, eosinophils, lymphocytes, and monocytes, and an average pixel-wise majority-voting F1 score of 80% for determining the cell class from semantic segmentation maps. Therefore, PICS renders and analyzes synthetically stained blood smears rapidly, at a reduced cost of sample preparation, providing quantitative clinical information.

Near-infrared, wavelength, and illumination scanning holographic tomography

We present a holographic tomography technique in which the projections are acquired using both wavelength and illumination scanning in the near-infrared region. We show how to process the acquired data to obtain correct values of three-dimensional refractive index distributions in both single-wavelength and multi-wavelength data acquisition schemes and how to properly account for the dispersion of the sample. We perform numerical and experimental comparisons of different illumination scenarios to determine the most efficient measurement protocol. We show that the multi-wavelength protocol is advantageous in terms of signal-to-noise ratio and contrast-to-noise ratio over single-wavelength protocols, even for the same number of projections used for reconstructions. Finally, we show that this approach is suitable for providing high-quality refractive index distributions of relatively thick colon cancer samples.

Cell image reconstruction using digital holography with an improved GS algorithm

Digital holography is an effective technology in image reconstruction as amplitude and phase information of cells can be acquired without any staining. In this paper, we propose a holographic technique with an improved Gerchberg-Saxton (GS) algorithm to reconstruct cell imaging based on phase reconstruction information. Comparative experiments are conducted on four specific models to investigate the effectiveness of the proposed method. The morphological parameters (such as shape, volume, and sphericity) of abnormal erythrocytes can be obtained by reconstructing cell hologram of urinary sediment. Notably, abnormal red blood cells can also be detected in mussy circumstances by the proposed method, owing to the significantly biophysical contrast (refractive index distribution and mass density) between two different cells. Therefore, this proposed method has a broad application prospect in cell image reconstruction and cell dynamic detection.

Automatic whole blood cell analysis from blood smear using label-free multi-modal imaging with deep neural networks

Whole blood cell analysis is widely used in medical applications since its results are indicators for diagnosing a series of diseases. In this work, we report automatic whole blood cell analysis from blood smear using label-free multi-modal imaging with deep neural networks. First, a commercial microscope equipped with our developed Phase Real-time Microscope Camera (PhaseRMiC) obtains both bright-field and quantitative phase images. Then, these images are automatically processed by our designed blood smear recognition networks (BSRNet) that recognize erythrocytes, leukocytes and platelets. Finally, blood cell parameters such as counts, shapes and volumes can be extracted according to both quantitative phase images and automatic recognition results. The proposed whole blood cell analysis technique provides high-quality blood cell images and supports accurate blood cell recognition and analysis. Moreover, this approach requires rather simple and cost-effective setups as well as easy and rapid sample preparations. Therefore, this proposed method has great potential application in blood testing aiming at disease diagnostics.

High-fidelity intensity diffraction tomography with a non-paraxial multiple-scattering model

We propose a novel intensity diffraction tomography (IDT) reconstruction algorithm based on the split-step non-paraxial (SSNP) model for recovering the 3D refractive index (RI) distribution of multiple-scattering biological samples. High-quality IDT reconstruction requires high-angle illumination to encode both low- and high- spatial frequency information of the 3D biological sample. We show that our SSNP model can more accurately compute multiple scattering from high-angle illumination compared to paraxial approximation-based multiple-scattering models. We apply this SSNP model to both sequential and multiplexed IDT techniques. We develop a unified reconstruction algorithm for both IDT modalities that is highly computationally efficient and is implemented by a modular automatic differentiation framework. We demonstrate the capability of our reconstruction algorithm on both weakly scattering buccal epithelial cells and strongly scattering live C. elegans worms and live C. elegans embryos.

Developing a Reliable Holographic Flow Cyto-Tomography Apparatus by Optimizing the Experimental Layout and Computational Processing

Digital Holographic Tomography (DHT) has recently been established as a means of retrieving the 3D refractive index mapping of single cells. To make DHT a viable system, it is necessary to develop a reliable and robust holographic apparatus in order that such technology can be utilized outside of specialized optics laboratories and operated in the in-flow modality. In this paper, we propose a quasi-common-path lateral-shearing holographic optical set-up to be used, for the first time, for DHT in a flow-cytometer modality. The proposed solution is able to withstand environmental vibrations that can severely affect the interference process. Furthermore, we have scaled down the system while ensuring that a full 360° rotation of the cells occurs in the field-of-view, in order to retrieve 3D phase-contrast tomograms of single cells flowing along a microfluidic channel. This was achieved by setting the camera sensor at 45° with respect to the microfluidic direction. Additional optimizations were made to the computational elements to ensure the reliable retrieval of 3D refractive index distributions by demonstrating an effective method of tomographic reconstruction, based on high-order total variation. The results were first demonstrated using realistic 3D numerical phantom cells to assess the performance of the proposed high-order total variation method in comparison with the gold-standard algorithm for tomographic reconstructions: namely, filtered back projection. Then, the proposed DHT system and the processing pipeline were experimentally validated for monocytes and mouse embryonic fibroblast NIH-3T3 cells lines. Moreover, the repeatability of these tomographic measurements was also investigated by recording the same cell multiple times and quantifying the ability to provide reliable and comparable tomographic reconstructions, as confirmed by a correlation coefficient greater than 95%. The reported results represent various steps forward in several key aspects of in-flow DHT, thus paving the way for its use in real-world applications.

Neurons: The Interplay between Cytoskeleton, Ion Channels/Transporters and Mitochondria

Neurons are permanent cells whose key feature is information transmission via chemical and electrical signals. Therefore, a finely tuned homeostasis is necessary to maintain function and preserve neuronal lifelong survival. The cytoskeleton, and in particular microtubules, are far from being inert actors in the maintenance of this complex cellular equilibrium, and they participate in the mobilization of molecular cargos and organelles, thus influencing neuronal migration, neuritis growth and synaptic transmission. Notably, alterations of cytoskeletal dynamics have been linked to alterations of neuronal excitability. In this review, we discuss the characteristics of the neuronal cytoskeleton and provide insights into alterations of this component leading to human diseases, addressing how these might affect excitability/synaptic activity, as well as neuronal functioning. We also provide an overview of the microscopic approaches to visualize and assess the cytoskeleton, with a specific focus on mitochondrial trafficking.

Phase-Separated Multienzyme Compartmentalization for Terpene Biosynthesis in a Prokaryote

Liquid-liquid phase separation (LLPS) forms biomolecular condensates or coacervates in cells. Metabolic enzymes can form phase-separated subcellular compartments that enrich enzymes, cofactors, and substrates. Herein, we report the construction of synthetic multienzyme condensates that catalyze the biosynthesis of a terpene, α-farnesene, in the prokaryote E. coli. RGGRGG derived from LAF-1 was used as the scaffold protein to form the condensates by LLPS. Multienzyme condensates were then formed by assembling two enzymes Idi and IspA through an RIAD/RIDD interaction. Multienzyme condensates constructed inside E. coli cells compartmentalized the cytosolic space into regions of high and low enzyme density and led to a significant enhancement of α-farnesene production. This work demonstrates LLPS-driven compartmentalization of the cytosolic space of prokaryotic cells, condensation of a biosynthetic pathway, and enhancement of the biosynthesis of α-farnesene.

Numerical analysis of the effect of reduced temporal coherence in quantitative phase microscopy and tomography

We present the numerical analysis of the effect of the temporarily partially coherent illumination on the phase measurement accuracy in digital holography microscopy (DHM) and optical diffraction tomography (ODT), as reconstruction algorithms tend to assume purely monochromatic conditions. In the regime of reduced temporal coherence, we simulate the hologram formation in two different optical setups, representing classical off-axis two-beam and grating common-path configurations. We consider two ODT variants: with sample rotation and angle-scanning of illumination. Besides the coherence degree of illumination, our simulation considers the influence of the sample normal dispersion, shape of the light spectrum, and optical parameters of the imaging setup. As reconstruction algorithms we employ Fourier hologram method and first-order Rytov approximation with direct inversion and nonnegativity constraints. Quantitative evaluation of the measurement results deviations introduced by the mentioned error sources is comprehensively analyzed, for the first time to the best of our knowledge. Obtained outcomes indicate low final DHM/ODT reconstruction errors for the grating-assisted common-path configuration. Nevertheless, dispersion and asymmetric spectrum introduce non-negligible overestimated refractive index values and noise, and should be thus carefully considered within experimental frameworks.

Multi-slice ptychographic imaging with multistage coarse-to-fine reconstruction

The ability to image 3D samples with optical sectioning is essential for the study of tomographic morphology in material and biological sciences. However, it is often hampered by limitations of acquisition speed and equipment complexity when performing 3D volumetric imaging. Here, we propose, to the best of our knowledge, a new method for 3D reconstruction from a minimum of four intensity-only measurements. The complementary structured patterns provided by the digital micromirror device (DMD) irradiate the outermost layer of the sample to generate the corresponding diffraction intensities for recording, which enables rapid scanning of loaded patterns for fast acquisition. Our multistage reconstruction algorithm first extracts the overall coarse-grained information, and then iteratively optimizes the information of different layers to obtain fine features, thereby achieving high-resolution 3D tomography. The high-fidelity reconstruction in experiments on two-slice resolution targets, unstained Polyrhachis vicina Roger and freely moving C. elegans proves the robustness of the method. Compared with traditional 3D reconstruction methods such as interferometry-based methods or Fourier ptychographic tomography (FPT), our method increases the reconstruction speed by at least 10 times and is suitable for label-free dynamic imaging in multiple-scattering samples. Such 3D reconstruction suggests potential applications in a wide range of fields.

A review of microscopic cell imaging and neural network recognition for synergistic cyanobacteria identification and enumeration

Real-time cyanobacteria/algal monitoring is a valuable tool for early detection of harmful algal blooms, water treatment efficacy evaluation, and assists tailored water quality risk assessments by considering taxonomy and cell counts. This review evaluates and proposes a synergistic approach using neural network image recognition and microscopic imaging devices by first evaluating published literature for both imaging microscopes and image recognition. Quantitative phase imaging was considered the most promising of the investigated imaging techniques due to the provision of enhanced information relative to alternatives. This information provides significant value to image recognition neural networks, such as the convolutional neural networks discussed within this review. Considering published literature, a cyanobacteria monitoring system and corresponding image processing workflow using in situ sample collection buoys and on-shore sample processing was proposed. This system can be implemented using commercially available equipment to facilitate accurate, real-time water quality monitoring.

Graphical abstract

Speeding up reconstruction of 3D tomograms in holographic flow cytometry via deep learning

Tomographic flow cytometry by digital holography is an emerging imaging modality capable of collecting multiple views of moving and rotating cells with the aim of recovering their refractive index distribution in 3D. Although this modality allows us to access high-resolution imaging with high-throughput, the huge amount of time-lapse holographic images to be processed (hundreds of digital holograms per cell) constitutes the actual bottleneck. This prevents the system from being suitable for lab-on-a-chip platforms in real-world applications, where fast analysis of measured data is mandatory. Here we demonstrate a significant speeding-up reconstruction of phase-contrast tomograms by introducing in the processing pipeline a multi-scale fully-convolutional context aggregation network. Although it was originally developed in the context of semantic image analysis, we demonstrate for the first time that it can be successfully adapted to a holographic lab-on-chip platform for achieving 3D tomograms through a faster computational process. We trained the network with input-output image pairs to reproduce the end-to-end holographic reconstruction process, i.e. recovering quantitative phase maps (QPMs) of single cells from their digital holograms. Then, the sequence of QPMs of the same rotating cell is used to perform the tomographic reconstruction. The proposed approach significantly reduces the computational time for retrieving tomograms, thus making them available in a few seconds instead of tens of minutes, while essentially preserving the high-content information of tomographic data. Moreover, we have accomplished a compact deep convolutional neural network parameterization that can fit into on-chip SRAM and a small memory footprint, thus demonstrating its possible exploitation to provide onboard computations for lab-on-chip devices with low processing hardware resources.

Computational multi-wavelength phase synthesis using convolutional neural networks [Invited]

Multi-wavelength digital holographic microscopy (MWDHM) provides indirect measurements of the refractive index for non-dispersive samples. Successive-shot MWDHM is not appropriate for dynamic samples and single-shot MWDHM significantly increases the complexity of the optical setup due to the need for multiple lasers or a wavelength tunable source. Here we consider deep learning convolutional neural networks for computational phase synthesis to obtain high-speed simultaneous phase estimates on different wavelengths and thus single-shot estimates of the integral refractive index without increased experimental complexity. This novel, to the best of our knowledge, computational concept is validated using cell phantoms consisting of internal refractive index variations representing cytoplasm and membrane-bound organelles, respectively, and a simulation of a realistic holographic recording process. Specifically, in this work we employed data-driven computational techniques to perform accurate dual-wavelength hologram synthesis (hologram-to-hologram prediction), dual-wavelength phase synthesis (unwrapped phase-to-phase prediction), direct phase-to-index prediction using a single wavelength, hologram-to-phase prediction, and 2D phase unwrapping with sharp discontinuities (wrapped-to-unwrapped phase prediction).

Off-axis image plane hologram compression in holographic tomography - metrological assessment

In this paper, we present a novel study on the impact of lossy data compression on the metrological properties of holographic tomography reconstruction of the refractive index (RI). We use a spatial bandwidth-optimized compression procedure that leverages the properties of image plane off-axis holograms and standardized compression codecs, both widely applied in research and industry. The compression procedure is tested at multiple bitrates, for four different objects and against three reconstruction algorithms. The metrological evaluation is primarily done by comparison to the reconstruction from original data using the root-mean-squared error (RMSE). We show that due to differences between objects and different noise sensitivities of the reconstruction algorithms, the rate-distortion behaviour varies, but in most cases allows for the compression below 1 bit per pixel, while maintaining an RI RMSE less than 10^-4.

Roadmap on Digital Holography-Based Quantitative Phase Imaging

Quantitative Phase Imaging (QPI) provides unique means for the imaging of biological or technical microstructures, merging beneficial features identified with microscopy, interferometry, holography, and numerical computations. This roadmap article reviews several digital holography-based QPI approaches developed by prominent research groups. It also briefly discusses the present and future perspectives of 2D and 3D QPI research based on digital holographic microscopy, holographic tomography, and their applications.

Tomographic refractive index profiling of direct laser written waveguides

The fabrication of complex integrated photonic devices via direct laser writing is a powerful and rapidly developing technology. However, the approach is still facing several challenges. One of them is the reliable quantitative characterization of refractive index (RI) changes induced upon laser exposure. To this end, we develop a tomographic reconstruction algorithm following a modern optimization approach, relying on accelerated proximal gradient descent, based on intensity images only. Very recently, such algorithms have become the state of the art in the community of bioimaging, but have never been applied to direct laser written structures such as waveguides. We adapt the algorithm to our concern of characterizing these translation-invariant structures and extend it in order to jointly estimate the aberrations introduced by the imaging system. We show that a correct estimation of these aberrations is necessary to make use of data recorded at larger angles and that it can increase the fidelity of the reconstructed RI profiles. Moreover, we present a method allowing to cross-validate the RI reconstructions by comparing en-face widefield images of thin waveguide sections with matching simulations based on the retrieved RI profile.

High speed, complex wavefront shaping using the digital micro-mirror device

Digital micro-mirror devices (DMDs) have been deployed in many optical applications. As compared to spatial light modulators (SLMs), they are characterized by their much faster refresh rates (full-frame refresh rates up to 32 kHz for binary patterns) compared to 120 Hz for most liquid crystal SLMs. DMDs however can only display binary, unipolar patterns and utilize temporal modulation to represent with excellent accuracy multiple gray-levels in display applications. We used the built-in time domain dynamic range representation of the DMD to project 8-bit complex-fields. With this method, we demonstrated 8-bit complex field modulation with a frame time of 38.4 ms (around 0.15 s for the entire complex-field). We performed phase conjugation by compensating the distortions incurred due to propagation through free-space and a scattering medium. For faster modulation speed, an electro-optic modulator was used in synchronization with the DMD in an amplitude modulation mode to create grayscale patterns with frame rate ~ 833 Hz with display time of only 1.2 ms instead of 38.4 ms for time multiplexing gaining a speed up by a factor of 32.

Optimizing sample illumination scanning for reflection and 4Pi tomographic diffractive microscopy

Tomographic diffractive microscopy (TDM) is increasingly gaining attention, owing to its high-resolution, label-free imaging capability. Fast acquisitions necessitate limiting the number of holograms to be recorded. Reconstructions then rely on optimal Fourier space filling to retain image quality and resolution, that is, they rely on optimal scanning of the tomographic illuminations. In this work, we theoretically study reflection TDM, and then the 4Pi TDM, a combination of transmission and reflection systems. Image simulations are conducted to determine optimal angular sweeping. We found that three-dimensional uniform scanning fills Fourier space the best for both reflection and 4Pi configurations, providing a better refractive index estimation for the observed sample.

Robust Phase Unwrapping via Deep Image Prior for Quantitative Phase Imaging

Quantitative phase imaging (QPI) is an emerging label-free technique that produces images containing morphological and dynamical information without contrast agents. Unfortunately, the phase is wrapped in most imaging system. Phase unwrapping is the computational process that recovers a more informative image. It is particularly challenging with thick and complex samples such as organoids. Recent works that rely on supervised training show that deep learning is a powerful method to unwrap the phase; however, supervised approaches require large and representative datasets which are difficult to obtain for complex biological samples. Inspired by the concept of deep image priors, we propose a deep-learning-based method that does not need any training set. Our framework relies on an untrained convolutional neural network to accurately unwrap the phase while ensuring the consistency of the measurements. We experimentally demonstrate that the proposed method faithfully recovers the phase of complex samples on both real and simulated data. Our work paves the way to reliable phase imaging of thick and complex samples with QPI.

Intracellular Detection and Localization of Nanoparticles by Refractive Index Measurement

The measuring of nanoparticle toxicity faces an important limitation since it is based on metrics exposure, the concentration at which cells are exposed instead the true concentration inside the cells. In vitro studies of nanomaterials would benefit from the direct measuring of the true intracellular dose of nanoparticles. The objective of the present study was to state whether the intracellular detection of nanodiamonds is possible by measuring the refractive index. Based on optical diffraction tomography of treated live cells, the results show that unlabeled nanoparticles can be detected and localized inside cells. The results were confirmed by fluorescence measurements. Optical diffraction tomography paves the way to measuring the true intracellular concentrations and the localization of nanoparticles which will improve the dose-response paradigm of pharmacology and toxicology in the field of nanomaterials.

Label-free screening of brain tissue myelin content using phase imaging with computational specificity (PICS)

Inadequate myelination in the central nervous system is associated with neurodevelopmental complications. Thus, quantitative, high spatial resolution measurements of myelin levels are highly desirable. We used spatial light interference microcopy (SLIM), a highly sensitive quantitative phase imaging (QPI) technique, to correlate the dry mass content of myelin in piglet brain tissue with dietary changes and gestational size. We combined SLIM micrographs with an artificial intelligence (AI) classifying model that allows us to discern subtle disparities in myelin distributions with high accuracy. This concept of combining QPI label-free data with AI for the purpose of extracting molecular specificity has recently been introduced by our laboratory as phase imaging with computational specificity. Training on 8000 SLIM images of piglet brain tissue with the 71-layer transfer learning model Xception, we created a two-parameter classification to differentiate gestational size and diet type with an accuracy of 82% and 80%, respectively. To our knowledge, this type of evaluation is impossible to perform by an expert pathologist or other techniques.

Common-path intrinsically achromatic optical diffraction tomography

In this work we propose an open-top like common-path intrinsically achromatic optical diffraction tomography system. It operates as a total-shear interferometer and employs Ronchi-type amplitude diffraction grating, positioned in between the camera and the tube lens without an additional 4f system, generating three-beam interferograms with achromatic second harmonic. Such configuration makes the proposed system low cost, compact and immune to vibrations. We present the results of the measurements of 3D-printed cell phantom using laser diode (coherent) and superluminescent diode (partially coherent) light sources. Broadband light sources can be naturally employed without the need for any cumbersome compensation because of the intrinsic achromaticity of the interferometric recording (holograms generated by -1st and +1st conjugated diffraction orders are not affected by the illumination wavelength). The results show that the decreased coherence offers much reduced coherent noise and higher fidelity tomographic reconstruction especially when applied nonnegativity constraint regularization procedure.