3-D visualization and identification of biological microorganisms using partially temporal incoherent light in-line computational holographic imaging

Multi-phase FZA lensless imaging via diffusion model

Lensless imaging shifts the burden of imaging from bulky and expensive hardware to computing, which enables new architectures for portable cameras. However, the twin image effect caused by the missing phase information in the light wave is a key factor limiting the quality of lensless imaging. Conventional single-phase encoding methods and independent reconstruction of separate channels pose challenges in removing twin images and preserving the color fidelity of the reconstructed image. In order to achieve high-quality lensless imaging, the multiphase lensless imaging via diffusion model (MLDM) is proposed. A multi-phase FZA encoder integrated on a single mask plate is used to expand the data channel of a single-shot image. The information association between the color image pixel channel and the encoded phase channel is established by extracting prior information of the data distribution based on multi-channel encoding. Finally, the reconstruction quality is improved through the use of the iterative reconstruction method. The results show that the proposed MLDM method effectively removes the influence of twin images and produces high-quality reconstructed images compared with traditional methods, and the results reconstructed using MLDM have higher structural similarity and peak signal-to-noise ratio.

Roadmap on digital holography [Invited]

This Roadmap article on digital holography provides an overview of a vast array of research activities in the field of digital holography. The paper consists of a series of 25 sections from the prominent experts in digital holography presenting various aspects of the field on sensing, 3D imaging and displays, virtual and augmented reality, microscopy, cell identification, tomography, label-free live cell imaging, and other applications. Each section represents the vision of its author to describe the significant progress, potential impact, important developments, and challenging issues in the field of digital holography.

Noise-free quantitative phase imaging in Gabor holography with conditional generative adversarial network

This paper shows that deep learning can eliminate the superimposed twin-image noise in phase images of Gabor holographic setup. This is achieved by the conditional generative adversarial model (C-GAN), trained by input-output pairs of noisy phase images obtained from synthetic Gabor holography and the corresponding quantitative noise-free contrast-phase image obtained by the off-axis digital holography. To train the model, Gabor holograms are generated from digital off-axis holograms with spatial shifting of the real image and twin image in the frequency domain and then adding them with the DC term in the spatial domain. Finally, the digital propagation of the Gabor hologram with Fresnel approximation generates a super-imposed phase image for the C-GAN model input. Two models were trained: a human red blood cell model and an elliptical cancer cell model. Following the training, several quantitative analyses were conducted on the bio-chemical properties and similarity between actual noise-free phase images and the model output. Surprisingly, it is discovered that our model can recover other elliptical cell lines that were not observed during the training. Additionally, some misalignments can also be compensated with the trained model. Particularly, if the reconstruction distance is somewhat incorrect, this model can still retrieve in-focus images.

Automated single cardiomyocyte characterization by nucleus extraction from dynamic holographic images using a fully convolutional neural network

Human-induced pluripotent stem cell-derived cardiomyocytes (hiPS-CMs) beating can be efficiently characterized by time-lapse quantitative phase imaging (QPIs) obtained by digital holographic microscopy. Particularly, the CM's nucleus section can precisely reflect the associated rhythmic beating pattern of the CM suitable for subsequent beating pattern characterization. In this paper, we describe an automated method to characterize single CMs by nucleus extraction from QPIs and subsequent beating pattern reconstruction and quantification. However, accurate CM's nucleus extraction from the QPIs is a challenging task due to the variations in shape, size, orientation, and lack of special geometry. To this end, we propose a novel fully convolutional neural network (FCN)-based network architecture for accurate CM's nucleus extraction using pixel classification technique and subsequent beating pattern characterization. Our experimental results show that the beating profile of multiple extracted single CMs is less noisy and more informative compared to the whole image slide. Applying this method allows CM characterization at the single-cell level. Consequently, several single CMs are extracted from the whole slide QPIs and multiple parameters regarding their beating profile of each isolated CM are efficiently measured.

Quantitative analysis of three-dimensional morphology and membrane dynamics of red blood cells during temperature elevation

The optimal functionality of red blood cells is closely associated with the surrounding environment. This study was undertaken to analyze the changes in membrane profile, mean corpuscular hemoglobin (MCH), and cell membrane fluctuations (CMF) of healthy red blood cells (RBC) at varying temperatures. The temperature was elevated from 17 °C to 41 °C within a duration of less than one hour, and the holograms were recorded by an off-axis configuration. After hologram reconstruction, we extracted single RBCs and evaluated their morphologically related features (projected surface area and sphericity coefficient), MCH, and CMF. We observed that elevating the temperature results in changes in the three-dimensional (3D) profile. Since CMF amplitude is highly correlated to the bending curvature of RBC membrane, temperature-induced shape changes can alter CMF’s map and amplitude; mainly larger fluctuations appear on dimple area at a higher temperature. Regardless of the shape changes, no alterations in MCH were seen with temperature variation.

Accurate color imaging of pathology slides using holography and absorbance spectrum estimation of histochemical stains

Holographic microscopy presents challenges for color reproduction due to the usage of narrow-band illumination sources, which especially impacts the imaging of stained pathology slides for clinical diagnoses. Here, an accurate color holographic microscopy framework using absorbance spectrum estimation is presented. This method uses multispectral holographic images acquired and reconstructed at a small number (e.g., three to six) of wavelengths, estimates the absorbance spectrum of the sample, and projects it onto a color tristimulus. Using this method, the wavelength selection is optimized to holographically image 25 pathology slide samples with different tissue and stain combinations to significantly reduce color errors in the final reconstructed images. The results can be used as a practical guide for various imaging applications and, in particular, to correct color distortions in holographic imaging of pathology samples spanning different dyes and tissue types.

Automated quantification study of human cardiomyocyte synchronization using holographic imaging

This paper investigates the rhythm strip and parameters of synchronization of human induced pluripotent stem cell (iPS) derived cardiomyocytes. The synchronization is evaluated from quantitative phase images of beating cardiomyocytes which are obtained using the time-lapse digital holographic imaging method. By quantitatively monitoring the dry mass redistribution, digital holography provides the physical contraction-relaxation signal caused by autonomous cardiac action potential. In order to analyze the synchronicity at the cell-to-cell level, we extracted single cardiac muscle cells, which contain the nuclei, from the phase images of cardiomyocytes containing multiple cells resulting from the fusion of k-means clustering and watershed segmentation algorithms. We demonstrate that mature cardiomyocyte cell synchronization can be automatically evaluated by time-lapse microscopic holographic imaging. Our proposed method can be applied for studies on cardiomyocyte disorders and drug safety testing systems.

Quantification of stored red blood cell fluctuations by time-lapse holographic cell imaging

We propose methods to quantitatively calculate the fluctuation rate of red blood cells with nanometric axial and millisecond temporal sensitivity at the single-cell level by using time-lapse holographic cell imaging. For this quantitative analysis, cell membrane fluctuations (CMFs) were measured for RBCs stored at different storage times. Measurements were taken over the whole membrane for both the ring and dimple sections separately. The measurements show that healthy RBCs that maintain their discocyte shape become stiffer with storage time. The correlation analysis demonstrates a significant negative correlation between CMFs and the sphericity coefficient, which characterizes the morphological type of erythrocyte. In addition, we show the correlation results between CMFs and other morphological properties such as projected surface area, surface area, mean corpuscular volume, and mean corpuscular hemoglobin.

Automated red blood cells extraction from holographic images using fully convolutional neural networks

In this paper, we present two models for automatically extracting red blood cells (RBCs) from RBCs holographic images based on a deep learning fully convolutional neural network (FCN) algorithm. The first model, called FCN-1, only uses the FCN algorithm to carry out RBCs prediction, whereas the second model, called FCN-2, combines the FCN approach with the marker-controlled watershed transform segmentation scheme to achieve RBCs extraction. Both models achieve good segmentation accuracy. In addition, the second model has much better performance in terms of cell separation than traditional segmentation methods. In the proposed methods, the RBCs phase images are first numerically reconstructed from RBCs holograms recorded with off-axis digital holographic microscopy. Then, some RBCs phase images are manually segmented and used as training data to fine-tune the FCN. Finally, each pixel in new input RBCs phase images is predicted into either foreground or background using the trained FCN models. The RBCs prediction result from the first model is the final segmentation result, whereas the result from the second model is used as the internal markers of the marker-controlled transform algorithm for further segmentation. Experimental results show that the given schemes can automatically extract RBCs from RBCs phase images and much better RBCs separation results are obtained when the FCN technique is combined with the marker-controlled watershed segmentation algorithm.

3D imaging of optically cleared tissue using a simplified CLARITY method and on-chip microscopy

High-throughput sectioning and optical imaging of tissue samples using traditional immunohistochemical techniques can be costly and inaccessible in resource-limited areas. We demonstrate three-dimensional (3D) imaging and phenotyping in optically transparent tissue using lens-free holographic on-chip microscopy as a low-cost, simple, and high-throughput alternative to conventional approaches. The tissue sample is passively cleared using a simplified CLARITY method and stained using 3,3'-diaminobenzidine to target cells of interest, enabling bright-field optical imaging and 3D sectioning of thick samples. The lens-free computational microscope uses pixel super-resolution and multi-height phase recovery algorithms to digitally refocus throughout the cleared tissue and obtain a 3D stack of complex-valued images of the sample, containing both phase and amplitude information. We optimized the tissue-clearing and imaging system by finding the optimal illumination wavelength, tissue thickness, sample preparation parameters, and the number of heights of the lens-free image acquisition and implemented a sparsity-based denoising algorithm to maximize the imaging volume and minimize the amount of the acquired data while also preserving the contrast-to-noise ratio of the reconstructed images. As a proof of concept, we achieved 3D imaging of neurons in a 200-μm-thick cleared mouse brain tissue over a wide field of view of 20.5 mm2. The lens-free microscope also achieved more than an order-of-magnitude reduction in raw data compared to a conventional scanning optical microscope imaging the same sample volume. Being low cost, simple, high-throughput, and data-efficient, we believe that this CLARITY-enabled computational tissue imaging technique could find numerous applications in biomedical diagnosis and research in low-resource settings.

Holographic deep learning for rapid optical screening of anthrax spores

Establishing early warning systems for anthrax attacks is crucial in biodefense. Despite numerous studies for decades, the limited sensitivity of conventional biochemical methods essentially requires preprocessing steps and thus has limitations to be used in realistic settings of biological warfare. We present an optical method for rapid and label-free screening of Bacillus anthracis spores through the synergistic application of holographic microscopy and deep learning. A deep convolutional neural network is designed to classify holographic images of unlabeled living cells. After training, the network outperforms previous techniques in all accuracy measures, achieving single-spore sensitivity and subgenus specificity. The unique "representation learning" capability of deep learning enables direct training from raw images instead of manually extracted features. The method automatically recognizes key biological traits encoded in the images and exploits them as fingerprints. This remarkable learning ability makes the proposed method readily applicable to classifying various single cells in addition to B. anthracis, as demonstrated for the diagnosis of Listeria monocytogenes, without any modification. We believe that our strategy will make holographic microscopy more accessible to medical doctors and biomedical scientists for easy, rapid, and accurate point-of-care diagnosis of pathogens.

Supervised classification of etoposide-treated in vitro adherent cells based on noninvasive imaging morphology

Single-cell studies using noninvasive imaging is a challenging, yet appealing way to study cellular characteristics over extended periods of time, for instance to follow cell interactions and the behavior of different cell types within the same sample. In some cases, e.g., transplantation culturing, real-time cellular monitoring, stem cell studies, in vivo studies, and embryo growth studies, it is also crucial to keep the sample intact and invasive imaging using fluorophores or dyes is not an option. Computerized methods are needed to improve throughput of image-based analysis and for use with noninvasive microscopy such methods are poorly developed. By combining a set of well-documented image analysis and classification tools with noninvasive microscopy, we demonstrate the ability for long-term image-based analysis of morphological changes in single cells as induced by a toxin, and show how these changes can be used to indicate changes in biological function. In this study, adherent cell cultures of DU-145 treated with low-concentration (LC) etoposide were imaged during 3 days. Single cells were identified by image segmentation and subsequently classified on image features, extracted for each cell. In parallel with image analysis, an MTS assay was performed to allow comparison between metabolic activity and morphological changes after long-term low-level drug response. Results show a decrease in proliferation rate for LC etoposide, accompanied by changes in cell morphology, primarily leading to an increase in cell area and textural changes. It is shown that changes detected by image analysis are already visible on day 1 for [Formula: see text] etoposide, whereas effects on MTS and viability are detected only on day 3 for [Formula: see text] etoposide concentration, leading to the conclusion that the morphological changes observed occur before and at lower concentrations than a reduction in cell metabolic activity or viability. Three classifiers are compared and we report a best case sensitivity of 88% and specificity of 94% for classification of cells as treated/untreated.

Cell identification using single beam lensless imaging with pseudo-random phase encoding

In this Letter, we propose a novel compact optical system for automated cell identification. Our system employs pseudo-random encoding of the light modulated by the cells under inspection to capture the unique opto-biological signature of the micro-organisms by an image sensor and without using a microscope objective lens to magnify the object beam. The proposed instrument can be fabricated using a compact light source, a thin diffuser, and an image sensor connected to computational hardware; thus, it can be compact and cost effective. Experiments are presented using the proposed system to identify and classify various micro-objects and demonstrate proof of concept. The captured opto-biological signature pattern can be attributed to the micro-object's morphology, size, sub-cellular complex structure, index of refraction, internal material composition, etc. Using the captured signature of the micro-object, we extract statistical features such as mean, variance, skewness, kurtosis, entropy, and correlation coefficients for cell identification using the random forest classifier. For comparison, similar identification experiments were repeated with a digital shearing interferometer. To the best of our knowledge, this is the first report on automated cell identification using the proposed approach.

Cell morphology-based classification of red blood cells using holographic imaging informatics

We present methods that automatically select a linear or nonlinear classifier for red blood cell (RBC) classification by analyzing the equality of the covariance matrices in Gabor-filtered holographic images. First, the phase images of the RBCs are numerically reconstructed from their holograms, which are recorded using off-axis digital holographic microscopy (DHM). Second, each RBC is segmented using a marker-controlled watershed transform algorithm and the inner part of the RBC is identified and analyzed. Third, the Gabor wavelet transform is applied to the segmented cells to extract a series of features, which then undergo a multivariate statistical test to evaluate the equality of the covariance matrices of the different shapes of the RBCs using selected features. When these covariance matrices are not equal, a nonlinear classification scheme based on quadratic functions is applied; otherwise, a linear classification is applied. We used the stomatocyte, discocyte, and echinocyte RBC for classifier training and testing. Simulation results demonstrated that 10 of the 14 RBC features are useful in RBC classification. Experimental results also revealed that the covariance matrices of the three main RBC groups are not equal and that a nonlinear classification method has a much lower misclassification rate. The proposed automated RBC classification method has the potential for use in drug testing and the diagnosis of RBC-related diseases.

Label-free identification of individual bacteria using Fourier transform light scattering

Rapid identification of bacterial species is crucial in medicine and food hygiene. In order to achieve rapid and label-free identification of bacterial species at the single bacterium level, we propose and experimentally demonstrate an optical method based on Fourier transform light scattering (FTLS) measurements and statistical classification. For individual rod-shaped bacteria belonging to four bacterial species (Listeria monocytogenes, Escherichia coli, Lactobacillus casei, and Bacillus subtilis), two-dimensional angle-resolved light scattering maps are precisely measured using FTLS technique. The scattering maps are then systematically analyzed, employing statistical classification in order to extract the unique fingerprint patterns for each species, so that a new unidentified bacterium can be identified by a single light scattering measurement. The single-bacterial and label-free nature of our method suggests wide applicability for rapid point-of-care bacterial diagnosis.

Wide-field optical detection of nanoparticles using on-chip microscopy and self-assembled nanolenses

The direct observation of nanoscale objects is a challenging task for optical microscopy because the scattering from an individual nanoparticle is typically weak at optical wavelengths. Electron microscopy therefore remains one of the gold standard visualization methods for nanoparticles, despite its high cost, limited throughput and restricted field-of-view. Here, we describe a high-throughput, on-chip detection scheme that uses biocompatible wetting films to self-assemble aspheric liquid nanolenses around individual nanoparticles to enhance the contrast between the scattered and background light. We model the effect of the nanolens as a spatial phase mask centred on the particle and show that the holographic diffraction pattern of this effective phase mask allows detection of sub-100 nm particles across a large field-of-view of >20 mm2. As a proof-of-concept demonstration, we report on-chip detection of individual polystyrene nanoparticles, adenoviruses and influenza A (H1N1) viral particles.

Early cell death detection with digital holographic microscopy

BACKGROUND

Digital holography provides a non-invasive measurement of the quantitative phase shifts induced by cells in culture, which can be related to cell volume changes. It has been shown previously that regulation of cell volume, in particular as it relates to ionic homeostasis, is crucially involved in the activation/inactivation of the cell death processes. We thus present here an application of digital holographic microscopy (DHM) dedicated to early and label-free detection of cell death.

METHODS AND FINDINGS

We provide quantitative measurements of phase signal obtained on mouse cortical neurons, and caused by early neuronal cell volume regulation triggered by excitotoxic concentrations of L-glutamate. We show that the efficiency of this early regulation of cell volume detected by DHM, is correlated with the occurrence of subsequent neuronal death assessed with the widely accepted trypan blue method for detection of cell viability.

CONCLUSIONS

The determination of the phase signal by DHM provides a simple and rapid optical method for the early detection of cell death.

Partially coherent lensfree tomographic microscopy [Invited]

Optical sectioning of biological specimens provides detailed volumetric information regarding their internal structure. To provide a complementary approach to existing three-dimensional (3D) microscopy modalities, we have recently demonstrated lensfree optical tomography that offers high-throughput imaging within a compact and simple platform. In this approach, in-line holograms of objects at different angles of partially coherent illumination are recorded using a digital sensor-array, which enables computing pixel super-resolved tomographic images of the specimen. This imaging modality, which forms the focus of this review, offers micrometer-scale 3D resolution over large imaging volumes of, for example, 10-15 mm(3), and can be assembled in light weight and compact architectures. Therefore, lensfree optical tomography might be particularly useful for lab-on-a-chip applications as well as for microscopy needs in resource-limited settings.

Optofluidic system for three-dimensional sensing and identification of micro-organisms with digital holographic microscopy

Optofluidic devices offer flexibility for a variety of tasks involving biological specimen. We propose a system for three-dimensional (3D) sensing and identification of biological micro-organisms. This system consists of a microfluidic device along with a digital holographic microscope and relevant statistical recognition algorithms. The microfluidic channel is used to house the micro-organisms, while the holographic microscope and a CCD camera record their digital holograms. The holograms can be computationally reconstructed in 3D using a variety of algorithms, such as the Fresnel transform. Statistical recognition algorithms are used to analyze and identify the micro-organisms from the reconstructed wavefront. Experimental results are presented. Because of computational reconstruction of wavefronts in holographic imaging, this technique offers unique advantages that allow one to image micro-organisms within a deep channel while removing the inherent microfluidic-induced aberration through interferometery.

Automated three-dimensional microbial sensing and recognition using digital holography and statistical sampling

We overview an approach to providing automated three-dimensional (3D) sensing and recognition of biological micro/nanoorganisms integrating Gabor digital holographic microscopy and statistical sampling methods. For 3D data acquisition of biological specimens, a coherent beam propagates through the specimen and its transversely and longitudinally magnified diffraction pattern observed by the microscope objective is optically recorded with an image sensor array interfaced with a computer. 3D visualization of the biological specimen from the magnified diffraction pattern is accomplished by using the computational Fresnel propagation algorithm. For 3D recognition of the biological specimen, a watershed image segmentation algorithm is applied to automatically remove the unnecessary background parts in the reconstructed holographic image. Statistical estimation and inference algorithms are developed to the automatically segmented holographic image. Overviews of preliminary experimental results illustrate how the holographic image reconstructed from the Gabor digital hologram of biological specimen contains important information for microbial recognition.

Parallel optical-path-length-shifting digital holography

The authors propose an optical-path-length-shifting digital holography as a technique capable of single-shot recording of three-dimensional information of objects. With a single image sensor, the proposed technique can simultaneously record all of the holograms required for the in-line digital holography that reconstruct the image of an object from two intensity measurements at different planes. The technique can be optically implemented by using an optical-path-length-shifting array device located in the common path of the reference and object waves. The array device has periodic structure of two-step optical-path difference. The configuration of the array device of the proposed technique is simpler than the phase-shifting array device required for parallel phase-shifting digital holographies. Therefore, the optical system of the proposed technique is more suitable for the realization of a single-shot in-line digital holography system that removes the conjugate image from the reconstructed image. The authors conducted both a numerical simulation and a preliminary experiment of the proposed technique. The reconstructed images were quantitatively evaluated by using root mean squared error. In comparison to single-shot digital holography using the Fresnel transform alone, with the proposed technique the root mean squared errors of the technique were reduced to less than 1/6 in amplitude and 1/3 in phase. Also the results of the simulation and experiment agreed well with the images of an object. Thus the effectiveness of the proposed technique is verified.