Super-resolution three-dimensional fluorescence and optical diffraction tomography of live cells using structured illumination generated by a digital micromirror device

Utilizing quantitative phase microscopy to localize fluorescence in three dimensions via the transport of intensity equation

We demonstrate the use of a novel, to the best of our knowledge, localization algorithm for digitally refocusing fluorescence images from a three-dimensional cell culture. Simultaneous phase and fluorescence intensity images are collected through a multimodal system that combines digital holography via quantitative phase microscopy (QPM) and fluorescence microscopy. Defocused fluorescence images are localized to a specific z-plane within the three-dimensional (3D) matrix using the transport of intensity equation (TIE) and depth-resolved information derived from the QPM measurements. This technique is applied to cells stained with different fluorescent tags suspended in 3D collagen hydrogel cultures. Experimental findings demonstrate the localization of defocused images, facilitating the analysis and comparison of cells within the hydrogel matrix. This method holds promise for comprehensive cellular imaging of fluorescence labeling in three-dimensional environments, enabling detailed investigations into cellular behavior and interactions.

Quantitative phase imaging based on holography: trends and new perspectives

In 1948, Dennis Gabor proposed the concept of holography, providing a pioneering solution to a quantitative description of the optical wavefront. After 75 years of development, holographic imaging has become a powerful tool for optical wavefront measurement and quantitative phase imaging. The emergence of this technology has given fresh energy to physics, biology, and materials science. Digital holography (DH) possesses the quantitative advantages of wide-field, non-contact, precise, and dynamic measurement capability for complex-waves. DH has unique capabilities for the propagation of optical fields by measuring light scattering with phase information. It offers quantitative visualization of the refractive index and thickness distribution of weak absorption samples, which plays a vital role in the pathophysiology of various diseases and the characterization of various materials. It provides a possibility to bridge the gap between the imaging and scattering disciplines. The propagation of wavefront is described by the complex amplitude. The complex-value in the complex-domain is reconstructed from the intensity-value measurement by camera in the real-domain. Here, we regard the process of holographic recording and reconstruction as a transformation between complex-domain and real-domain, and discuss the mathematics and physical principles of reconstruction. We review the DH in underlying principles, technical approaches, and the breadth of applications. We conclude with emerging challenges and opportunities based on combining holographic imaging with other methodologies that expand the scope and utility of holographic imaging even further. The multidisciplinary nature brings technology and application experts together in label-free cell biology, analytical chemistry, clinical sciences, wavefront sensing, and semiconductor production.

Single-shot quantitative phase-fluorescence imaging using cross-grating wavefront microscopy

The article introduces an optical microscopy technique capable of simultaneously acquiring quantitative fluorescence and phase (or equivalently wavefront) images with a single camera sensor, avoiding any delay between both images, or registration of images acquired separately. The method is based on the use of a 2-dimensional diffraction grating (aka cross-grating) positioned at a millimeter distance from a 2-color camera. Fluorescence and wavefront images are extracted from the two color channels of the camera, and retrieved by image demodulation. The applicability of the method is illustrated on various samples, namely fluorescent micro-beads, bacteria and mammalian cells.

High-resolution assessment of multidimensional cellular mechanics using label-free refractive-index traction force microscopy

A critical requirement for studying cell mechanics is three-dimensional assessment of cellular shapes and forces with high spatiotemporal resolution. Traction force microscopy with fluorescence imaging enables the measurement of cellular forces, but it is limited by photobleaching and a slow acquisition speed. Here, we present refractive-index traction force microscopy (RI-TFM), which simultaneously quantifies the volumetric morphology and traction force of cells using a high-speed illumination scheme with 0.5-Hz temporal resolution. Without labelling, our method enables quantitative analyses of dry-mass distributions and shear (in-plane) and normal (out-of-plane) tractions of single cells on the extracellular matrix. When combined with a constrained total variation-based deconvolution algorithm, it provides 0.55-Pa shear and 1.59-Pa normal traction sensitivity for a 1-kPa hydrogel substrate. We demonstrate its utility by assessing the effects of compromised intracellular stress and capturing the rapid dynamics of cellular junction formation in the spatiotemporal changes in non-planar traction components.

Quantitative phase imaging for characterization of single cell growth dynamics

Quantitative phase imaging (QPI) has emerged as an indispensable tool in the field of biomedicine, offering the ability to obtain quantitative maps of phase changes due to optical path length delays without the need for contrast agents. These maps provide valuable information about cellular morphology and dynamics, unperturbed by the introduction of exogenous substances. In this review, a summary of recent studies that have focused on elucidating the growth dynamics of individual cells using QPI is presented. Specifically, investigations into cellular changes occurring during mitosis, the differentiation of cellular organelles, the assessment of distinct cell death processes (i.e., apoptosis, necrosis, and oncosis) and the precise measurement of live cell temperature are explored. Furthermore, the captivating applications of QPI in theragnostics, where its potential for transformative impact is prominently showcased, are highlighted. Finally, the challenges that need to be overcome for its wider adoption and successful integration into biomedical research are outlined.

Structured illumination phase and fluorescence microscopy for bioimaging

This study presents a dual-modality microscopic imaging approach that combines quantitative phase microscopy and fluorescence microscopy based on structured illumination (SI) to provide structural and functional information for the same sample. As the first imaging modality, structured illumination digital holographic microscopy (SI-DHM) is implemented along the transmission beam path. SI-DHM acts as a label-free, noninvasive approach and provides high-contrast and quantitative phase images utilizing the refractive index contrast of the inner structures of samples against the background. As the second imaging modality, structured illumination (fluorescence) microscopy (SIM) is constructed along the reflection beam path. SIM utilizes fluorescent labeling and provides super-resolution images for specific functional structures of samples. We first experimentally demonstrated phase imaging of SI-DHM on rice leaves and fluorescence (SIM) imaging on mouse kidney sections. Then, we demonstrated dual-modality imaging of biological samples, using DHM to acquire the overall cell morphology and SIM to obtain specific functional structures. These results prove that the proposed technique is of great importance in biomedical studies, such as providing insight into cell physiology by visualizing and quantifying subcellular structures.

Multimodal study of CHI3L1 inhibition and its effect on angiogenesis, migration, immune response and refractive index of cellular structures in glioblastoma

Glioblastoma is one of the most aggressive tumours with a poor response to treatment and a poor prognosis for patients. One of the proteins expressed in glioblastoma tissue is CHI3L1 (YKL-40), which is upregulated and known for its angiogenesis-supporting and pro-tumour immunomodulatory effects in a variety of cancers. In this paper we present the anti-angiogenic, anti-migratory and immunomodulatory effects of the compound G721-0282, an inhibitor of CHI3L1. The inhibitor-induced changes were investigated using conventional techniques as well as the novel label-free digital holographic tomography (DHT), a quantitative phase imaging technique that allows the reconstruction of the refractive index (RI), which is used as an image contrast for 3D visualisation of living cells. DHT allowed digital staining of individual cells and intercellular structures based only on their specific RI. Quantitative spatially resolved analysis of the RI data shows that the concentration of G721-0282 leads to significant changes in the density of cells and their intracellular structures (in particular the cytoplasm and nucleus), in the volume of lipid droplets and in protein concentrations. Studies in the U-87 MG glioblastoma cell line, THP-1 monocytes differentiated into macrophages, human microvascular endothelial cells (HMEC-1) and in the spheroid model of glioblastoma composed of U-87 MG, HMEC-1 and macrophages suggest that inhibition of CHI3L1 may have potential in the antitumour treatment of glioblastoma. In this paper, we also propose a spheroid model for in vitro studies that mimics this type of tumour.

Three-dimensional bi-functional refractive index and fluorescence microscopy (BRIEF)

Fluorescence microscopy is a powerful tool for imaging biological samples with molecular specificity. In contrast, phase microscopy provides label-free measurement of the sample's refractive index (RI), which is an intrinsic optical property that quantitatively relates to cell morphology, mass, and stiffness. Conventional imaging techniques measure either the labeled fluorescence (functional) information or the label-free RI (structural) information, though it may be valuable to have both. For example, biological tissues have heterogeneous RI distributions, causing sample-induced scattering that degrades the fluorescence image quality. When both fluorescence and 3D RI are measured, one can use the RI information to digitally correct multiple-scattering effects in the fluorescence image. Here, we develop a new computational multi-modal imaging method based on epi-mode microscopy that reconstructs both 3D fluorescence and 3D RI from a single dataset. We acquire dozens of fluorescence images, each 'illuminated' by a single fluorophore, then solve an inverse problem with a multiple-scattering forward model. We experimentally demonstrate our method for epi-mode 3D RI imaging and digital correction of multiple-scattering effects in fluorescence images.

Multi-moded high-index contrast optical waveguide for super-contrast high-resolution label-free microscopy

The article elucidates the physical mechanism behind the generation of superior-contrast and high-resolution label-free images using an optical waveguide. Imaging is realized by employing a high index contrast multi-moded waveguide as a partially coherent light source. The modes provide near-field illumination of unlabeled samples, thereby repositioning the higher spatial frequencies of the sample into the far-field. These modes coherently scatter off the sample with different phases and are engineered to have random spatial distributions within the integration time of the camera. This mitigates the coherent speckle noise and enhances the contrast (2-10) × as opposed to other imaging techniques. Besides, the coherent scattering of the different modes gives rise to fluctuations in intensity. The technique demonstrated here is named chip-based Evanescent Light Scattering (cELS). The concepts introduced through this work are described mathematically and the high-contrast image generation process using a multi-moded waveguide as the light source is explained. The article then explores the feasibility of utilizing fluctuations in the captured images along with fluorescence-based techniques, like intensity-fluctuation algorithms, to mitigate poor-contrast and diffraction-limited resolution in the coherent imaging regime. Furthermore, a straight waveguide is demonstrated to have limited angular diversity between its multiple modes and therefore, for isotropic sample illumination, a multiple-arms waveguide geometry is used. The concepts introduced are validated experimentally via high-contrast label-free imaging of weakly scattering nanosized specimens such as extra-cellular vesicles (EVs), liposomes, nanobeads and biological cells such as fixed and live HeLa cells.

Optical diffraction tomography of 3D microstructures using a low coherence source

Optical diffraction tomography (ODT) is a label-free technique for three dimensional imaging of micron-sized objects. Coherence and limited sampling of 3D Fourier space are often responsible for the appearance of artifacts. Here we present an ODT microscope that uses low temporal coherence light and spatial light modulators to retrieve reliable 3D maps of the refractive index. A common-path interferometer, based on a spatial light modulator, measures the complex fields transmitted by a sample. Measured fields, acquired while scanning the illumination direction using a digital micro-mirror device, are fed into a Rytov reconstruction algorithm to obtain refractive index maps whose accuracy is directly evaluated on microfabricated 3D test objects. Even for challenging shapes such as pyramids, bridges, and dumbbells, we obtain volumetric reconstructions that compare very well with electron microscopy images.

Quantitative Stimulated Raman Scattering Microscopy: Promises and Pitfalls

Since its first demonstration, stimulated Raman scattering (SRS) microscopy has become a powerful chemical imaging tool that shows promise in numerous biological and biomedical applications. The spectroscopic capability of SRS enables identification and tracking of specific molecules or classes of molecules, often without labeling. SRS microscopy also has the hallmark advantage of signal strength that is directly proportional to molecular concentration, allowing for in situ quantitative analysis of chemical composition of heterogeneous samples with submicron spatial resolution and subminute temporal resolution. However, it is important to recognize that quantification through SRS microscopy requires assumptions regarding both system and sample. Such assumptions are often taken axiomatically, which may lead to erroneous conclusions without proper validation. In this review, we focus on the tacitly accepted, yet complex, quantitative aspect of SRS microscopy. We discuss the various approaches to quantitative analysis, examples of such approaches, challenges in different systems, and potential solutions. Through our examination of published literature, we conclude that a scrupulous approach to experimental design can further expand the powerful and incisive quantitative capabilities of SRS microscopy.

Protein and lipid mass concentration measurement in tissues by stimulated Raman scattering microscopy

Cell mass and chemical composition are important aggregate cellular properties that are especially relevant to physiological processes, such as growth control and tissue homeostasis. Despite their importance, it has been difficult to measure these features quantitatively at the individual cell level in intact tissue. Here, we introduce normalized Raman imaging (NoRI), a stimulated Raman scattering (SRS) microscopy method that provides the local concentrations of protein, lipid, and water from live or fixed tissue samples with high spatial resolution. Using NoRI, we demonstrate that protein, lipid, and water concentrations at the single cell are maintained in a tight range in cells under the same physiological conditions and are altered in different physiological states, such as cell cycle stages, attachment to substrates of different stiffness, or by entering senescence. In animal tissues, protein and lipid concentration varies with cell types, yet an unexpected cell-to-cell heterogeneity was found in cerebellar Purkinje cells. The protein and lipid concentration profile provides means to quantitatively compare disease-related pathology, as demonstrated using models of Alzheimer’s disease. This demonstration shows that NoRI is a broadly applicable technique for probing the biological regulation of protein mass, lipid mass, and water mass for studies of cellular and tissue growth, homeostasis, and disease.

Recent Progress in the Correlative Structured Illumination Microscopy

The super-resolution imaging technique of structured illumination microscopy (SIM) enables the mixing of high-frequency information into the optical transmission domain via light-source modulation, thus breaking the optical diffraction limit. Correlative SIM, which combines other techniques with SIM, offers more versatility or higher imaging resolution than traditional SIM. In this review, we first briefly introduce the imaging mechanism and development trends of conventional SIM. Then, the principles and recent developments of correlative SIM techniques are reviewed. Finally, the future development directions of SIM and its correlative microscopies are presented.

Roadmap on Digital Holography-Based Quantitative Phase Imaging

Quantitative Phase Imaging (QPI) provides unique means for the imaging of biological or technical microstructures, merging beneficial features identified with microscopy, interferometry, holography, and numerical computations. This roadmap article reviews several digital holography-based QPI approaches developed by prominent research groups. It also briefly discusses the present and future perspectives of 2D and 3D QPI research based on digital holographic microscopy, holographic tomography, and their applications.

Multimodal on-chip nanoscopy and quantitative phase imaging reveals the nanoscale morphology of liver sinusoidal endothelial cells

Visualization of three-dimensional (3D) morphological changes in the subcellular structures of a biological specimen is a major challenge in life science. Here, we present an integrated chip-based optical nanoscopy combined with quantitative phase microscopy (QPM) to obtain 3D morphology of liver sinusoidal endothelial cells (LSEC). LSEC have unique morphology with small nanopores (50-300 nm in diameter) in the plasma membrane, called fenestrations. The fenestrations are grouped in discrete clusters, which are around 100 to 200 nm thick. Thus, imaging and quantification of fenestrations and sieve plate thickness require resolution and sensitivity of sub-100 nm along both the lateral and the axial directions, respectively. In chip-based nanoscopy, the optical waveguides are used both for hosting and illuminating the sample. The fluorescence signal is captured by an upright microscope, which is converted into a Linnik-type interferometer to sequentially acquire both superresolved images and phase information of the sample. The multimodal microscope provided an estimate of the fenestration diameter of 119 ± 53 nm and average thickness of the sieve plates of 136.6 ± 42.4 nm, assuming the constant refractive index of cell membrane to be 1.38. Further, LSEC were treated with cytochalasin B to demonstrate the possibility of precise detection in the cell height. The mean phase value of the fenestrated area in normal and treated cells was found to be 161 ± 50 mrad and 109 ± 49 mrad, respectively. The proposed multimodal technique offers nanoscale visualization of both the lateral size and the thickness map, which would be of broader interest in the fields of cell biology and bioimaging.

Application of quantitative cell imaging using label-free optical diffraction tomography

The cell is three-dimensionally and dynamically organized into cellular compartments, including the endoplasmic reticulum, mitochondria, vesicles, and nucleus, which have high relative molecular density. The structure and functions of these compartments and organelles may be deduced from the diffusion and interaction of related biomolecules. Among these cellular components, various protein molecules can freely access the nucleolus or mitotic chromosome through Brownian diffusion, even though they have a densely packed structure. However, physicochemical properties of the nucleolus and chromosomes, such as molecular density and volume, are not yet fully understood under changing cellular conditions. Many studies have been conducted based on high-resolution imaging and analysis techniques using fluorescence. However, there are limitations in imaging only fluorescently labeled molecules, and cytotoxicity occurs during three-dimensional imaging. Alternatively, the recently developed label-free three-dimensional optical diffraction tomography (ODT) imaging technique can divide various organelles in cells into volumes and analyze them by refractive index, although specific molecules cannot be observed. A previous study established an analytical method that provides comprehensive insights into the physical properties of the nucleolus and mitotic chromosome by utilizing the advantages of ODT and fluorescence techniques, such as fluorescence correlation spectroscopy and confocal laser scanning microscopy. This review article summarizes a recent study and discusses the future aspects of the ODT for cellular compartments.

Fast 3D super-resolution imaging using a digital micromirror device and binary holography

SIGNIFICANCE

High-speed three-dimensional (3D) super-resolution microscopy is a unique tool to investigate various biological phenomena; yet the technology is not broadly adopted due to its high cost and complex system design.

AIM

We present a compact, low-cost, and high-speed 3D structured illumination microscopy (SIM) based on a digital micromirror device and binary holography to visualize fast biological events with super-resolution.

APPROACH

The 3D SIM uses a digital micromirror device to generate three laser foci with individually controllable positions, phases, and amplitudes via binary holography at the back aperture of objective lens to form optimal 3D structured patterns. Fifteen raw images are sequentially recorded and processed by the 3D SIM algorithm to reconstruct a super-resolved image.

RESULTS

Super-resolution 3D imaging at a speed of 26.7 frames per second is achieved with a lateral and axial resolution of 155 and 487 nm, which corresponds to a 1.65- and 1.63-times resolution enhancement, respectively, comparing with standard deconvolution microscopy.

CONCLUSIONS

The 3D SIM realizes fast super-resolution imaging with optimal 3D structured illumination, which may find important applications in biophotonics.

Terahertz non-label subwavelength imaging with composite photonics-plasmonics structured illumination

Inspired by the capability of structured illumination microscopy (SIM) in subwavelength imaging, many researchers devoted themselves to investigating this methodology. However, due to the free-propagating feature of the traditional structured illumination fields, the resolution can be only improved up to two-fold of the diffraction-limited microscopy. Besides, most of the previous studies, relying on incoherent illumination sources, are restricted to fluorescent samples. In this work, a subwavelength non-fluorescent imaging method is proposed based on the illumination of terahertz traveling waves and plasmonics. Excited along with a metal grating, the spoof surface plasmons (SSPs) are employed as one of the illuminating sources. When the scattering waves with the SSPs illumination are captured, the sample's high-order spatial frequencies (SF) components are already encoded into the obtainable low-order ones. Then, a modified post-processing algorithm is exploited to shift the modulated SF components to their actual positions in the SF domain. In this manner, the fine information of samples is introduced to reconstruct the desired imaging, leading to an enhancement of the resolution up to 0.12λ₀. Encouragingly, the resolution can be further enhanced by attaching extra illumination of SSPs with an elaborately selected frequency. This method holds promise for some important applications in terahertz non-fluorescent microscopy and sample detection with weak scattering.

Structured illumination microscopy with partially coherent illumination for phase and fluorescent imaging

This study presents a partially coherent illumination based (PCI-based) SIM apparatus for dual-modality (phase and fluorescent) microscopic imaging. The partially coherent illumination (PCI) is generated by placing a rotating diffuser on a monochromatic laser beam, which suppresses speckle noise in the dual-modality images and endows the apparatus with sound sectioning capability. With this system, label-free quantitative phase and super-resolved/sectioned fluorescent images can be obtained for the same sample. We have demonstrated the superiority of the system in phase imaging of transparent cells with high endogenous contrast and in a quantitative manner. In the meantime, we have also demonstrated fluorescent imaging of fluorescent beads, rat tail crosscut, wheat anther, and hibiscus pollen with super-resolution and optical sectioning. We envisage that the proposed method can be applied to many fields, including but not limited to biomedical, industrial, chemistry fields.

CD45 pre-exclusion from the tips of T cell microvilli prior to antigen recognition

The tyrosine phosphatase CD45 is a major gatekeeper for restraining T cell activation. Its exclusion from the immunological synapse (IS) is crucial for T cell receptor (TCR) signal transduction. Here, we use expansion super-resolution microscopy to reveal that CD45 is mostly pre-excluded from the tips of microvilli (MV) on primary T cells prior to antigen encounter. This pre-exclusion is diminished by depleting cholesterol or by engineering the transmembrane domain of CD45 to increase its membrane integration length, but is independent of the CD45 extracellular domain. We further show that brief MV-mediated contacts can induce Ca2+ influx in mouse antigen-specific T cells engaged by antigen-pulsed antigen presenting cells (APC). We propose that the scarcity of CD45 phosphatase activity at the tips of MV enables or facilitates TCR triggering from brief T cell-APC contacts before formation of a stable IS, and that these MV-mediated contacts represent the earliest step in the initiation of a T cell adaptive immune response.

Simulating digital micromirror devices for patterning coherent excitation light in structured illumination microscopy

Digital micromirror devices (DMDs) are spatial light modulators that employ the electro-mechanical movement of miniaturized mirrors to steer and thus modulate the light reflected off a mirror array. Their wide availability, low cost and high speed make them a popular choice both in consumer electronics such as video projectors, and scientific applications such as microscopy. High-end fluorescence microscopy systems typically employ laser light sources, which by their nature provide coherent excitation light. In super-resolution microscopy applications that use light modulation, most notably structured illumination microscopy (SIM), the coherent nature of the excitation light becomes a requirement to achieve optimal interference pattern contrast. The universal combination of DMDs and coherent light sources, especially when working with multiple different wavelengths, is unfortunately not straight forward. The substructure of the tilted micromirror array gives rise to a blazed grating, which has to be understood and which must be taken into account when designing a DMD-based illumination system. Here, we present a set of simulation frameworks that explore the use of DMDs in conjunction with coherent light sources, motivated by their application in SIM, but which are generalizable to other light patterning applications. This framework provides all the tools to explore and compute DMD-based diffraction effects and to simulate possible system alignment configurations computationally, which simplifies the system design process and provides guidance for setting up DMD-based microscopes. This article is part of the Theo Murphy meeting 'Super-resolution structured illumination microscopy (part 1)'.

Multicolor structured illumination microscopy and quantitative control of polychromatic light with a digital micromirror device

Linear structured illumination microscopy (SIM) is a super-resolution microscopy technique that does not impose photophysics requirements on fluorescent samples. Multicolor SIM implementations typically rely on liquid crystal on silicon (LCoS) spatial light modulators (SLM's) for patterning the excitation light, but digital micromirror devices (DMD's) are a promising alternative, owing to their lower cost and higher speed. However, existing coherent DMD SIM implementations use only a single wavelength of light, limited by the lack of efficient approaches for solving the blazed grating effect for polychromatic light. We develop the requisite quantitative tools, including a closed form solution of the blaze and diffraction conditions, forward models of DMD diffraction and pattern projection, and a model of DMD aberrations. Based on these advances, we constructed a three-color DMD microscope, quantified the effect of aberrations from the DMD, developed a high-resolution optical transfer function measurement technique, and demonstrated SIM on fixed and live cells. This opens the door to applying DMD's in polychromatic applications previously restricted to LCoS SLM's.

Multicolor structured illumination microscopy and quantitative control of polychromatic light with a digital micromirror device

Linear structured illumination microscopy (SIM) is a super-resolution microscopy technique that does not impose photophysics requirements on fluorescent samples. Multicolor SIM implementations typically rely on liquid crystal on silicon (LCoS) spatial light modulators (SLM's) for patterning the excitation light, but digital micromirror devices (DMD's) are a promising alternative, owing to their lower cost and higher speed. However, existing coherent DMD SIM implementations use only a single wavelength of light, limited by the lack of efficient approaches for solving the blazed grating effect for polychromatic light. We develop the requisite quantitative tools, including a closed form solution of the blaze and diffraction conditions, forward models of DMD diffraction and pattern projection, and a model of DMD aberrations. Based on these advances, we constructed a three-color DMD microscope, quantified the effect of aberrations from the DMD, developed a high-resolution optical transfer function measurement technique, and demonstrated SIM on fixed and live cells. This opens the door to applying DMD's in polychromatic applications previously restricted to LCoS SLM's.

Application of digital holographic tomography in antitumor effect of cantharides complex on 4T1 breast cancer cells

The study focuses on a methodology providing noninvasive monitoring and evaluation of the antitumor effect of traditional Chinese medicine, cantharides complex (canth), on 4T1 breast tumor cells. Digital holographic tomography (DHT) and developed data post-processing algorithms were used for quantitative estimation of changes in optical and morphological parameters of cells. We calculated and compared data on the refractive index, thickness, and projected area of 4T1 breast tumor cells in control untreated specimens and those treated with doxorubicin hydrochloride (DOX), canth, and their combinations. Post-treatment changes in cellular morphology recorded by DHT demonstrated that the two drugs led to noticeably different morphological changes in cells that can be presumably associated with different pathways of their death, apoptosis, or necrosis. The effect of combined treatment with these two drugs strongly depended on their relative concentrations and could lead to changes characteristic either for DOX or for canth; however, being more profound than those obtained when using each drug solely. The results obtained by DHT are in a good correspondence with commonly used cell viability analysis and immunofluorescent analysis of changes in cellular cytoskeleton.

Motionless Polarizing Structured Illumination Microscopy

In this investigation, we propose a motionless polarizing structured illumination microscopy as an axially sectioning and reflective-type device to measure the 3D surface profiles of specimens. Based on the spatial phase-shifting technique to obtain the visibility of the illumination pattern. Instead of using a grid, a Wollaston prism is used to generate the light pattern by the stable interference of two beams. As the polarization states of two beams are orthogonal with each other, a polarization pixelated CMOS camera can simultaneously obtain four phase-shifted patterns with the beams after passing through a quarter wave plate based on the spatial phase-shifting technique with polarization. In addition, a focus tunable lens is used to eliminate a mechanical moving part for the axial scanning of the specimen. In the experimental result, a step height sample and a concave mirror were measured with 0.05 µm and 0.2 mm repeatabilities of step height and the radius of curvature, respectively.

Optimizing sample illumination scanning in transmission tomographic diffractive microscopy

Due to the sequential nature of data acquisition, it is preferable to limit the number of illuminations to be used in tomographic diffractive microscopy experiments, especially if fast imaging is foreseen. On the other hand, for high-quality, high-resolution imaging, the Fourier space has to be optimally filled. Up to now, the problem of optimal Fourier space filling has not been investigated in itself. In this paper, we perform a comparative study to analyze the effect of sample scanning patterns on Fourier space filling for a transmission setup. Optical transfer functions for several illumination patterns are studied. Simulation as well as experiments are conducted to compare associated image reconstructions. We found that 3D uniform angular sweeping best fills the Fourier space, leading to better quality images.

Label-free three-dimensional observations and quantitative characterisation of on-chip vasculogenesis using optical diffraction tomography

Label-free, three-dimensional (3D) quantitative observations of on-chip vasculogenesis were achieved using optical diffraction tomography. Exploiting 3D refractive index maps as an intrinsic imaging contrast, the vascular structures, multicellular activities, and subcellular organelles of endothelial cells were imaged and analysed throughout vasculogenesis to characterise mature vascular networks without exogenous labelling.

Holotomography: Refractive Index as an Intrinsic Imaging Contrast for 3-D Label-Free Live Cell Imaging

Live cell imaging provides essential information in the investigation of cell biology and related pathophysiology. Refractive index (RI) can serve as intrinsic optical imaging contrast for 3-D label-free and quantitative live cell imaging, and provide invaluable information to understand various dynamics of cells and tissues for the study of numerous fields. Recently significant advances have been made in imaging methods and analysis approaches utilizing RI, which are now being transferred to biological and medical research fields, providing novel approaches to investigate the pathophysiology of cells. To provide insight into how RI can be used as an imaging contrast for imaging of biological specimens, here we provide the basic principle of RI-based imaging techniques and summarize recent progress on applications, ranging from microbiology, hematology, infectious diseases, hematology, and histopathology.

Low-coherence optical diffraction tomography using a ferroelectric liquid crystal spatial light modulator

Optical diffraction tomography (ODT) is a three-dimensional (3D) label-free imaging technique. The 3D refractive index distribution of a sample can be reconstructed from multiple two-dimensional optical field images via ODT. Herein, we introduce a temporally low-coherence ODT technique using a ferroelectric liquid crystal spatial light modulator (FLC SLM). The fast binary-phase modulation provided by the FLC SLM ensures the high spatiotemporal resolution. To reduce coherent noise, a superluminescent light-emitting diode is used as an economic low-coherence light source. We demonstrate the performance of the proposed system using various samples, including colloidal microspheres and live epithelial cells.

Deep-learning-based three-dimensional label-free tracking and analysis of immunological synapses of CAR-T cells

The immunological synapse (IS) is a cell-cell junction between a T cell and a professional antigen-presenting cell. Since the IS formation is a critical step for the initiation of an antigen-specific immune response, various live-cell imaging techniques, most of which rely on fluorescence microscopy, have been used to study the dynamics of IS. However, the inherent limitations associated with the fluorescence-based imaging, such as photo-bleaching and photo-toxicity, prevent the long-term assessment of dynamic changes of IS with high frequency. Here, we propose and experimentally validate a label-free, volumetric, and automated assessment method for IS dynamics using a combinational approach of optical diffraction tomography and deep learning-based segmentation. The proposed method enables an automatic and quantitative spatiotemporal analysis of IS kinetics of morphological and biochemical parameters associated with IS dynamics, providing a new option for immunological research.

Virtual organelle self-coding for fluorescence imaging via adversarial learning

SIGNIFICANCE

Our study introduces an application of deep learning to virtually generate fluorescence images to reduce the burdens of cost and time from considerable effort in sample preparation related to chemical fixation and staining.

AIM

The objective of our work was to determine how successfully deep learning methods perform on fluorescence prediction that depends on structural and/or a functional relationship between input labels and output labels.

APPROACH

We present a virtual-fluorescence-staining method based on deep neural networks (VirFluoNet) to transform co-registered images of cells into subcellular compartment-specific molecular fluorescence labels in the same field-of-view. An algorithm based on conditional generative adversarial networks was developed and trained on microscopy datasets from breast-cancer and bone-osteosarcoma cell lines: MDA-MB-231 and U2OS, respectively. Several established performance metrics-the mean absolute error (MAE), peak-signal-to-noise ratio (PSNR), and structural-similarity-index (SSIM)-as well as a novel performance metric, the tolerance level, were measured and compared for the same algorithm and input data.

RESULTS

For the MDA-MB-231 cells, F-actin signal performed the fluorescent antibody staining of vinculin prediction better than phase-contrast as an input. For the U2OS cells, satisfactory metrics of performance were archieved in comparison with ground truth. MAE is <0.005, 0.017, 0.012; PSNR is >40  /  34  /  33  dB; and SSIM is >0.925  /  0.926  /  0.925 for 4',6-diamidino-2-phenylindole/hoechst, endoplasmic reticulum, and mitochondria prediction, respectively, from channels of nucleoli and cytoplasmic RNA, Golgi plasma membrane, and F-actin.

CONCLUSIONS

These findings contribute to the understanding of the utility and limitations of deep learning image-regression to predict fluorescence microscopy datasets of biological cells. We infer that predicted image labels must have either a structural and/or a functional relationship to input labels. Furthermore, the approach introduced here holds promise for modeling the internal spatial relationships between organelles and biomolecules within living cells, leading to detection and quantification of alterations from a standard training dataset.

Angle-resolved spectral reflectometry with a digital light processing projector

We describe a novel approach for angle-resolved spectral reflectometry using a digital light processing (DLP) projector. Here, the DLP generates ring patterned images which are projected on the back focal plane of an objective lens. This way, the proposed method quickly changes the angle of incidence with ease based on the relation between the radius of the back focal plane and the angle of incidence. As a result, a detector captures the intensity of the image plane based on the angular and spectral axis. As the proposed method detects the interesting spot of a sample image, it can easily locate the measurement spot with viewing the full field of view, and the spot size is reduced by adopting the fiber. This method is verified by comparing the measurement output of the thin-film samples with a commercial ellipsometer. The result shows that our the proposed method enables the high accuracy of the thin-film inspection.

Digital Holographic Multimodal Cross-Sectional Fluorescence and Quantitative Phase Imaging System

We present a multimodal imaging system based on simple off-axis digital holography, for simultaneous recording and retrieval of cross-sectional fluorescence and quantitative phase imaging of the biological specimen. Synergism in the imaging capabilities can be achieved by incorporating two off-axis digital holographic microscopes integrated to record different information at the same time. The cross-sectional fluorescence imaging is realized by a common-path configuration of the single-shot off-axis incoherent digital holographic system. The quantitative phase imaging, on the other hand, is achieved by another off-axis coherent digital holographic microscopy operating in transmission mode. The fundamental characteristics of the proposed multimodal system are confirmed by performing various experiments on fluorescent beads and fluorescent protein-labeled living cells of the moss Physcomitrella patens lying at different axial depth positions. Furthermore, the cross-sectional live fluorescence and phase imaging of the fluorescent beads are demonstrated by the proposed multimodal system. The experimental results presented here corroborate the feasibility of the proposed system and indicate its potential in the applications to analyze the functional and structural behavior of biological cells and tissues.

Fourier ptychography: current applications and future promises

Traditional imaging systems exhibit a well-known trade-off between the resolution and the field of view of their captured images. Typical cameras and microscopes can either "zoom in" and image at high-resolution, or they can "zoom out" to see a larger area at lower resolution, but can rarely achieve both effects simultaneously. In this review, we present details about a relatively new procedure termed Fourier ptychography (FP), which addresses the above trade-off to produce gigapixel-scale images without requiring any moving parts. To accomplish this, FP captures multiple low-resolution, large field-of-view images and computationally combines them in the Fourier domain into a high-resolution, large field-of-view result. Here, we present details about the various implementations of FP and highlight its demonstrated advantages to date, such as aberration recovery, phase imaging, and 3D tomographic reconstruction, to name a few. After providing some basics about FP, we list important details for successful experimental implementation, discuss its relationship with other computational imaging techniques, and point to the latest advances in the field while highlighting persisting challenges.

Common-path multimodal three-dimensional fluorescence and phase imaging system

<p>A stable multimodal system is developed by combining two common-path digital holographic microscopes (DHMs): coherent and incoherent, for simultaneous recording and retrieval of three-dimensional (3-D) phase and 3-D fluorescence imaging (FI), respectively, of a biological specimen. The 3-D FI is realized by a single-shot common-path off-axis fluorescent DHM developed recently by our group. In addition, we accomplish, the phase imaging by another single-shot, highly stable common-path off-axis DHM based on a beam splitter. In this DHM configuration, a beam splitter is used to divide the incoming object beam into two beams. One beam serves as the object beam carrying the useful information of the object under study, whereas another beam is spatially filtered at its Fourier plane by using a pinhole and it serves as a reference beam. This DHM setup, owing to a common-path geometry, is less vibration-sensitive and compact, having a similar field of view but with high temporal phase stability in comparison to a two-beam Mach-Zehnder-type DHM. The performance of the proposed common-path DHM and the multimodal system is verified by conducting various experiments on fluorescent microspheres and fluorescent protein-labeled living cells of the moss <italic>Physcomitrella patens</italic>. Moreover, the potential capability of the proposed multimodal system for 3-D live fluorescence and phase imaging of the fluorescent beads is also demonstrated. The obtained experimental results corroborate the feasibility of the proposed multimodal system and indicate its potential applications for the analysis of functional and structural behaviors of a biological specimen and enhancement of the understanding of physiological mechanisms and various biological diseases.</p>.

Super-resolution fluorescence-assisted diffraction computational tomography reveals the three-dimensional landscape of the cellular organelle interactome

The emergence of super-resolution (SR) fluorescence microscopy has rejuvenated the search for new cellular sub-structures. However, SR fluorescence microscopy achieves high contrast at the expense of a holistic view of the interacting partners and surrounding environment. Thus, we developed SR fluorescence-assisted diffraction computational tomography (SR-FACT), which combines label-free three-dimensional optical diffraction tomography (ODT) with two-dimensional fluorescence Hessian structured illumination microscopy. The ODT module is capable of resolving the mitochondria, lipid droplets, the nuclear membrane, chromosomes, the tubular endoplasmic reticulum, and lysosomes. Using dual-mode correlated live-cell imaging for a prolonged period of time, we observed novel subcellular structures named dark-vacuole bodies, the majority of which originate from densely populated perinuclear regions, and intensively interact with organelles such as the mitochondria and the nuclear membrane before ultimately collapsing into the plasma membrane. This work demonstrates the unique capabilities of SR-FACT, which suggests its wide applicability in cell biology in general.

Intelligent frequency-shifted optofluidic time-stretch quantitative phase imaging

Optofluidic time-stretch quantitative phase imaging (OTS-QPI) is a powerful tool as it enables high-throughput (>10,000 cell/s) QPI of single live cells. OTS-QPI is based on decoding temporally stretched spectral interferograms that carry the spatial profiles of cells flowing on a microfluidic chip. However, the utility of OTS-QPI is troubled by difficulties in phase retrieval from the high-frequency region of the temporal interferograms, such as phase-unwrapping errors, high instrumentation cost, and large data volume. To overcome these difficulties, we propose and experimentally demonstrate frequency-shifted OTS-QPI by bringing the phase information to the baseband region. Furthermore, to show its boosted utility, we use it to demonstrate image-based classification of leukemia cells with high accuracy over 96% and evaluation of drug-treated leukemia cells via deep learning.

Picosecond-resolution phase-sensitive imaging of transparent objects in a single shot

With the growing interest in the optical imaging of ultrafast phenomena in transparent objects, from shock wave to neuronal action potentials, high contrast imaging at high frame rates has become desirable. While phase sensitivity provides the contrast, the frame rates and sequence depths are highly limited by the detectors. Here, we present phase-sensitive compressed ultrafast photography (pCUP) for single-shot real-time ultrafast imaging of transparent objects by combining the contrast of dark-field imaging with the speed and the sequence depth of CUP. By imaging the optical Kerr effect and shock wave propagation, we demonstrate that pCUP can image light-speed phase signals in a single shot with up to 350 frames captured at up to 1 trillion frames per second. We expect pCUP to be broadly used for a vast range of fundamental and applied sciences.

Adaptive wavefront correction structured illumination holographic tomography

In this study, a novel adaptive wavefront correction (AWC) technique is implemented on a compactly developed structured illumination holographic tomography (SI-HT) system. We propose a mechanical movement-free compact scanning architecture for SI-HT systems with AWC, implemented by designing and displaying a series of computer-generated holograms (CGH) composed of blazed grating with phase Fresnel lens on a phase-only spatial light modulator (SLM). In the proposed SI-HT, the aberrations of the optical system are sensed by digital holography and are used to design the CGH-based AWC to compensate the phase aberrations of the tomographic imaging system. The proposed method was validated using a standard Siemens star target, its potential application was demonstrated using a live candida rugosa sample, and its label-free three-dimensional refractive index profile was generated at its subcellular level. The experimental results obtained reveal the ability of the proposed method to enhance the imaging performance in both lateral and axial directions.

Speckle-structured illumination for 3D phase and fluorescence computational microscopy

High-content biological microscopy targets high-resolution imaging across large fields-of-view, often achieved by computational imaging approaches. Previously, we demonstrated 2D multimodal high-content microscopy via structured illumination microscopy (SIM) with resolution > 2 × the diffraction limit, using speckle illumination from Scotch tape. In this work, we extend the method to 3D by leveraging the fact that the speckle illumination is in fact a 3D structured pattern. We use both a coherent and an incoherent imaging model to develop algorithms for joint retrieval of the 3D super-resolved fluorescent and complex-field distributions of the sample. Our reconstructed images resolve features beyond the physical diffraction-limit set by the system's objective and demonstrate 3D multimodal imaging with ∼ 0.6 × 0.6 × 6   μ m³ resolution over a volume of ∼ 314 × 500 × 24   μ m³.

Computational structured illumination for high-content fluorescence and phase microscopy

High-content biological microscopy targets high-resolution imaging across large fields-of-view (FOVs). Recent works have demonstrated that computational imaging can provide efficient solutions for high-content microscopy. Here, we use speckle structured illumination microscopy (SIM) as a robust and cost-effective solution for high-content fluorescence microscopy with simultaneous high-content quantitative phase (QP). This multi-modal compatibility is essential for studies requiring cross-correlative biological analysis. Our method uses laterally-translated Scotch tape to generate high-resolution speckle illumination patterns across a large FOV. Custom optimization algorithms then jointly reconstruct the sample's super-resolution fluorescent (incoherent) and QP (coherent) distributions, while digitally correcting for system imperfections such as unknown speckle illumination patterns, system aberrations and pattern translations. Beyond previous linear SIM works, we achieve resolution gains of 4× the objective's diffraction-limited native resolution, resulting in 700 nm fluorescence and 1.2 μm QP resolution, across a FOV of 2 × 2.7 mm 2 , giving a space-bandwidth product (SBP) of 60 megapixels.

Combining Three-Dimensional Quantitative Phase Imaging and Fluorescence Microscopy for the Study of Cell Pathophysiology

Quantitative phase imaging (QPI) has emerged as one of the powerful imaging tools for the study of live cells in a non-invasive manner. In particular, multimodal approaches combining QPI and fluorescence microscopic techniques have been recently developed for superior spatiotemporal resolution as well as high molecular specificity. In this review, we briefly summarize recent advances in three-dimensional QPI combined with fluorescence techniques for the correlative study of cell pathophysiology. Through this review, biologists and clinicians can be provided with insights on this rapidly growing field of research and may find broader applications to investigate unrevealed nature in cell physiology and related diseases.

Measurements of three-dimensional refractive index tomography and membrane deformability of live erythrocytes from Pelophylax nigromaculatus

Unlike mammalian erythrocytes, amphibian erythrocytes have distinct morphological features including large cell sizes and the presence of nuclei. The sizes of the cytoplasm and nuclei of erythrocytes vary significantly over different species, their environments, or pathophysiology, which makes hematological studies important for investigating amphibian species. Here, we present a label-free three-dimensional optical quantification of individual amphibian erythrocytes from frogs Pelophylax nigromaculatus (Rana nigromaculata). Using optical diffraction tomography, we measured three-dimensional refractive index (RI) tomograms of the cells, which clearly distinguished the cytoplasm and nuclei of the erythrocytes. From the measured RI tomograms, we extracted the relevant biochemical parameters of the cells, including hemoglobin contents and hemoglobin concentrations. Furthermore, we measured dynamic membrane fluctuations and investigated the mechanical properties of the cell membrane. From the statistical and correlative analysis of these retrieved parameters, we investigated interspecific differences between frogs and previously studied mammals.

Three-dimensional label-free imaging and analysis of Pinus pollen grains using optical diffraction tomography

The structure of pollen grains is related to the reproductive function of the plants. Here, three-dimensional (3D) refractive index maps were obtained for individual conifer pollen grains using optical diffraction tomography (ODT). The 3D morphological features of pollen grains from pine trees were investigated using measured refractive index maps, in which distinct substructures were clearly distinguished and analyzed. Morphological and physiochemical parameters of the pollen grains were quantified from the obtained refractive index (RI) maps and used to quantitatively study the interspecific differences of pollen grains from different strains. Our results demonstrate that ODT can assess the structure of pollen grains. This label-free and rapid 3D imaging approach may provide a new platform for understanding the physiology of pollen grains.