Artificial intelligence-enabled quantitative phase imaging methods for life sciences

Label-free spatiotemporal decoding of single-cell fate via acoustic driven 3D tomography

Label-free three-dimensional imaging plays a crucial role in unraveling the complexities of cellular functions and interactions in biomedical research. Conventional single-cell optical tomography techniques offer affordability and the convenience of bypassing laborious cell labelling protocols. However, these methods are encumbered by restricted illumination scanning ranges on abaxial plane, resulting in the loss of intricate cellular imaging details. The ability to fully control cellular rotation across all angles has emerged as an optimal solution for capturing comprehensive structural details of cells. Here, we introduce a label-free, cost-effective, and readily fabricated contactless acoustic-induced vibration system, specifically designed to enable multi-degree-of-freedom rotation of cells, ultimately attaining stable in-situ rotation. Furthermore, by integrating this system with advanced deep learning technologies, we perform 3D reconstruction and morphological analysis on diverse cell types, thus validating groups of high-precision cell identification. Notably, long-term observation of cells reveals distinct features associated with drug-induced apoptosis in both cancerous and normal cells populations. This methodology, based on deep learning-enabled cell 3D reconstruction, charts a novel trajectory for groups of real-time cellular visualization, offering promising advancements in the realms of drug screening and post-single-cell analysis, thereby addressing potential clinical requisites.

Neural network-based processing and reconstruction of compromised biophotonic image data

In recent years, the integration of deep learning techniques with biophotonic setups has opened new horizons in bioimaging. A compelling trend in this field involves deliberately compromising certain measurement metrics to engineer better bioimaging tools in terms of e.g., cost, speed, and form-factor, followed by compensating for the resulting defects through the utilization of deep learning models trained on a large amount of ideal, superior or alternative data. This strategic approach has found increasing popularity due to its potential to enhance various aspects of biophotonic imaging. One of the primary motivations for employing this strategy is the pursuit of higher temporal resolution or increased imaging speed, critical for capturing fine dynamic biological processes. Additionally, this approach offers the prospect of simplifying hardware requirements and complexities, thereby making advanced imaging standards more accessible in terms of cost and/or size. This article provides an in-depth review of the diverse measurement aspects that researchers intentionally impair in their biophotonic setups, including the point spread function (PSF), signal-to-noise ratio (SNR), sampling density, and pixel resolution. By deliberately compromising these metrics, researchers aim to not only recuperate them through the application of deep learning networks, but also bolster in return other crucial parameters, such as the field of view (FOV), depth of field (DOF), and space-bandwidth product (SBP). Throughout this article, we discuss various biophotonic methods that have successfully employed this strategic approach. These techniques span a wide range of applications and showcase the versatility and effectiveness of deep learning in the context of compromised biophotonic data. Finally, by offering our perspectives on the exciting future possibilities of this rapidly evolving concept, we hope to motivate our readers from various disciplines to explore novel ways of balancing hardware compromises with compensation via artificial intelligence (AI).

Harnessing the capabilities of VCSELs: unlocking the potential for advanced integrated photonic devices and systems

Vertical cavity surface emitting lasers (VCSELs) have emerged as a versatile and promising platform for developing advanced integrated photonic devices and systems due to their low power consumption, high modulation bandwidth, small footprint, excellent scalability, and compatibility with monolithic integration. By combining these unique capabilities of VCSELs with the functionalities offered by micro/nano optical structures (e.g. metasurfaces), it enables various versatile energy-efficient integrated photonic devices and systems with compact size, enhanced performance, and improved reliability and functionality. This review provides a comprehensive overview of the state-of-the-art versatile integrated photonic devices/systems based on VCSELs, including photonic neural networks, vortex beam emitters, holographic devices, beam deflectors, atomic sensors, and biosensors. By leveraging the capabilities of VCSELs, these integrated photonic devices/systems open up new opportunities in various fields, including artificial intelligence, large-capacity optical communication, imaging, biosensing, and so on. Through this comprehensive review, we aim to provide a detailed understanding of the pivotal role played by VCSELs in integrated photonics and highlight their significance in advancing the field towards efficient, compact, and versatile photonic solutions.

Perovskite Probe-Based Machine Learning Imaging Model for Rapid Pathologic Diagnosis of Cancers

Accurately distinguishing tumor cells from normal cells is a key issue in tumor diagnosis, evaluation, and treatment. Fluorescence-based immunohistochemistry as the standard method faces the inherent challenges of the heterogeneity of tumor cells and the lack of big data analysis of probing images. Here, we have demonstrated a machine learning-driven imaging method for rapid pathological diagnosis of five types of cancers (breast, colon, liver, lung, and stomach) using a perovskite nanocrystal probe. After conducting the bioanalysis of survivin expression in five different cancers, high-efficiency perovskite nanocrystal probes modified with the survivin antibody can recognize the cancer tissue section at the single cell level. The tumor to normal (T/N) ratio is 10.3-fold higher than that of a conventional fluorescent probe, which can successfully differentiate between tumors and adjacent normal tissues within 10 min. The features of the fluorescence intensity and pathological texture morphology have been extracted and analyzed from 1000 fluorescence images by machine learning. The final integrated decision model makes the area under the receiver operating characteristic curve (area under the curve) value of machine learning classification of breast, colon, liver, lung, and stomach above 90% while predicting the tumor organ of 92% of positive patients. This method demonstrates a high T/N ratio probe in the precise diagnosis of multiple cancers, which will be good for improving the accuracy of surgical resection and reducing cancer mortality.

Roadmap on computational methods in optical imaging and holography [invited]

Computational methods have been established as cornerstones in optical imaging and holography in recent years. Every year, the dependence of optical imaging and holography on computational methods is increasing significantly to the extent that optical methods and components are being completely and efficiently replaced with computational methods at low cost. This roadmap reviews the current scenario in four major areas namely incoherent digital holography, quantitative phase imaging, imaging through scattering layers, and super-resolution imaging. In addition to registering the perspectives of the modern-day architects of the above research areas, the roadmap also reports some of the latest studies on the topic. Computational codes and pseudocodes are presented for computational methods in a plug-and-play fashion for readers to not only read and understand but also practice the latest algorithms with their data. We believe that this roadmap will be a valuable tool for analyzing the current trends in computational methods to predict and prepare the future of computational methods in optical imaging and holography.

Supplementary Information

The online version contains supplementary material available at 10.1007/s00340-024-08280-3.

In silico labeling in cell biology: Potential and limitations

In silico labeling is the computational cross-modality image translation where the output modality is a subcellular marker that is not specifically encoded in the input image, for example, in silico localization of organelles from transmitted light images. In principle, in silico labeling has the potential to facilitate rapid live imaging of multiple organelles with reduced photobleaching and phototoxicity, a technology enabling a major leap toward understanding the cell as an integrated complex system. However, five years have passed since feasibility was attained, without any demonstration of using in silico labeling to uncover new biological insight. In here, we discuss the current state of in silico labeling, the limitations preventing it from becoming a practical tool, and how we can overcome these limitations to reach its full potential.

Eagle-Eye Inspired Meta-Device for Phase Imaging

The dual-focus vision observed in eagles' eyes is an intriguing phenomenon captivates scientists since a long time. Inspired by this natural occurrence, the authors' research introduces a novel bifocal meta-device incorporating a polarized camera capable of simultaneously capturing images for two different polarizations with slightly different focal distances. This innovative approach facilitates the concurrent acquisition of underfocused and overfocused images in a single snapshot, enabling the effective extraction of quantitative phase information from the object using the transport of intensity equation. Experimental demonstrations showcase the application of quantitative phase imaging to artificial objects and human embryonic kidney cells, particularly emphasizing the meta-device's relevance in dynamic scenarios such as laser-induced ablation in human embryonic kidney cells. Moreover, it provides a solution for the quantification during the dynamic process at the cellular level. Notably, the proposed eagle-eye inspired meta-device for phase imaging (EIMPI), due to its simplicity and compact nature, holds promise for significant applications in fields such as endoscopy and headsets, where a lightweight and compact setup is essential.

Rapid Glutathione Analysis with SERS Microneedles for Deep Glioblastoma Tissue Differentiation

Rapid tissue differentiation at the molecular level is a prerequisite for precise surgical resection, which is of special value for the treatment of malignant tumors, such as glioblastoma (GBM). Herein, a SERS-active microneedle is prepared by modifying glutathione (GSH)-responsive molecules, 5,5'-dithiobis(2-nitrobenzoic acid) (DTNB), on the surface of Au@Ag substrates for the distinction of different GBM tissues. Since the Raman signals on the surface of the DTNB@Au@Ag microneedle can be collected by both portable and benchtop Raman spectrometers, the distribution of GSH in different tissues at centimeter scale can be displayed through Raman spectroscopy and Raman imaging, and the entire analysis process can be accomplished within 12 min. Accordingly, in vivo brain tissues of orthotopic GBM xenograft mice and ex vivo tissues of GBM patients are accurately differentiated with the microneedle, and the results are well consistent with tissue staining and postoperative pathological reports. In addition, the outline of tumor, peritumoral, and normal tissues can be indicated by the DTNB@Au@Ag microneedle for at least 56 days. Considering that the tumor tissues are quickly discriminated at the molecular level without the restriction of depth, the DTNB@Au@Ag microneedle is promising to be a powerful intraoperative diagnostic tool for surgery navigation.

Mantis: high-throughput 4D imaging and analysis of the molecular and physical architecture of cells

High-throughput dynamic imaging of cells and organelles is essential for understanding complex cellular responses. We report Mantis, a high-throughput 4D microscope that integrates two complementary, gentle, live-cell imaging technologies: remote-refocus label-free microscopy and oblique light-sheet fluorescence microscopy. Additionally, we report shrimPy, an open-source software for high-throughput imaging, deconvolution, and single-cell phenotyping of 4D data. Using Mantis and shrimPy, we achieved high-content correlative imaging of molecular dynamics and the physical architecture of 20 cell lines every 15 minutes over 7.5 hours. This platform also facilitated detailed measurements of the impacts of viral infection on the architecture of host cells and host proteins. The Mantis platform can enable high-throughput profiling of intracellular dynamics, long-term imaging and analysis of cellular responses to perturbations, and live-cell optical screens to dissect gene regulatory networks.

THP-1 Monocytic Cells Are Polarized to More Antitumorigenic Macrophages by Serial Treatment with Phorbol-12-Myristate-13-Acetate and PD98059

Background and Objectives: As modulators of the tumor microenvironment, macrophages have been extensively studied for their potential in developing anticancer strategies, particularly in regulating macrophage polarization towards an antitumorigenic (M1) phenotype rather than a protumorigenic (M2) one in various experimental models. Here, we evaluated the effect of PD98059, a mitogen-activated protein kinase kinase MAPKK MEK1-linked pathway inhibitor, on the differentiation and polarization of THP-1 monocytes in response to phorbol-12-myristate-13-acetate (PMA) under various culture conditions for tumor microenvironmental application. Materials and Methods: Differentiation and polarization of THP-1 were analyzed by flow cytometry and RT-PCR. Polarized THP-1 subsets with different treatment were compared by motility, phagocytosis, and so on. Results: Clearly, PMA induced THP-1 differentiation occurs in adherent culture conditions more than nonadherent culture conditions by increasing CD11b expression up to 90%, which was not affected by PD98059 when cells were exposed to PMA first (post-PD) but inhibited when PD98059 was treated prior to PMA treatment (pre-PD). CD11bhigh THP-1 cells treated with PMA and PMA-post-PD were categorized into M0 (HLA-DRlow and CD206low), M1 (HLA-DRhigh and CD206low), and M2 (HLA-DRlow and CD206high), resulting in an increased population of M1 macrophages. The transcription levels of markers of macrophage differentiation and polarization confirmed the increased M1 polarization of THP-1 cells with post-PD treatment rather than with PMA-only treatment. The motility and cytotoxicity of THP-1 cells with post-PD treatment were higher than THP-1 cells with PMA, suggesting that post-PD treatment enhanced the anti-tumorigenicity of THP-1 cells. Confocal microscopy and flow cytometry showed the effect of post-PD treatment on phagocytosis by THP-1 cells. Conclusions: We have developed an experimental model of macrophage polarization with THP-1 cells which will be useful for further studies related to the tumor microenvironment.

Dynamic label-free analysis of SARS-CoV-2 infection reveals virus-induced subcellular remodeling

Assessing the impact of SARS-CoV-2 on organelle dynamics allows a better understanding of the mechanisms of viral replication. We combine label-free holotomographic microscopy with Artificial Intelligence to visualize and quantify the subcellular changes triggered by SARS-CoV-2 infection. We study the dynamics of shape, position and dry mass of nucleoli, nuclei, lipid droplets and mitochondria within hundreds of single cells from early infection to syncytia formation and death. SARS-CoV-2 infection enlarges nucleoli, perturbs lipid droplets, changes mitochondrial shape and dry mass, and separates lipid droplets from mitochondria. We then used Bayesian network modeling on organelle dry mass states to define organelle cross-regulation networks and report modifications of organelle cross-regulation that are triggered by infection and syncytia formation. Our work highlights the subcellular remodeling induced by SARS-CoV-2 infection and provides an Artificial Intelligence-enhanced, label-free methodology to study in real-time the dynamics of cell populations and their content.

Quantitative phase imaging by gradient retardance optical microscopy

Quantitative phase imaging (QPI) has become a vital tool in bioimaging, offering precise measurements of wavefront distortion and, thus, of key cellular metabolism metrics, such as dry mass and density. However, only a few QPI applications have been demonstrated in optically thick specimens, where scattering increases background and reduces contrast. Building upon the concept of structured illumination interferometry, we introduce Gradient Retardance Optical Microscopy (GROM) for QPI of both thin and thick samples. GROM transforms any standard Differential Interference Contrast (DIC) microscope into a QPI platform by incorporating a liquid crystal retarder into the illumination path, enabling independent phase-shifting of the DIC microscope's sheared beams. GROM greatly simplifies related configurations, reduces costs, and eradicates energy losses in parallel imaging modalities, such as fluorescence. We successfully tested GROM on a diverse range of specimens, from microbes and red blood cells to optically thick (~ 300 μm) plant roots without fixation or clearing.

Nanoplastics Penetrate Human Bronchial Smooth Muscle and Small Airway Epithelial Cells and Affect Mitochondrial Metabolism

Micro- and nanoplastic particles, including common forms like polyethylene and polystyrene, have been identified as relevant pollutants, potentially causing health problems in living organisms. The mechanisms at the cellular level largely remain to be elucidated. This study aims to visualize nanoplastics in bronchial smooth muscle (BSMC) and small airway epithelial cells (SAEC), and to assess the impact on mitochondrial metabolism. Healthy and asthmatic human BSMC and SAEC in vitro cultures were stimulated with polystyrene nanoplastics (PS-NPs) of 25 or 50 nm size, for 1 or 24 h. Live cell, label-free imaging by holotomography microscopy and mitochondrial respiration and glycolysis assessment were performed. Furthermore, 25 and 50 nm NPs were shown to penetrate SAEC, along with healthy and diseased BSMC, and they impaired bioenergetics and induce mitochondrial dysfunction compared to cells not treated with NPs, including changes in oxygen consumption rate and extracellular acidification rate. NPs pose a serious threat to human health by penetrating airway tissues and cells, and affecting both oxidative and glycolytic metabolism.

The 3D Revolution in Cancer Discovery

SUMMARY

The transition from 2D to 3D spatial profiling marks a revolutionary era in cancer research, offering unprecedented potential to enhance cancer diagnosis and treatment. This commentary outlines the experimental and computational advancements and challenges in 3D spatial molecular profiling, underscoring the innovation needed in imaging tools, software, artificial intelligence, and machine learning to overcome implementation hurdles and harness the full potential of 3D analysis in the field.

Recent advances in label-free imaging and quantification techniques for the study of lipid droplets in cells

Lipid droplets (LDs), once considered mere storage depots for lipids, have gained recognition for their intricate roles in cellular processes, including metabolism, membrane trafficking, and disease states like obesity and cancer. This review explores label-free imaging techniques' applications in LD research. We discuss holotomography and vibrational spectroscopic microscopy, emphasizing their potential for studying LDs without molecular labels, and we highlight the growing integration of artificial intelligence. Clinical applications in disease diagnosis and therapy are also considered.

On the use of deep learning for phase recovery

Phase recovery (PR) refers to calculating the phase of the light field from its intensity measurements. As exemplified from quantitative phase imaging and coherent diffraction imaging to adaptive optics, PR is essential for reconstructing the refractive index distribution or topography of an object and correcting the aberration of an imaging system. In recent years, deep learning (DL), often implemented through deep neural networks, has provided unprecedented support for computational imaging, leading to more efficient solutions for various PR problems. In this review, we first briefly introduce conventional methods for PR. Then, we review how DL provides support for PR from the following three stages, namely, pre-processing, in-processing, and post-processing. We also review how DL is used in phase image processing. Finally, we summarize the work in DL for PR and provide an outlook on how to better use DL to improve the reliability and efficiency of PR. Furthermore, we present a live-updating resource ( https://github.com/kqwang/phase-recovery ) for readers to learn more about PR.