Cell Cycle Stage Classification Using Phase Imaging with Computational Specificity

CellSNAP: a fast, accurate algorithm for 3D cell segmentation in quantitative phase imaging

Significance

Three-dimensional quantitative phase imaging (QPI) has rapidly emerged as a complementary tool to fluorescence imaging, as it provides an objective measure of cell morphology and dynamics, free of variability due to contrast agents. It has opened up new directions of investigation by providing systematic and correlative analysis of various cellular parameters without limitations of photobleaching and phototoxicity. While current QPI systems allow the rapid acquisition of tomographic images, the pipeline to analyze these raw three-dimensional (3D) tomograms is not well-developed. We focus on a critical, yet often underappreciated, step of the analysis pipeline that of 3D cell segmentation from the acquired tomograms.

Aim

We report the CellSNAP (Cell Segmentation via Novel Algorithm for Phase Imaging) algorithm for the 3D segmentation of QPI images.

Approach

The cell segmentation algorithm mimics the gemstone extraction process, initiating with a coarse 3D extrusion from a two-dimensional (2D) segmented mask to outline the cell structure. A 2D image is generated, and a segmentation algorithm identifies the boundary in the x - y plane. Leveraging cell continuity in consecutive z -stacks, a refined 3D segmentation, akin to fine chiseling in gemstone carving, completes the process.

Results

The CellSNAP algorithm outstrips the current gold standard in terms of speed, robustness, and implementation, achieving cell segmentation under 2 s per cell on a single-core processor. The implementation of CellSNAP can easily be parallelized on a multi-core system for further speed improvements. For the cases where segmentation is possible with the existing standard method, our algorithm displays an average difference of 5% for dry mass and 8% for volume measurements. We also show that CellSNAP can handle challenging image datasets where cells are clumped and marred by interferogram drifts, which pose major difficulties for all QPI-focused AI-based segmentation tools.

Conclusion

Our proposed method is less memory intensive and significantly faster than existing methods. The method can be easily implemented on a student laptop. Since the approach is rule-based, there is no need to collect a lot of imaging data and manually annotate them to perform machine learning based training of the model. We envision our work will lead to broader adoption of QPI imaging for high-throughput analysis, which has, in part, been stymied by a lack of suitable image segmentation tools.

Perspective on quantitative phase imaging to improve precision cancer medicine

Significance

Quantitative phase imaging (QPI) offers a label-free approach to non-invasively characterize cellular processes by exploiting their refractive index based intrinsic contrast. QPI captures this contrast by translating refractive index associated phase shifts into intensity-based quantifiable data with nanoscale sensitivity. It holds significant potential for advancing precision cancer medicine by providing quantitative characterization of the biophysical properties of cells and tissue in their natural states.

Aim

This perspective aims to discuss the potential of QPI to increase our understanding of cancer development and its response to therapeutics. It also explores new developments in QPI methods towards advancing personalized cancer therapy and early detection.

Approach

We begin by detailing the technical advancements of QPI, examining its implementations across transmission and reflection geometries and phase retrieval methods, both interferometric and non-interferometric. The focus then shifts to QPI's applications in cancer research, including dynamic cell mass imaging for drug response assessment, cancer risk stratification, and in-vivo tissue imaging.

Results

QPI has emerged as a crucial tool in precision cancer medicine, offering insights into tumor biology and treatment efficacy. Its sensitivity to detecting nanoscale changes holds promise for enhancing cancer diagnostics, risk assessment, and prognostication. The future of QPI is envisioned in its integration with artificial intelligence, morpho-dynamics, and spatial biology, broadening its impact in cancer research.

Conclusions

QPI presents significant potential in advancing precision cancer medicine and redefining our approach to cancer diagnosis, monitoring, and treatment. Future directions include harnessing high-throughput dynamic imaging, 3D QPI for realistic tumor models, and combining artificial intelligence with multi-omics data to extend QPI's capabilities. As a result, QPI stands at the forefront of cancer research and clinical application in cancer care.

Capturing cell morphology dynamics with high temporal resolution using single-shot quantitative phase gradient imaging

Significance

Label-free quantitative phase imaging can potentially measure cellular dynamics with minimal perturbation, motivating efforts to develop faster and more sensitive instrumentation. We characterize fast, single-shot quantitative phase gradient microscopy (ss-QPGM) that simultaneously acquires multiple polarization components required to reconstruct phase images. We integrate a computationally efficient least squares algorithm to provide real-time, video-rate imaging (up to 75   frames / s ). The developed instrument was used to observe changes in cellular morphology and correlate these to molecular measures commonly obtained by staining.

Aim

We aim to characterize a fast approach to ss-QPGM and record morphological changes in single-cell phase images. We also correlate these with biochemical changes indicating cell death using concurrently acquired fluorescence images.

Approach

Here, we examine nutrient deprivation and anticancer drug-induced cell death in two different breast cell lines, viz., M2 and MCF7. Our approach involves in-line measurements of ss-QPGM and fluorescence imaging of the cells biochemically labeled for viability.

Results

We validate the accuracy of the phase measurement using a USAF1951 pattern phase target. The ss-QPGM system resolves 912.3    lp / mm , and our analysis scheme accurately retrieves the phase with a high correlation coefficient ( ∼ 0.99 ), as measured by calibrated sample thicknesses. Analyzing the contrast in phase, we estimate the spatial resolution achievable to be 0.55    μ m for this microscope. ss-QPGM time-lapse live-cell imaging reveals multiple intracellular and morphological changes during biochemically induced cell death. Inferences from co-registered images of quantitative phase and fluorescence suggest the possibility of necrosis, which agrees with previous findings.

Conclusions

Label-free ss-QPGM with high-temporal resolution and high spatial fidelity is demonstrated. Its application for monitoring dynamic changes in live cells offers promising prospects.

Continuous distribution of cancer cells in the cell cycle unveiled by AI-segmented imaging of 37,000 HeLa FUCCI cells

Classification of live or fixed cells based on their unlabeled microscopic images would be a powerful tool for cell biology and pathology. For such software, the first step is the generation of a ground truth database that can be used for training and testing AI classification algorithms. The Application of cells expressing fluorescent reporter proteins allows the building of ground truth datasets in a straightforward way. In this study, we present an automated imaging pipeline utilizing the Cellpose algorithm for the precise cell segmentation and measurement of fluorescent cellular intensities across multiple channels. We analyzed the cell cycle of HeLa-FUCCI cells expressing fluorescent red and green reporter proteins at various levels depending on the cell cycle state. To build the dataset, 37,000 fixed cells were automatically scanned using a standard motorized microscope, capturing phase contrast and fluorescent red/green images. The fluorescent pixel intensity of each cell was integrated to calculate the total fluorescence of cells based on cell segmentation in the phase contrast channel. It resulted in a precise intensity value for each cell in both channels. Furthermore, we conducted a comparative analysis of Cellpose 1.0 and Cellpose 2.0 in cell segmentation performance. Cellpose 2.0 demonstrated notable improvements, achieving a significantly reduced false positive rate of 2.7 % and 1.4 % false negative. The cellular fluorescence was visualized in a 2D plot (map) based on the red and green intensities of the FUCCI construct revealing the continuous distribution of cells in the cell cycle. This 2D map enables the selection and potential isolation of single cells in a specific phase. In the corresponding heatmap, two clusters appeared representing cells in the red and green states. Our pipeline allows the high-throughput and accurate measurement of cellular fluorescence providing extensive statistical information on thousands of cells with potential applications in developmental and cancer biology. Furthermore, our method can be used to build ground truth datasets automatically for training and testing AI cell classification. Our automated pipeline can be used to analyze thousands of cells within 2 h after putting the sample onto the microscope.

Improved accuracy for estrous cycle staging using supervised object detection

The estrous cycle regulates reproductive events and hormone changes in female mammals and is analogous to the menstrual cycle in humans. Monitoring this cycle is necessary as it serves as a biomarker for overall health and is crucial for interpreting study results. The estrous cycle comprises four stages influenced by fluctuating levels of hormones, mainly estradiol and progesterone. Tracking the cycle traditionally relies on vaginal cytology, which categorizes stages based on three epithelial cell concentrations. However, this method has limitations, including time-consuming training and variable accuracy among researchers. To address these challenges, this study assessed the feasibility and reliability of two machine learning methods. An object detection-based machine learning model, Object Detection Estrous Staging (ODES), was employed to identify cell types throughout the estrous cycle in mice. A dataset of 555 vaginal cytology images with four different stains was annotated, with 335 images for training, 45 for validation, and 175 for testing. A novel, accurate set of rules for classification was derived by analyzing training images. ODES achieved an average accuracy of 87% in classifying cycle stages and took only 3.9 minutes to analyze 175 test images. The use of object detection machine learning significantly improved accuracy and efficiency compared to previously derived supervised image classification models (33-45% accuracy) and human accuracy (66% accuracy), refining research practices for female studies. These findings facilitate the integration of the estrous cycle into research, enhancing the quality of scientific results by allowing for efficient and accurate identification of the cycle stage.

Cell-TIMP: Cellular Trajectory Inference based on Morphological Parameter

Cellular morphology, shaped by various genetic and environmental influences, is pivotal to studying experimental cell biology, necessitating precise measurement and analysis techniques. Traditional approaches, which rely on geometric metrics derived from stained images, encounter obstacles stemming from both the imaging and analytical domains. Staining processes can disrupt the cell's natural state and diminish accuracy due to photobleaching, while conventional analysis techniques, which categorize cells based on shape to discern pathophysiological conditions, often fail to capture the continuous and asynchronous nature of biological processes such as cell differentiation, immune responses, and cancer progression. In this work, we propose the use of quantitative phase imaging for morphological assessment due to its label-free nature. For analysis, we repurposed the genomic analysis toolbox to perform trajectory inference analysis purely based on morphology information. We applied the developed framework to study the progression of leukemia and breast cancer metastasis. Our approach revealed a clear pattern of morphological evolution tied to the diseases' advancement, highlighting the efficacy of our method in identifying functionally significant shape changes where conventional techniques falter. This advancement offers a fresh perspective on analyzing cellular morphology and holds significant potential for the broader research community, enabling a deeper understanding of complex biological dynamics.

EVATOM: an optical, label-free, machine learning assisted embryo health assessment tool

The combination of a good quality embryo and proper maternal health factors promise higher chances of a successful in vitro fertilization (IVF) procedure leading to clinical pregnancy and live birth. Of these two factors, selection of a good embryo is a controllable aspect. The current gold standard in clinical practice is visual assessment of an embryo based on its morphological appearance by trained embryologists. More recently, machine learning has been incorporated into embryo selection "packages". Here, we report EVATOM: a machine-learning assisted embryo health assessment tool utilizing an optical quantitative phase imaging technique called artificial confocal microscopy (ACM). We present a label-free nucleus detection method with, to the best of our knowledge, novel quantitative embryo health biomarkers. Two viability assessment models are presented for grading embryos into two classes: healthy/intermediate (H/I) or sick (S) class. The models achieve a weighted F1 score of 1.0 and 0.99 respectively on the in-distribution test set of 72 fixed embryos and a weighted F1 score of 0.9 and 0.95 respectively on the out-of-distribution test dataset of 19 time-instances from 8 live embryos.

Beyond fluorescence: advances in computational label-free full specificity in 3D quantitative phase microscopy

Despite remarkable progresses in quantitative phase imaging (QPI) microscopes, their wide acceptance is limited due to the lack of specificity compared with the well-established fluorescence microscopy. In fact, the absence of fluorescent tag prevents to identify subcellular structures in single cells, making challenging the interpretation of label-free 2D and 3D phase-contrast data. Great effort has been made by many groups worldwide to address and overcome such limitation. Different computational methods have been proposed and many more are currently under investigation to achieve label-free microscopic imaging at single-cell level to recognize and quantify different subcellular compartments. This route promises to bridge the gap between QPI and FM for real-world applications.

On the use of deep learning for phase recovery

Phase recovery (PR) refers to calculating the phase of the light field from its intensity measurements. As exemplified from quantitative phase imaging and coherent diffraction imaging to adaptive optics, PR is essential for reconstructing the refractive index distribution or topography of an object and correcting the aberration of an imaging system. In recent years, deep learning (DL), often implemented through deep neural networks, has provided unprecedented support for computational imaging, leading to more efficient solutions for various PR problems. In this review, we first briefly introduce conventional methods for PR. Then, we review how DL provides support for PR from the following three stages, namely, pre-processing, in-processing, and post-processing. We also review how DL is used in phase image processing. Finally, we summarize the work in DL for PR and provide an outlook on how to better use DL to improve the reliability and efficiency of PR. Furthermore, we present a live-updating resource ( https://github.com/kqwang/phase-recovery ) for readers to learn more about PR.

Label-Free Intracellular Multi-Specificity in Yeast Cells by Phase-Contrast Tomographic Flow Cytometry

In-flow phase-contrast tomography provides a 3D refractive index of label-free cells in cytometry systems. Its major limitation, as with any quantitative phase imaging approach, is the lack of specificity compared to fluorescence microscopy, thus restraining its huge potentialities in single-cell analysis and diagnostics. Remarkable results in introducing specificity are obtained through artificial intelligence (AI), but only for adherent cells. However, accessing the 3D fluorescence ground truth and obtaining accurate voxel-level co-registration of image pairs for AI training is not viable for high-throughput cytometry. The recent statistical inference approach is a significant step forward for label-free specificity but remains limited to cells' nuclei. Here, a generalized computational strategy based on a self-consistent statistical inference to achieve intracellular multi-specificity is shown. Various subcellular compartments (i.e., nuclei, cytoplasmic vacuoles, the peri-vacuolar membrane area, cytoplasm, vacuole-nucleus contact site) can be identified and characterized quantitatively at different phases of the cells life cycle by using yeast cells as a biological model. Moreover, for the first time, virtual reality is introduced for handling the information content of multi-specificity in single cells. Full fruition is proofed for exploring and interacting with 3D quantitative biophysical parameters of the identified compartments on demand, thus opening the route to a metaverse for 3D microscopy.

Panoramic quantitative phase imaging of adherent live cells in a microfluidic environment

Understanding how cells respond to external stimuli is crucial. However, there are a lack of inspection systems capable of simultaneously stimulating and imaging cells, especially in their natural states. This study presents a novel microfluidic stimulation and observation system equipped with flat-fielding quantitative phase contrast microscopy (FF-QPCM). This system allowed us to track the behavior of organelles in live cells experiencing controlled microfluidic stimulation. Using this innovative imaging platform, we successfully quantified the cellular response to shear stress including directional cellular shrinkage and mitochondrial distribution change in a label-free manner. Additionally, we detected and characterized the cellular response, particularly mitochondrial behavior, under varying fluidic conditions such as temperature and drug induction time. The proposed imaging platform is highly suitable for various microfluidic applications at the organelle level. We advocate that this platform will significantly facilitate life science research in microfluidic environments.

Phenotyping neuroblastoma cells through intelligent scrutiny of stain-free biomarkers in holographic flow cytometry

To efficiently tackle certain tumor types, finding new biomarkers for rapid and complete phenotyping of cancer cells is highly demanded. This is especially the case for the most common pediatric solid tumor of the sympathetic nervous system, namely, neuroblastoma (NB). Liquid biopsy is in principle a very promising tool for this purpose, but usually enrichment and isolation of circulating tumor cells in such patients remain difficult due to the unavailability of universal NB cell-specific surface markers. Here, we show that rapid screening and phenotyping of NB cells through stain-free biomarkers supported by artificial intelligence is a viable route for liquid biopsy. We demonstrate the concept through a flow cytometry based on label-free holographic quantitative phase-contrast microscopy empowered by machine learning. In detail, we exploit a hierarchical decision scheme where at first level NB cells are classified from monocytes with 97.9% accuracy. Then we demonstrate that different phenotypes are discriminated within NB class. Indeed, for each cell classified as NB its belonging to one of four NB sub-populations (i.e., CHP212, SKNBE2, SHSY5Y, and SKNSH) is evaluated thus achieving accuracy in the range 73.6%-89.1%. The achieved results solve the realistic problem related to the identification circulating tumor cell, i.e., the possibility to recognize and detect tumor cells morphologically similar to blood cells, which is the core issue in liquid biopsy based on stain-free microscopy. The presented approach operates at lab-on-chip scale and emulates real-world scenarios, thus representing a future route for liquid biopsy by exploiting intelligent biomedical imaging.

CellSNAP: A fast, accurate algorithm for 3D cell segmentation in quantitative phase imaging

Quantitative phase imaging (QPI) has rapidly emerged as a complementary tool to fluorescence imaging, as it provides an objective measure of cell morphology and dynamics, free of variability due to contrast agents. In particular, three-dimensional (3D) tomographic imaging of live cells has opened up new directions of investigation by providing systematic and correlative analysis of various cellular parameters without limitations of photobleaching and phototoxicity. While current QPI systems allow the rapid acquisition of tomographic images, the pipeline to analyze these raw 3D tomograms is not well-developed. This work focuses on a critical, yet often underappreciated, step of the analysis pipeline, that of 3D cell segmentation from the acquired tomograms. The current method employed for such tasks is the Otsu-based 3D watershed algorithm, which works well for isolated cells; however, it is very challenging to draw boundaries when the cells are clumped. This process is also memory intensive since the processing requires computation on a 3D stack of images. We report the CellSNAP (Cell Segmentation via Novel Algorithm for Phase Imaging) algorithm for the segmentation of QPI images, which outstrips the current gold standard in terms of speed, robustness, and implementation, achieving cell segmentation under 2 seconds per cell on a single-core processor. The implementation of CellSNAP can easily be parallelized on a multi-core system for further speed improvements. For the cases where segmentation is possible with the existing standard method, our algorithm displays an average difference of 5% for dry mass and 8% for volume measurements. We also show that CellSNAP can handle challenging image datasets where cells are clumped and marred by interferogram drifts, which pose major difficulties for all QPI-focused segmentation tools. We envision our work will lead to the broader adoption of QPI imaging for high-throughput analysis, which has, in part, been stymied by a lack of suitable image segmentation tools.

Machine learning assisted health viability assay for mouse embryos with artificial confocal microscopy (ACM)

The combination of a good quality embryo and proper maternal health factors promise higher chances of a successful in vitro fertilization (IVF) procedure leading to clinical pregnancy and live birth. Of these two factors, selection of a good embryo is a controllable aspect. The current gold standard in clinical practice is visual assessment of an embryo based on its morphological appearance by trained embryologists. More recently, machine learning has been incorporated into embryo selection "packages". Here, we report a machine-learning assisted embryo health assessment tool utilizing a quantitative phase imaging technique called artificial confocal microscopy (ACM). We present a label-free nucleus detection method with novel quantitative embryo health biomarkers. Two viability assessment models are presented for grading embryos into two classes: healthy/intermediate (H/I) or sick (S) class. The models achieve a weighted F1 score of 1.0 and 0.99 respectively on the in-distribution test set of 72 fixed embryos and a weighted F1 score of 0.9 and 0.95 respectively on the out-of-distribution test dataset of 19 time-instances from 8 live embryos.

Digital staining facilitates biomedical microscopy

Traditional staining of biological specimens for microscopic imaging entails time-consuming, laborious, and costly procedures, in addition to producing inconsistent labeling and causing irreversible sample damage. In recent years, computational "virtual" staining using deep learning techniques has evolved into a robust and comprehensive application for streamlining the staining process without typical histochemical staining-related drawbacks. Such virtual staining techniques can also be combined with neural networks designed to correct various microscopy aberrations, such as out-of-focus or motion blur artifacts, and improve upon diffracted-limited resolution. Here, we highlight how such methods lead to a host of new opportunities that can significantly improve both sample preparation and imaging in biomedical microscopy.

Microsystem Advances through Integration with Artificial Intelligence

Microfluidics is a rapidly growing discipline that involves studying and manipulating fluids at reduced length scale and volume, typically on the scale of micro- or nanoliters. Under the reduced length scale and larger surface-to-volume ratio, advantages of low reagent consumption, faster reaction kinetics, and more compact systems are evident in microfluidics. However, miniaturization of microfluidic chips and systems introduces challenges of stricter tolerances in designing and controlling them for interdisciplinary applications. Recent advances in artificial intelligence (AI) have brought innovation to microfluidics from design, simulation, automation, and optimization to bioanalysis and data analytics. In microfluidics, the Navier-Stokes equations, which are partial differential equations describing viscous fluid motion that in complete form are known to not have a general analytical solution, can be simplified and have fair performance through numerical approximation due to low inertia and laminar flow. Approximation using neural networks trained by rules of physical knowledge introduces a new possibility to predict the physicochemical nature. The combination of microfluidics and automation can produce large amounts of data, where features and patterns that are difficult to discern by a human can be extracted by machine learning. Therefore, integration with AI introduces the potential to revolutionize the microfluidic workflow by enabling the precision control and automation of data analysis. Deployment of smart microfluidics may be tremendously beneficial in various applications in the future, including high-throughput drug discovery, rapid point-of-care-testing (POCT), and personalized medicine. In this review, we summarize key microfluidic advances integrated with AI and discuss the outlook and possibilities of combining AI and microfluidics.

A machine learning based approach for quantitative evaluation of cell migration in Transwell assays based on deformation characteristics

Many pathological and physiological processes, including embryonic development, immune response and cancer metastasis, involve studies on cell migration, and especially detection methods, for which it is difficult to satisfy the requirements for rapid and quantitative evaluation and analysis. In view of the shortcomings in simultaneously quantifying the number of migrated cells and non-migrated cells using Transwell assays, we propose a novelty approach for the evaluation of cell migration by distinguishing whether the cells have migrated based on the regularity of the cell morphology changes. Traditionally, the status of living cells and dead cells are detected and analyzed by machine learning using some common morphological characteristics, e.g., area and perimeter of the cells. However, the accuracy of detecting whether cells have migrated or not using these common characteristics is not high, and the characteristics are not appropriate for our studies. Therefore, from the point of view of mechanism analysis for the migration behavior, we examined the regularity of different morphology changes of migrated cells and non-migrated cells, and thus discovered the distinguishable morphological characteristics. Then, two deformation characteristics, deformation index and taper index are proposed. Then, a machine learning based algorithm that can identify migrated cells according to the proposed deformation characteristics was devised. In addition, images of migrated cells and non-migrated cells were obtained from the Transwell assays. This algorithm was trained, and was able to successfully identify migrated cells with an accuracy of 84% using the proposed morphological characteristics. This method greatly improves the identification accuracy when compared with the identification of traditional characteristics of which the accuracy was about 54.7%. This machine learning based method might be employed as a potential tool for cell counting and evaluation of cell migration with the aim of reducing time and improving automation compared with the traditional method. This method is effective, rapid, and incorporate advances in artificial intelligence which could be used for adapting the current evaluation methods.

Developing a Reliable Holographic Flow Cyto-Tomography Apparatus by Optimizing the Experimental Layout and Computational Processing

Digital Holographic Tomography (DHT) has recently been established as a means of retrieving the 3D refractive index mapping of single cells. To make DHT a viable system, it is necessary to develop a reliable and robust holographic apparatus in order that such technology can be utilized outside of specialized optics laboratories and operated in the in-flow modality. In this paper, we propose a quasi-common-path lateral-shearing holographic optical set-up to be used, for the first time, for DHT in a flow-cytometer modality. The proposed solution is able to withstand environmental vibrations that can severely affect the interference process. Furthermore, we have scaled down the system while ensuring that a full 360° rotation of the cells occurs in the field-of-view, in order to retrieve 3D phase-contrast tomograms of single cells flowing along a microfluidic channel. This was achieved by setting the camera sensor at 45° with respect to the microfluidic direction. Additional optimizations were made to the computational elements to ensure the reliable retrieval of 3D refractive index distributions by demonstrating an effective method of tomographic reconstruction, based on high-order total variation. The results were first demonstrated using realistic 3D numerical phantom cells to assess the performance of the proposed high-order total variation method in comparison with the gold-standard algorithm for tomographic reconstructions: namely, filtered back projection. Then, the proposed DHT system and the processing pipeline were experimentally validated for monocytes and mouse embryonic fibroblast NIH-3T3 cells lines. Moreover, the repeatability of these tomographic measurements was also investigated by recording the same cell multiple times and quantifying the ability to provide reliable and comparable tomographic reconstructions, as confirmed by a correlation coefficient greater than 95%. The reported results represent various steps forward in several key aspects of in-flow DHT, thus paving the way for its use in real-world applications.