Automated Cell Segmentation for Quantitative Phase Microscopy

CellSNAP: a fast, accurate algorithm for 3D cell segmentation in quantitative phase imaging

Significance

Three-dimensional quantitative phase imaging (QPI) has rapidly emerged as a complementary tool to fluorescence imaging, as it provides an objective measure of cell morphology and dynamics, free of variability due to contrast agents. It has opened up new directions of investigation by providing systematic and correlative analysis of various cellular parameters without limitations of photobleaching and phototoxicity. While current QPI systems allow the rapid acquisition of tomographic images, the pipeline to analyze these raw three-dimensional (3D) tomograms is not well-developed. We focus on a critical, yet often underappreciated, step of the analysis pipeline that of 3D cell segmentation from the acquired tomograms.

Aim

We report the CellSNAP (Cell Segmentation via Novel Algorithm for Phase Imaging) algorithm for the 3D segmentation of QPI images.

Approach

The cell segmentation algorithm mimics the gemstone extraction process, initiating with a coarse 3D extrusion from a two-dimensional (2D) segmented mask to outline the cell structure. A 2D image is generated, and a segmentation algorithm identifies the boundary in the x - y plane. Leveraging cell continuity in consecutive z -stacks, a refined 3D segmentation, akin to fine chiseling in gemstone carving, completes the process.

Results

The CellSNAP algorithm outstrips the current gold standard in terms of speed, robustness, and implementation, achieving cell segmentation under 2 s per cell on a single-core processor. The implementation of CellSNAP can easily be parallelized on a multi-core system for further speed improvements. For the cases where segmentation is possible with the existing standard method, our algorithm displays an average difference of 5% for dry mass and 8% for volume measurements. We also show that CellSNAP can handle challenging image datasets where cells are clumped and marred by interferogram drifts, which pose major difficulties for all QPI-focused AI-based segmentation tools.

Conclusion

Our proposed method is less memory intensive and significantly faster than existing methods. The method can be easily implemented on a student laptop. Since the approach is rule-based, there is no need to collect a lot of imaging data and manually annotate them to perform machine learning based training of the model. We envision our work will lead to broader adoption of QPI imaging for high-throughput analysis, which has, in part, been stymied by a lack of suitable image segmentation tools.

CellSNAP: A fast, accurate algorithm for 3D cell segmentation in quantitative phase imaging

Quantitative phase imaging (QPI) has rapidly emerged as a complementary tool to fluorescence imaging, as it provides an objective measure of cell morphology and dynamics, free of variability due to contrast agents. In particular, three-dimensional (3D) tomographic imaging of live cells has opened up new directions of investigation by providing systematic and correlative analysis of various cellular parameters without limitations of photobleaching and phototoxicity. While current QPI systems allow the rapid acquisition of tomographic images, the pipeline to analyze these raw 3D tomograms is not well-developed. This work focuses on a critical, yet often underappreciated, step of the analysis pipeline, that of 3D cell segmentation from the acquired tomograms. The current method employed for such tasks is the Otsu-based 3D watershed algorithm, which works well for isolated cells; however, it is very challenging to draw boundaries when the cells are clumped. This process is also memory intensive since the processing requires computation on a 3D stack of images. We report the CellSNAP (Cell Segmentation via Novel Algorithm for Phase Imaging) algorithm for the segmentation of QPI images, which outstrips the current gold standard in terms of speed, robustness, and implementation, achieving cell segmentation under 2 seconds per cell on a single-core processor. The implementation of CellSNAP can easily be parallelized on a multi-core system for further speed improvements. For the cases where segmentation is possible with the existing standard method, our algorithm displays an average difference of 5% for dry mass and 8% for volume measurements. We also show that CellSNAP can handle challenging image datasets where cells are clumped and marred by interferogram drifts, which pose major difficulties for all QPI-focused segmentation tools. We envision our work will lead to the broader adoption of QPI imaging for high-throughput analysis, which has, in part, been stymied by a lack of suitable image segmentation tools.

A review of open-source image analysis tools for mammalian cell culture: algorithms, features and implementations

Cell culture is undeniably important for multiple scientific applications, including pharmaceuticals, transplants, and cosmetics. However, cell culture involves multiple manual steps, such as regularly analyzing cell images for their health and morphology. Computer scientists have developed algorithms to automate cell imaging analysis, but they are not widely adopted by biologists, especially those lacking an interactive platform. To address the issue, we compile and review existing open-source cell image processing tools that provide interactive interfaces for management and prediction tasks. We highlight the prediction tools that can detect, segment, and track different mammalian cell morphologies across various image modalities and present a comparison of algorithms and unique features of these tools, whether they work locally or in the cloud. This would guide non-experts to determine which is best suited for their purposes and, developers to acknowledge what is worth further expansion. In addition, we provide a general discussion on potential implementations of the tools for a more extensive scope, which guides the reader to not restrict them to prediction tasks only. Finally, we conclude the article by stating new considerations for the development of interactive cell imaging tools and suggesting new directions for future research.

Phase Contrast Image Restoration by Formulating Its Imaging Principle and Reversing the Formulation With Deep Neural Networks

Phase contrast microscopy, as a noninvasive imaging technique, has been widely used to monitor the behavior of transparent cells without staining or altering them. Due to the optical principle of the specifically-designed microscope, phase contrast microscopy images contain artifacts such as halo and shade-off which hinder the cell segmentation and detection tasks. Some previous works developed simplified computational imaging models for phase contrast microscopes by linear approximations and convolutions. The approximated models do not exactly reflect the imaging principle of the phase contrast microscope and accordingly the image restoration by solving the corresponding deconvolution process is not perfect. In this paper, we revisit the optical principle of the phase contrast microscope to precisely formulate its imaging model without any approximation. Based on this model, we propose an image restoration procedure by reversing this imaging model with a deep neural network, instead of mathematically deriving the inverse operator of the model which is technically impossible. Extensive experiments are conducted to demonstrate the superiority of the newly derived phase contrast microscopy imaging model and the power of the deep neural network on modeling the inverse imaging procedure. Moreover, the restored images enable that high quality cell segmentation task can be easily achieved by simply thresholding methods. Implementations of this work are publicly available at https://github.com/LiangHann/Phase-Contrast-Microscopy-Image-Restoration.

GEMA-An Automatic Segmentation Method for Real-Time Analysis of Mammalian Cell Growth in Microfluidic Devices

Nowadays, image analysis has a relevant role in most scientific and research areas. This process is used to extract and understand information from images to obtain a model, knowledge, and rules in the decision process. In the case of biological areas, images are acquired to describe the behavior of a biological agent in time such as cells using a mathematical and computational approach to generate a system with automatic control. In this paper, MCF7 cells are used to model their growth and death when they have been injected with a drug. These mammalian cells allow understanding of behavior, gene expression, and drug resistance to breast cancer. For this, an automatic segmentation method called GEMA is presented to analyze the apoptosis and confluence stages of culture by measuring the increase or decrease of the image area occupied by cells in microfluidic devices. In vitro, the biological experiments can be analyzed through a sequence of images taken at specific intervals of time. To automate the image segmentation, the proposed algorithm is based on a Gabor filter, a coefficient of variation (CV), and linear regression. This allows the processing of images in real time during the evolution of biological experiments. Moreover, GEMA has been compared with another three representative methods such as gold standard (manual segmentation), morphological gradient, and a semi-automatic algorithm using FIJI. The experiments show promising results, due to the proposed algorithm achieving an accuracy above 90% and a lower computation time because it requires on average 1 s to process each image. This makes it suitable for image-based real-time automatization of biological lab-on-a-chip experiments.

Loss of KAP3 decreases intercellular adhesion and impairs intracellular transport of laminin in signet ring cell carcinoma of the stomach

Signet-ring cell carcinoma (SRCC) is a unique subtype of gastric cancer that is impaired for cell-cell adhesion. The pathogenesis of SRCC remains unclear. Here, we show that expression of kinesin-associated protein 3 (KAP3), a cargo adaptor subunit of the kinesin superfamily protein 3 (KIF3), a motor protein, is specifically decreased in SRCC of the stomach. CRISPR/Cas9-mediated gene knockout experiments indicated that loss of KAP3 impairs the formation of circumferential actomyosin cables by inactivating RhoA, leading to the weakening of cell-cell adhesion. Furthermore, in KAP3 knockout cells, post-Golgi transport of laminin, a key component of the basement membrane, was inhibited, resulting in impaired basement membrane formation. Together, these findings uncover a potential role for KAP3 in the pathogenesis of SRCC of the stomach.

Cellular analysis using label-free parallel array microscopy with Fourier ptychography

Quantitative phase imaging (QPI) is an ideal method to non-invasively monitor cell populations and provide label-free imaging and analysis. QPI offers enhanced sample characterization and cell counting compared to conventional label-free techniques. We demonstrate this in the current study through a comparison of cell counting data from digital phase contrast (DPC) imaging and from QPI using a system based on Fourier ptychographic microscopy (FPM). Our FPM system offers multi-well, parallel imaging and a QPI-specific cell segmentation method to establish automated and reliable cell counting. Three cell types were studied and FPM showed improvement in the ability to resolve fine details and thin cells, despite limitations of the FPM system incurred by imaging artifacts. Relative to manually counted fluorescence ground-truth, cell counting results after automated segmentation showed improved accuracy with QPI over DPC.

Self-supervised pretraining for transferable quantitative phase image cell segmentation

In this paper, a novel U-Net-based method for robust adherent cell segmentation for quantitative phase microscopy image is designed and optimised. We designed and evaluated four specific post-processing pipelines. To increase the transferability to different cell types, non-deep learning transfer with adjustable parameters is used in the post-processing step. Additionally, we proposed a self-supervised pretraining technique using nonlabelled data, which is trained to reconstruct multiple image distortions and improved the segmentation performance from 0.67 to 0.70 of object-wise intersection over union. Moreover, we publish a new dataset of manually labelled images suitable for this task together with the unlabelled data for self-supervised pretraining.

A Joint Tracking Approach via Ant Colony Evolution for Quantitative Cell Cycle Analysis

In this paper, we use an ant colony heuristic method to tackle the integration of data association and state estimation in the presence of cell mitosis, morphological change and uncertainty of measurement. Our approach first models the scouting behavior of an unlabeled ant colony as a chaotic process to generate a set of cell candidates in the current frame, then a labeled ant colony foraging process is modeled to construct an interframe matching between previously estimated cell states and current cell candidates through minimizing the optimal sub-pattern assignment metric for track (OSPA-T). The states of cells in the current frame are finally estimated using labeled ant colonies via a multi-Bernoulli parameter set approximated by individual food pheromone fields and heuristic information within the same region of support, the resulting trail pheromone fields over frames constitutes the cell lineage trees of the tracks. A four-stage track recovery strategy is proposed to monitor the history of all established tracks to reconstruct broken tracks in a computationally economic way. The labeling method used in this work is an improvement on previous techniques. The method has been evaluated on publicly available, challenging cell image sequences, and a satisfied performance improvement is achieved in contrast to the state-of-the-art methods.

Biophysical phenotyping of mesenchymal stem cells along the osteogenic differentiation pathway

Mesenchymal stem cells represent an important resource, for bone regenerative medicine and therapeutic applications. This review focuses on new advancements and biophysical tools which exploit different physical and chemical markers of mesenchymal stem cell populations, to finely characterize phenotype changes along their osteogenic differentiation process. Special attention is paid to recently developed label-free methods, which allow monitoring cell populations with minimal invasiveness. Among them, quantitative phase imaging, suitable for single-cell morphometric analysis, and nanoindentation, functional to cellular biomechanics investigation. Moreover, the pool of ion channels expressed in cells during differentiation is discussed, with particular interest for calcium homoeostasis.Altogether, a biophysical perspective of osteogenesis is proposed, offering a valuable tool for the assessment of the cell stage, but also suggesting potential physiological links between apparently independent phenomena.

Acoustofluidic phase microscopy in a tilted segmentation-free configuration

A low-cost device for registration-free quantitative phase microscopy (QPM) based on the transport of intensity equation of cells in continuous flow is presented. The method uses acoustic focusing to align cells into a single plane where all cells move at a constant speed. The acoustic focusing plane is tilted with respect to the microscope's focal plane in order to obtain cell images at multiple focal positions. As the cells are displaced at constant speed, phase maps can be generated without the need to segment and register individual objects. The proposed inclined geometry allows for the acquisition of a vertical stack without the need for any moving part, and it enables a cost-effective and robust implementation of QPM. The suitability of the solution for biological imaging is tested on blood samples, demonstrating the ability to recover the phase map of single red blood cells flowing through the microchip.

Practical application of microsphere samples for benchmarking a quantitative phase imaging system

Quantitative phase imaging (QPI) provides an approach for monitoring the dry mass of individual cells by measuring the optical pathlength of visible light as it passes through cells. A distinct advantage of QPI is that the measurements result in optical path length quantities that are, in principle, instrument independent. Reference materials that induce a well‐defined optical pathlength shift and are compatible with QPI imaging systems will be valuable in assuring the accuracy of such measurements on different instruments. In this study, we evaluate seven combinations of microspheres embedded in index refraction matching media as candidate reference materials for benchmarking the performance of a QPI system and as calibration standards for the optical pathlength measurement. Poly(methyl metharylate) microspheres and mineral oil were used to evaluate the range of illumination apertures, signal‐to‐noise ratios, and focus positions that allow an accurate quantitative optical pathlength measurement. The microsphere‐based reference material can be used to verify settings on an instrument that are suitable for obtaining an accurate pathlength measurement from biological cells. The microsphere/media reference material is applied to QPI‐based dry mass measurements of a population of HEK293 cells to benchmark and provide evidence that the QPI image data are accurate.

Quantitative Phase Dynamics of Cancer Cell Populations Affected by Blue Light

Increased exposition to blue light may induce many changes in cell behavior and significantly affect the critical characteristics of cells. Here we show that multimodal holographic microscopy (MHM) within advanced image analysis is capable of correctly distinguishing between changes in cell motility, cell dry mass, cell density, and cell death induced by blue light. We focused on the effect of blue light with a wavelength of 485 nm on morphological and dynamical parameters of four cell lines, malignant PC-3, A2780, G361 cell lines, and the benign PNT1A cell line. We used MHM with blue light doses 24 mJ/cm2, 208 mJ/cm2 and two kinds of expositions (500 and 1000 ms) to acquire real-time quantitative phase information about cellular parameters. It has been shown that specific doses of the blue light significantly influence cell motility, cell dry mass and cell density. These changes were often specific for the malignant status of tested cells. Blue light dose 208 mJ/cm2 × 1000 ms affected malignant cell motility but did not change the motility of benign cell line PNT1A. This light dose also significantly decreased proliferation activity in all tested cell lines but was not so deleterious for benign cell line PNT1A as for malignant cells. Light dose 208 mJ/cm2 × 1000 ms oppositely affected cell mass in A2780 and PC-3 cells and induced different types of cell death in A2780 and G361 cell lines. Cells obtained the least damage on lower doses of light with shorter time of exposition.

Cell segmentation methods for label-free contrast microscopy: review and comprehensive comparison

BACKGROUND

Because of its non-destructive nature, label-free imaging is an important strategy for studying biological processes. However, routine microscopic techniques like phase contrast or DIC suffer from shadow-cast artifacts making automatic segmentation challenging. The aim of this study was to compare the segmentation efficacy of published steps of segmentation work-flow (image reconstruction, foreground segmentation, cell detection (seed-point extraction) and cell (instance) segmentation) on a dataset of the same cells from multiple contrast microscopic modalities.

RESULTS

We built a collection of routines aimed at image segmentation of viable adherent cells grown on the culture dish acquired by phase contrast, differential interference contrast, Hoffman modulation contrast and quantitative phase imaging, and we performed a comprehensive comparison of available segmentation methods applicable for label-free data. We demonstrated that it is crucial to perform the image reconstruction step, enabling the use of segmentation methods originally not applicable on label-free images. Further we compared foreground segmentation methods (thresholding, feature-extraction, level-set, graph-cut, learning-based), seed-point extraction methods (Laplacian of Gaussians, radial symmetry and distance transform, iterative radial voting, maximally stable extremal region and learning-based) and single cell segmentation methods. We validated suitable set of methods for each microscopy modality and published them online.

CONCLUSIONS

We demonstrate that image reconstruction step allows the use of segmentation methods not originally intended for label-free imaging. In addition to the comprehensive comparison of methods, raw and reconstructed annotated data and Matlab codes are provided.