CellSNAP: a fast, accurate algorithm for 3D cell segmentation in quantitative phase imaging

Significance

Three-dimensional quantitative phase imaging (QPI) has rapidly emerged as a complementary tool to fluorescence imaging, as it provides an objective measure of cell morphology and dynamics, free of variability due to contrast agents. It has opened up new directions of investigation by providing systematic and correlative analysis of various cellular parameters without limitations of photobleaching and phototoxicity. While current QPI systems allow the rapid acquisition of tomographic images, the pipeline to analyze these raw three-dimensional (3D) tomograms is not well-developed. We focus on a critical, yet often underappreciated, step of the analysis pipeline that of 3D cell segmentation from the acquired tomograms.

Aim

We report the CellSNAP (Cell Segmentation via Novel Algorithm for Phase Imaging) algorithm for the 3D segmentation of QPI images.

Approach

The cell segmentation algorithm mimics the gemstone extraction process, initiating with a coarse 3D extrusion from a two-dimensional (2D) segmented mask to outline the cell structure. A 2D image is generated, and a segmentation algorithm identifies the boundary in the x - y plane. Leveraging cell continuity in consecutive z -stacks, a refined 3D segmentation, akin to fine chiseling in gemstone carving, completes the process.

Results

The CellSNAP algorithm outstrips the current gold standard in terms of speed, robustness, and implementation, achieving cell segmentation under 2 s per cell on a single-core processor. The implementation of CellSNAP can easily be parallelized on a multi-core system for further speed improvements. For the cases where segmentation is possible with the existing standard method, our algorithm displays an average difference of 5% for dry mass and 8% for volume measurements. We also show that CellSNAP can handle challenging image datasets where cells are clumped and marred by interferogram drifts, which pose major difficulties for all QPI-focused AI-based segmentation tools.

Conclusion

Our proposed method is less memory intensive and significantly faster than existing methods. The method can be easily implemented on a student laptop. Since the approach is rule-based, there is no need to collect a lot of imaging data and manually annotate them to perform machine learning based training of the model. We envision our work will lead to broader adoption of QPI imaging for high-throughput analysis, which has, in part, been stymied by a lack of suitable image segmentation tools.

An approach of separating the overlapped cells or nuclei based on the outer Canny edges and morphological erosion

In biomedicine, the automatic processing of medical microscope images plays a key role in the subsequent analysis and diagnosis. Cell or nucleus segmentation is one of the most challenging tasks for microscope image processing. Due to the frequently occurred overlapping, few segmentation methods can achieve satisfactory segmentation accuracy yet. In this paper, we propose an approach to separate the overlapped cells or nuclei based on the outer Canny edges and morphological erosion. The threshold selection is first used to segment the foreground and background of cell or nucleus images. For each binary connected domain in the segmentation image, an intersection based edge selection method is proposed to choose the outer Canny edges of the overlapped cells or nuclei. The outer Canny edges are used to generate a binary cell or nucleus image that is then used to compute the cell or nucleus seeds by the proposed morphological erosion method. The nuclei of the Human U2OS cells, the mouse NIH3T3 cells and the synthetic cells are used for evaluating our proposed approach. The quantitative quantification accuracy is computed by the Dice score and 95.53% is achieved by the proposed approach. Both the quantitative and the qualitative comparisons show that the accuracy of the proposed approach is better than those of the area constrained morphological erosion (ACME) method, the iterative erosion (IE) method, the morphology and watershed (MW) method, the Generalized Laplacian of Gaussian filters (GLGF) method and ellipse fitting (EF) method in separating the cells or nuclei in three publicly available datasets.

U-NTCA: nnUNet and nested transformer with channel attention for corneal cell segmentation

Background

Automatic segmentation of corneal stromal cells can assist ophthalmologists to detect abnormal morphology in confocal microscopy images, thereby assessing the virus infection or conical mutation of corneas, and avoiding irreversible pathological damage. However, the corneal stromal cells often suffer from uneven illumination and disordered vascular occlusion, resulting in inaccurate segmentation.

Methods

In response to these challenges, this study proposes a novel approach: a nnUNet and nested Transformer-based network integrated with dual high-order channel attention, named U-NTCA. Unlike nnUNet, this architecture allows for the recursive transmission of crucial contextual features and direct interaction of features across layers to improve the accuracy of cell recognition in low-quality regions. The proposed methodology involves multiple steps. Firstly, three underlying features with the same channel number are sent into an attention channel named gnConv to facilitate higher-order interaction of local context. Secondly, we leverage different layers in U-Net to integrate Transformer nested with gnConv, and concatenate multiple Transformers to transmit multi-scale features in a bottom-up manner. We encode the downsampling features, corresponding upsampling features, and low-level feature information transmitted from lower layers to model potential correlations between features of varying sizes and resolutions. These multi-scale features play a pivotal role in refining the position information and morphological details of the current layer through recursive transmission.

Results

Experimental results on a clinical dataset including 136 images show that the proposed method achieves competitive performance with a Dice score of 82.72% and an AUC (Area Under Curve) of 90.92%, which are higher than the performance of nnUNet.

Conclusion

The experimental results indicate that our model provides a cost-effective and high-precision segmentation solution for corneal stromal cells, particularly in challenging image scenarios.

3DCellComposer - A Versatile Pipeline Utilizing 2D Cell Segmentation Methods for 3D Cell Segmentation

Background

Cell segmentation is crucial in bioimage informatics, as its accuracy directly impacts conclusions drawn from cellular analyses. While many approaches to 2D cell segmentation have been described, 3D cell segmentation has received much less attention. 3D segmentation faces significant challenges, including limited training data availability due to the difficulty of the task for human annotators, and inherent three-dimensional complexity. As a result, existing 3D cell segmentation methods often lack broad applicability across different imaging modalities.

Results

To address this, we developed a generalizable approach for using 2D cell segmentation methods to produce accurate 3D cell segmentations. We implemented this approach in 3DCellComposer, a versatile, open-source package that allows users to choose any existing 2D segmentation model appropriate for their tissue or cell type(s) without requiring any additional training. Importantly, we have enhanced our open source CellSegmentationEvaluator quality evaluation tool to support 3D images. It provides metrics that allow selection of the best approach for a given imaging source and modality, without the need for human annotations to assess performance. Using these metrics, we demonstrated that our approach produced high-quality 3D segmentations of tissue images, and that it could outperform an existing 3D segmentation method on the cell culture images with which it was trained.

Conclusions

3DCellComposer, when paired with well-trained 2D segmentation models, provides an important alternative to acquiring human-annotated 3D images for new sample types or imaging modalities and then training 3D segmentation models using them. It is expected to be of significant value for large scale projects such as the Human BioMolecular Atlas Program.

A Foundation Model for Cell Segmentation

Cells are a fundamental unit of biological organization, and identifying them in imaging data - cell segmentation - is a critical task for various cellular imaging experiments. While deep learning methods have led to substantial progress on this problem, most models in use are specialist models that work well for specific domains. Methods that have learned the general notion of "what is a cell" and can identify them across different domains of cellular imaging data have proven elusive. In this work, we present CellSAM, a foundation model for cell segmentation that generalizes across diverse cellular imaging data. CellSAM builds on top of the Segment Anything Model (SAM) by developing a prompt engineering approach for mask generation. We train an object detector, CellFinder, to automatically detect cells and prompt SAM to generate segmentations. We show that this approach allows a single model to achieve human-level performance for segmenting images of mammalian cells (in tissues and cell culture), yeast, and bacteria collected across various imaging modalities. We show that CellSAM has strong zero-shot performance and can be improved with a few examples via few-shot learning. We also show that CellSAM can unify bioimaging analysis workflows such as spatial transcriptomics and cell tracking. A deployed version of CellSAM is available at https://cellsam.deepcell.org/.

Segmentation, feature extraction and classification of leukocytes leveraging neural networks, a comparative study

The gold standard of leukocyte differentiation is a manual examination of blood smears, which is not only time and labor intensive but also susceptible to human error. As to automatic classification, there is still no comparative study of cell segmentation, feature extraction, and cell classification, where a variety of machine and deep learning models are compared with home-developed approaches. In this study, both traditional machine learning of K-means clustering versus deep learning of U-Net, U-Net + ResNet18, and U-Net + ResNet34 were used for cell segmentation, producing segmentation accuracies of 94.36% versus 99.17% for the dataset of CellaVision and 93.20% versus 98.75% for the dataset of BCCD, confirming that deep learning produces higher performance than traditional machine learning in leukocyte classification. In addition, a series of deep-learning approaches, including AlexNet, VGG16, and ResNet18, was adopted to conduct feature extraction and cell classification of leukocytes, producing classification accuracies of 91.31%, 97.83%, and 100% of CellaVision as well as 81.18%, 91.64% and 97.82% of BCCD, confirming the capability of the increased deepness of neural networks in leukocyte classification. As to the demonstrations, this study further conducted cell-type classification of ALL-IDB2 and PCB-HBC datasets, producing high accuracies of 100% and 98.49% among all literature, validating the deep learning model used in this study.

Software Tools for 2D Cell Segmentation

Cell segmentation is an important task in the field of image processing, widely used in the life sciences and medical fields. Traditional methods are mainly based on pixel intensity and spatial relationships, but have limitations. In recent years, machine learning and deep learning methods have been widely used, providing more-accurate and efficient solutions for cell segmentation. The effort to develop efficient and accurate segmentation software tools has been one of the major focal points in the field of cell segmentation for years. However, each software tool has unique characteristics and adaptations, and no universal cell-segmentation software can achieve perfect results. In this review, we used three publicly available datasets containing multiple 2D cell-imaging modalities. Common segmentation metrics were used to evaluate the performance of eight segmentation tools to compare their generality and, thus, find the best-performing tool.

MyoV: a deep learning-based tool for the automated quantification of muscle fibers

Accurate approaches for quantifying muscle fibers are essential in biomedical research and meat production. In this study, we address the limitations of existing approaches for hematoxylin and eosin-stained muscle fibers by manually and semiautomatically labeling over 660 000 muscle fibers to create a large dataset. Subsequently, an automated image segmentation and quantification tool named MyoV is designed using mask regions with convolutional neural networks and a residual network and feature pyramid network as the backbone network. This design enables the tool to allow muscle fiber processing with different sizes and ages. MyoV, which achieves impressive detection rates of 0.93-0.96 and precision levels of 0.91-0.97, exhibits a superior performance in quantification, surpassing both manual methods and commonly employed algorithms and software, particularly for whole slide images (WSIs). Moreover, MyoV is proven as a powerful and suitable tool for various species with different muscle development, including mice, which are a crucial model for muscle disease diagnosis, and agricultural animals, which are a significant meat source for humans. Finally, we integrate this tool into visualization software with functions, such as segmentation, area determination and automatic labeling, allowing seamless processing for over 400 000 muscle fibers within a WSI, eliminating the model adjustment and providing researchers with an easy-to-use visual interface to browse functional options and realize muscle fiber quantification from WSIs.

Three-dimensional topology-based analysis segments volumetric and spatiotemporal fluorescence microscopy

Image analysis techniques provide objective and reproducible statistics for interpreting microscopy data. At higher dimensions, three-dimensional (3D) volumetric and spatiotemporal data highlight additional properties and behaviors beyond the static 2D focal plane. However, increased dimensionality carries increased complexity, and existing techniques for general segmentation of 3D data are either primitive, or highly specialized to specific biological structures. Borrowing from the principles of 2D topological data analysis (TDA), we formulate a 3D segmentation algorithm that implements persistent homology to identify variations in image intensity. From this, we derive two separate variants applicable to spatial and spatiotemporal data, respectively. We demonstrate that this analysis yields both sensitive and specific results on simulated data and can distinguish prominent biological structures in fluorescence microscopy images, regardless of their shape. Furthermore, we highlight the efficacy of temporal TDA in tracking cell lineage and the frequency of cell and organelle replication.

Optimizing Deep Learning-Based Segmentation of Densely Packed Cells using Cell Surface Markers

Background

Spatial molecular profiling depends on accurate cell segmentation. Identification and quantitation of individual cells in dense tissues, e.g. highly inflamed tissue caused by viral infection or immune reaction, remains a challenge.

Methods

We first assess the performance of 18 deep learning-based cell segmentation models, either pre-trained or trained by us using two public image sets, on a set of immunofluorescence images stained with immune cell surface markers in skin tissue obtained during human herpes simplex virus (HSV) infection. We then further train eight of these models using up to 10,000+ training instances from the current image set. Finally, we seek to improve performance by tuning parameters of the most successful method from the previous step.

Results

The best model before fine-tuning achieves a mean Average Precision (mAP) of 0.516. Prediction performance improves substantially after training. The best model is the cyto model from Cellpose. After training, it achieves an mAP of 0.694; with further parameter tuning, the mAP reaches 0.711.

Conclusion

Selecting the best model among the existing approaches and further training the model with images of interest produce the most gain in prediction performance. The performance of the resulting model compares favorably to human performance. The imperfection of the final model performance can be attributed to the moderate signal-to-noise ratio i the imageset.

Bering: joint cell segmentation and annotation for spatial transcriptomics with transferred graph embeddings

Single-cell spatial transcriptomics such as in-situ hybridization or sequencing technologies can provide subcellular resolution that enables the identification of individual cell identities, locations, and a deep understanding of subcellular mechanisms. However, accurate segmentation and annotation that allows individual cell boundaries to be determined remains a major challenge that limits all the above and downstream insights. Current machine learning methods heavily rely on nuclei or cell body staining, resulting in the significant loss of both transcriptome depth and the limited ability to learn latent representations of spatial colocalization relationships. Here, we propose Bering, a graph deep learning model that leverages transcript colocalization relationships for joint noise-aware cell segmentation and molecular annotation in 2D and 3D spatial transcriptomics data. Graph embeddings for the cell annotation are transferred as a component of multi-modal input for cell segmentation, which is employed to enrich gene relationships throughout the process. To evaluate performance, we benchmarked Bering with state-of-the-art methods and observed significant improvement in cell segmentation accuracies and numbers of detected transcripts across various spatial technologies and tissues. To streamline segmentation processes, we constructed expansive pre-trained models, which yield high segmentation accuracy in new data through transfer learning and self-distillation, demonstrating the generalizability of Bering.

CellSNAP: A fast, accurate algorithm for 3D cell segmentation in quantitative phase imaging

Quantitative phase imaging (QPI) has rapidly emerged as a complementary tool to fluorescence imaging, as it provides an objective measure of cell morphology and dynamics, free of variability due to contrast agents. In particular, three-dimensional (3D) tomographic imaging of live cells has opened up new directions of investigation by providing systematic and correlative analysis of various cellular parameters without limitations of photobleaching and phototoxicity. While current QPI systems allow the rapid acquisition of tomographic images, the pipeline to analyze these raw 3D tomograms is not well-developed. This work focuses on a critical, yet often underappreciated, step of the analysis pipeline, that of 3D cell segmentation from the acquired tomograms. The current method employed for such tasks is the Otsu-based 3D watershed algorithm, which works well for isolated cells; however, it is very challenging to draw boundaries when the cells are clumped. This process is also memory intensive since the processing requires computation on a 3D stack of images. We report the CellSNAP (Cell Segmentation via Novel Algorithm for Phase Imaging) algorithm for the segmentation of QPI images, which outstrips the current gold standard in terms of speed, robustness, and implementation, achieving cell segmentation under 2 seconds per cell on a single-core processor. The implementation of CellSNAP can easily be parallelized on a multi-core system for further speed improvements. For the cases where segmentation is possible with the existing standard method, our algorithm displays an average difference of 5% for dry mass and 8% for volume measurements. We also show that CellSNAP can handle challenging image datasets where cells are clumped and marred by interferogram drifts, which pose major difficulties for all QPI-focused segmentation tools. We envision our work will lead to the broader adoption of QPI imaging for high-throughput analysis, which has, in part, been stymied by a lack of suitable image segmentation tools.

Cell segmentation in images without structural fluorescent labels

High-content screening (HCS) provides an excellent tool to understand the mechanism of action of drugs on disease-relevant model systems. Careful selection of fluorescent labels (FLs) is crucial for successful HCS assay development. HCS assays typically comprise (a) FLs containing biological information of interest, and (b) additional structural FLs enabling instance segmentation for downstream analysis. However, the limited number of available fluorescence microscopy imaging channels restricts the degree to which these FLs can be experimentally multiplexed. In this article, we present a segmentation workflow that overcomes the dependency on structural FLs for image segmentation, typically freeing two fluorescence microscopy channels for biologically relevant FLs. It consists in extracting structural information encoded within readouts that are primarily biological, by fine-tuning pre-trained state-of-the-art generalist cell segmentation models for different combinations of individual FLs, and aggregating the respective segmentation results together. Using annotated datasets that we provide, we confirm our methodology offers improvements in performance and robustness across several segmentation aggregation strategies and image acquisition methods, over different cell lines and various FLs. It thus enables the biological information content of HCS assays to be maximized without compromising the robustness and accuracy of computational single-cell profiling.

DeepSea is an efficient deep-learning model for single-cell segmentation and tracking in time-lapse microscopy

Time-lapse microscopy is the only method that can directly capture the dynamics and heterogeneity of fundamental cellular processes at the single-cell level with high temporal resolution. Successful application of single-cell time-lapse microscopy requires automated segmentation and tracking of hundreds of individual cells over several time points. However, segmentation and tracking of single cells remain challenging for the analysis of time-lapse microscopy images, in particular for widely available and non-toxic imaging modalities such as phase-contrast imaging. This work presents a versatile and trainable deep-learning model, termed DeepSea, that allows for both segmentation and tracking of single cells in sequences of phase-contrast live microscopy images with higher precision than existing models. We showcase the application of DeepSea by analyzing cell size regulation in embryonic stem cells.

OMIP-088: Twenty-target imaging mass cytometry panel for major cell populations in mouse formalin fixed paraffin embedded liver

The purpose of this 20-target imaging mass cytometry (IMC) panel is to identify the main cell types in formalin fixed paraffin embedded (FFPE) mouse liver tissue with the Hyperion™ mass cytometer from Standard BioTools (formerly Fluidigm). The antibody panel includes markers to identify hepatocytes (E-cadherin, HNF4α (hepatocyte nuclear factor 4 alpha), Arginase-1), liver sinusoidal endothelial cells (LSECs; CD206), Kupffer cells (F4/80, CD206), neutrophils (Ly6G, CD11b), bone marrow derived myeloid cells (BMDMs; CD11b), cholangiocytes (E-cadherin high), endothelial cells (CD31, α-SMA), plasmacytoid dendritic cells (CD317), B cells (CD19), T cells (CD3e, CD4, CD8a), NK cells (CD161) as well markers of cell activation (CD44, CD74), proliferation (Ki-67) and to aid in cell segmentation (Pan-Actin, E-cadherin, histone H3). The panel has been tested in other mouse tissues, namely the spleen, colon and lung, and therefore is likely to work across various mouse FFPE samples of interest. It has not been tested using human samples, frozen samples or in suspension mass cytometry because FFPE treatment profoundly changes epitope conformation. In summary, this panel is a powerful tool for pre-clinical research to determine cellular abundance and spatial distribution within mouse tissues and serves as a scaffold, to which more targets can be added for project specific requirements.

Label-free cell segmentation of diverse lymphoid tissues in 2D and 3D

Unlocking and quantifying fundamental biological processes through tissue microscopy requires accurate, in situ segmentation of all cells imaged. Currently, achieving this is complex and requires exogenous fluorescent labels that occupy significant spectral bandwidth, increasing the duration and complexity of imaging experiments while limiting the number of channels remaining to address the study's objectives. We demonstrate that the excitation light reflected during routine confocal microscopy contains sufficient information to achieve accurate, label-free cell segmentation in 2D and 3D. This is achieved using a simple convolutional neural network trained to predict the probability that reflected light pixels belong to either nucleus, cytoskeleton, or background classifications. We demonstrate the approach across diverse lymphoid tissues and provide video tutorials demonstrating deployment in Python and MATLAB or via standalone software for Windows.

Automated cell segmentation for reproducibility in bioimage analysis

Live-cell imaging is extremely common in synthetic biology research, but its ability to be applied reproducibly across laboratories can be hindered by a lack of standardized image analysis. Here, we introduce a novel cell segmentation method developed as part of a broader Independent Verification & Validation (IV&V) program aimed at characterizing engineered Dictyostelium cells. Standardizing image analysis was found to be highly challenging: the amount of human judgment required for parameter optimization, algorithm tweaking, training and data pre-processing steps forms serious challenges for reproducibility. To bring automation and help remove bias from live-cell image analysis, we developed a self-supervised learning (SSL) method that recursively trains itself directly from motion in live-cell microscopy images without any end-user input, thus providing objective cell segmentation. Here, we highlight this SSL method applied to characterizing the engineered Dictyostelium cells of the original IV&V program. This approach is highly generalizable, accepting images from any cell type or optical modality without the need for manual training or parameter optimization. This method represents an important step toward automated bioimage analysis software and reflects broader efforts to design accessible measurement technologies to enhance reproducibility in synthetic biology research.

YOUPI: Your powerful and intelligent tool for segmenting cells from imaging mass cytometry data

The recent emergence of imaging mass cytometry technology has led to the generation of an increasing amount of high-dimensional data and, with it, the need for suitable performant bioinformatics tools dedicated to specific multiparametric studies. The first and most important step in treating the acquired images is the ability to perform highly efficient cell segmentation for subsequent analyses. In this context, we developed YOUPI (Your Powerful and Intelligent tool) software. It combines advanced segmentation techniques based on deep learning algorithms with a friendly graphical user interface for non-bioinformatics users. In this article, we present the segmentation algorithm developed for YOUPI. We have set a benchmark with mathematics-based segmentation approaches to estimate its robustness in segmenting different tissue biopsies.

Implementing High Dimensional Reduction Analysis on Histocytometric Data

In a previous protocol article, we demonstrated construction of a histocytometry pipeline that is capable of both segmenting highly aggregated cell populations and retaining the original intensity data range of the input microscopy images. In the protocol presented here, using the output from the aforementioned article, we demonstrate how to phenotype the data using the high dimensional reduction analysis technique optimized t-distributed stochastic neighbor embedding (opt-t-SNE) and compare it to traditional manual gating. Additionally, we present a protocol illustrating the advantage of the inclusion of cell junction/membrane markers for accurately segmenting highly aggregated cell populations in ilastik. © 2022 Wiley Periodicals LLC. Basic Protocol 1: Phenotyping lymph node populations using manual gating Basic Protocol 2: Phenotyping lymph node populations using t-SNE dimensional reduction Support Protocol: ilastik segmentation using a pan marker.

Simultaneous and Spatially-Resolved Analysis of T-Lymphocytes, Macrophages and PD-L1 Immune Checkpoint in Rare Cancers

Simple Summary

To study various biomarkers, it is necessary to analyze multiple tissue sections through serial histological sections, which is challenging when only a small tissue sample is available. In this work we have developed a validated and objective method for combined biomarker immunostaining and its digital image analysis using open informatics tools, which is necessary for comprehensive understanding of the tumor microenvironment in rare cancers and in cases of limited samples with very significant clinical features.

Abstract

Penile, vulvar and anal neoplasms show an incidence lower than 0.5% of the population per year and therefore can be considered as rare cancers but with a dramatic impact on quality of life and survival. This work describes the experience of a Chilean cancer center using multiplexed immunofluorescence to study a case series of four penile cancers, two anal cancers and one vulvar cancer and simultaneous detection of CD8, CD68, PD-L1, Cytokeratin and Ki-67 in FFPE samples. Fluorescent image analyses were performed using open sources for automated tissue segmentation and cell phenotyping. Our results showed an objective and reliable counting of objects with a single or combined labeling or within a specific tissue compartment. The variability was below 10%, and the correlation between analytical events was 0.92–0.97. Critical cell phenotypes, such as TILs, PD-L1+ or proliferative tumor cells were detected in a supervised and unsupervised manner with a limit of detection of less than 1% of relative abundance. Finally, the observed diversity and abundance of the different cell phenotypes within the tumor microenvironment for the three studied tumor types confirmed that our methodology is useful and robust to be applicable for many other solid tumors.

Creation of High-Dimensional Reduction Analysis-Compatible Histocytometry Files from Images of Densely-Packed Cells and/or Variable Stain Intensity

The power of high-dimensional reduction techniques using multiparameter images has been demonstrated across a variety of different publications. Recently, we published an end-to-end low-cost GUI-based protocol for performing histocytometric spatial analysis on images derived from the most common microscope image formats. However, this protocol is limited by the normalized marker intensity outputs and the difficulty in processing images of highly aggregated and/or exceptionally heterogenous cell populations. Here we present the basic protocols required to construct an advanced histocytometric data file using only freeware. This data file is compatible with images containing cell nuclei clusters that are difficult to segment, and results in histocytometry files retaining the original marker intensity values of the microscopic images they were derived from. This is especially useful in cells that are phenotyped based on relative marker expression levels. Histocytometry data files produced by these protocols are compatible with high-dimensional reduction analysis using marker intensity data, such as tSNEs. This methodology is showcased using stitched microscopic images of murine lymph nodes, complex organs with highly aggregated heterogenous cell populations, that are typically difficult to segment. © 2022 Wiley Periodicals LLC. Basic Protocol 1: Image preprocessing and generation of nuclei marker probability maps Basic Protocol 2: Cell segmentation using ilastik-derived probability maps Basic Protocol 3: Generation of histocytometric .fcs files.

How to Build an Image-Processing Pipeline for Automating Multiparameter Histocytometry Analysis

Until relatively recently, analysis of imaging data has been primarily quantitative and limited to 3-4 markers. The advancement of various technologies overcoming this marker limitation provided the capability of analyzing multiparameter imaging data down to the single cell level, termed histocytometry. Currently, most published end-to-end histocytometric analysis of imaging data is performed using expensive commercial programs or freely available analysis packages that require significant knowledge of programming languages for execution. Here we present a protocol that performs cell segmentation, phenotyping and spatial analysis, using software with easy-to-use GUIs (graphical user interfaces). These protocols allow the user to derive spatial and phenotypical data for the analysis of multiparameter microscopic images from most imaging platforms in a low-cost manner. © 2022 Wiley Periodicals LLC. Basic Protocol 1: Cell Segmentation and generation of histocytometric .csv file Basic Protocol 2: Phenotyping of cell populations Basic Protocol 3: Spatial relationship analyses of phenotyped populations Support Protocol 1: Nuclei Segmentation Accuracy Test Support Protocol 2: Correcting y-axis Inversion of Histocytometry Data Relative to Original Image File.

Graph-Theoretic Post-Processing of Segmentation With Application to Dense Biofilms

Recent deep learning methods have provided successful initial segmentation results for generalized cell segmentation in microscopy. However, for dense arrangements of small cells with limited ground truth for training, the deep learning methods produce both over-segmentation and under-segmentation errors. Post-processing attempts to balance the trade-off between the global goal of cell counting for instance segmentation, and local fidelity to the morphology of identified cells. The need for post-processing is especially evident for segmenting 3D bacterial cells in densely-packed communities called biofilms. A graph-based recursive clustering approach, m-LCuts, is proposed to automatically detect collinearly structured clusters and applied to post-process unsolved cells in 3D bacterial biofilm segmentation. Construction of outlier-removed graphs to extract the collinearity feature in the data adds additional novelty to m-LCuts. The superiority of m-LCuts is observed by the evaluation in cell counting with over 90% of cells correctly identified, while a lower bound of 0.8 in terms of average single-cell segmentation accuracy is maintained. This proposed method does not need manual specification of the number of cells to be segmented. Furthermore, the broad adaptation for working on various applications, with the presence of data collinearity, also makes m-LCuts stand out from the other approaches.

Active Cell Appearance Model Induced Generative Adversarial Networks for Annotation-Efficient Cell Segmentation and Identification on Adaptive Optics Retinal Images

Data annotation is a fundamental precursor for establishing large training sets to effectively apply deep learning methods to medical image analysis. For cell segmentation, obtaining high quality annotations is an expensive process that usually requires manual grading by experts. This work introduces an approach to efficiently generate annotated images, called "A-GANs", created by combining an active cell appearance model (ACAM) with conditional generative adversarial networks (C-GANs). ACAM is a statistical model that captures a realistic range of cell characteristics and is used to ensure that the image statistics of generated cells are guided by real data. C-GANs utilize cell contours generated by ACAM to produce cells that match input contours. By pairing ACAM-generated contours with A-GANs-based generated images, high quality annotated images can be efficiently generated. Experimental results on adaptive optics (AO) retinal images showed that A-GANs robustly synthesize realistic, artificial images whose cell distributions are exquisitely specified by ACAM. The cell segmentation performance using as few as 64 manually-annotated real AO images combined with 248 artificially-generated images from A-GANs was similar to the case of using 248 manually-annotated real images alone (Dice coefficients of 88% for both). Finally, application to rare diseases in which images exhibit never-seen characteristics demonstrated improvements in cell segmentation without the need for incorporating manual annotations from these new retinal images. Overall, A-GANs introduce a methodology for generating high quality annotated data that statistically captures the characteristics of any desired dataset and can be used to more efficiently train deep-learning-based medical image analysis applications.

High-Dimensional Image Analysis using Histocytometry

Histocytometry is a technique for processing multiparameter microscopy images using computational approaches to identify and quantify cellular phenotypes. It allows for spatial analyses of cellular phenotypes in relation to each other and within defined spatial regions. The benefit of this technique over manual annotation and characterization of cells is a high degree of automation/throughput, significantly decreased user bias, and increased reproducibility. Recently, an increase in freely available software amenable to or deliberately designed for histocytometry has resulted in these complex analyses being available to a broader base of users who have amassed multi-component microscopic imaging data. This article provides an overview of a histocytometry pipeline, focusing on the strategic planning and software requirements to allow readers to perform cell segmentation, phenotyping, and spatial analyses to advance their research outputs. © 2021 Wiley Periodicals LLC.

Efficient Cell Segmentation from Electroluminescent Images of Single-Crystalline Silicon Photovoltaic Modules and Cell-Based Defect Identification Using Deep Learning with Pseudo-Colorization

Solar cells may possess defects during the manufacturing process in photovoltaic (PV) industries. To precisely evaluate the effectiveness of solar PV modules, manufacturing defects are required to be identified. Conventional defect inspection in industries mainly depends on manual defect inspection by highly skilled inspectors, which may still give inconsistent, subjective identification results. In order to automatize the visual defect inspection process, an automatic cell segmentation technique and a convolutional neural network (CNN)-based defect detection system with pseudo-colorization of defects is designed in this paper. High-resolution Electroluminescence (EL) images of single-crystalline silicon (sc-Si) solar PV modules are used in our study for the detection of defects and their quality inspection. Firstly, an automatic cell segmentation methodology is developed to extract cells from an EL image. Secondly, defect detection can be actualized by CNN-based defect detector and can be visualized with pseudo-colors. We used contour tracing to accurately localize the panel region and a probabilistic Hough transform to identify gridlines and busbars on the extracted panel region for cell segmentation. A cell-based defect identification system was developed using state-of-the-art deep learning in CNNs. The detected defects are imposed with pseudo-colors for enhancing defect visualization using K-means clustering. Our automatic cell segmentation methodology can segment cells from an EL image in about 2.71 s. The average segmentation errors along the x-direction and y-direction are only 1.6 pixels and 1.4 pixels, respectively. The defect detection approach on segmented cells achieves 99.8% accuracy. Along with defect detection, the defect regions on a cell are furnished with pseudo-colors to enhance the visualization.

Machine Learning Algorithms, Applied to Intact Islets of Langerhans, Demonstrate Significantly Enhanced Insulin Staining at the Capillary Interface of Human Pancreatic β Cells

Pancreatic β cells secrete the hormone insulin into the bloodstream and are critical in the control of blood glucose concentrations. β cells are clustered in the micro-organs of the islets of Langerhans, which have a rich capillary network. Recent work has highlighted the intimate spatial connections between β cells and these capillaries, which lead to the targeting of insulin secretion to the region where the β cells contact the capillary basement membrane. In addition, β cells orientate with respect to the capillary contact point and many proteins are differentially distributed at the capillary interface compared with the rest of the cell. Here, we set out to develop an automated image analysis approach to identify individual β cells within intact islets and to determine if the distribution of insulin across the cells was polarised. Our results show that a U-Net machine learning algorithm correctly identified β cells and their orientation with respect to the capillaries. Using this information, we then quantified insulin distribution across the β cells to show enrichment at the capillary interface. We conclude that machine learning is a useful analytical tool to interrogate large image datasets and analyse sub-cellular organisation.

Deep neural net tracking of human pluripotent stem cells reveals intrinsic behaviors directing morphogenesis

Lineage tracing is a powerful tool in developmental biology to interrogate the evolution of tissue formation, but the dense, three-dimensional nature of tissue limits the assembly of individual cell trajectories into complete reconstructions of development. Human induced pluripotent stem cells (hiPSCs) can recapitulate aspects of developmental processes, providing an in vitro platform to assess the dynamic collective behaviors directing tissue morphogenesis. Here, we trained an ensemble of neural networks to track individual hiPSCs in time-lapse microscopy, generating longitudinal measures of cell and cellular neighborhood properties on timescales from minutes to days. Our analysis reveals that, while individual cell parameters are not strongly affected by pluripotency maintenance conditions or morphogenic cues, regional changes in cell behavior predict cell fate and colony organization. By generating complete multicellular reconstructions of hiPSC behavior, our tracking pipeline enables fine-grained understanding of morphogenesis by elucidating the role of regional behavior in early tissue formation.

Label-Free Deep Profiling of the Tumor Microenvironment

Label-free nonlinear microscopy enables nonperturbative visualization of structural and metabolic contrast within living cells in their native tissue microenvironment. Here a computational pipeline was developed to provide a quantitative view of the microenvironmental architecture within cancerous tissue from label-free nonlinear microscopy images. To enable single-cell and single-extracellular vesicle (EV) analysis, individual cells, including tumor cells and various types of stromal cells, and EVs were segmented by a multiclass pixelwise segmentation neural network and subsequently analyzed for their metabolic status and molecular structure in the context of the local cellular neighborhood. By comparing cancer tissue with normal tissue, extensive tissue reorganization and formation of a patterned cell-EV neighborhood was observed in the tumor microenvironment. The proposed analytic pipeline is expected to be useful in a wide range of biomedical tasks that benefit from single-cell, single-EV, and cell-to-EV analysis. SIGNIFICANCE: The proposed computational framework allows label-free microscopic analysis that quantifies the complexity and heterogeneity of the tumor microenvironment and opens possibilities for better characterization and utilization of the evolving cancer landscape.

Nuclei Detection for 3D Microscopy With a Fully Convolutional Regression Network

Advances in three-dimensional microscopy and tissue clearing are enabling whole-organ imaging with single-cell resolution. Fast and reliable image processing tools are needed to analyze the resulting image volumes, including automated cell detection, cell counting and cell analytics. Deep learning approaches have shown promising results in two- and three-dimensional nuclei detection tasks, however detecting overlapping or non-spherical nuclei of different sizes and shapes in the presence of a blurring point spread function remains challenging and often leads to incorrect nuclei merging and splitting. Here we present a new regression-based fully convolutional network that located a thousand nuclei centroids with high accuracy in under a minute when combined with V-net, a popular three-dimensional semantic-segmentation architecture. High nuclei detection F1-scores of 95.3% and 92.5% were obtained in two different whole quail embryonic hearts, a tissue type difficult to segment because of its high cell density, and heterogeneous and elliptical nuclei. Similar high scores were obtained in the mouse brain stem, demonstrating that this approach is highly transferable to nuclei of different shapes and intensities. Finally, spatial statistics were performed on the resulting centroids. The spatial distribution of nuclei obtained by our approach most resembles the spatial distribution of manually identified nuclei, indicating that this approach could serve in future spatial analyses of cell organization.

Deep learning based digital cell profiles for risk stratification of urine cytology images

Urine cytology is a test for the detection of high-grade bladder cancer. In clinical practice, the pathologist would manually scan the sample under the microscope to locate atypical and malignant cells. They would assess the morphology of these cells to make a diagnosis. Accurate identification of atypical and malignant cells in urine cytology is a challenging task and is an essential part of identifying different diagnosis with low-risk and high-risk malignancy. Computer-assisted identification of malignancy in urine cytology can be complementary to the clinicians for treatment management and in providing advice for carrying out further tests. In this study, we presented a method for identifying atypical and malignant cells followed by their profiling to predict the risk of diagnosis automatically. For cell detection and classification, we employed two different deep learning-based approaches. Based on the best performing network predictions at the cell level, we identified low-risk and high-risk cases using the count of atypical cells and the total count of atypical and malignant cells. The area under the receiver operating characteristic (ROC) curve shows that a total count of atypical and malignant cells is comparably better at diagnosis as compared to the count of malignant cells only. We obtained area under the ROC curve with the count of malignant cells and the total count of atypical and malignant cells as 0.81 and 0.83, respectively. Our experiments also demonstrate that the digital risk could be a better predictor of the final histopathology-based diagnosis. We also analyzed the variability in annotations at both cell and whole slide image level and also explored the possible inherent rationales behind this variability.

Bioimage Analysis and Cell Motility

Bioimage analysis (BIA) has historically helped study how and why cells move; biological experiments evolved in intimate feedback with the most classical image processing techniques because they contribute objectivity and reproducibility to an eminently qualitative science. Cell segmentation, tracking, and morphology descriptors are all discussed here. Using ameboid motility as a case study, these methods help us illustrate how proper quantification can augment biological data, for example, by choosing mathematical representations that amplify initially subtle differences, by statistically uncovering general laws or by integrating physical insight. More recently, the non-invasive nature of quantitative imaging is fertilizing two blooming fields: mechanobiology, where many biophysical measurements remain inaccessible, and microenvironments, where the quest for physiological relevance has exploded data size. From relief to remedy, this trend indicates that BIA is to become a main vector of biological discovery as human visual analysis struggles against ever more complex data.

Improved efficiency of urine cell image segmentation using droplet microfluidics technology

Recent advances in the recognition of biological samples using machine vision have made this technology increasingly important in research and detection. Image segmentation is an important step in this process. This study focuses on how to reduce the interference factors such as the overlap between different types (or within the same type) of urine cells according to microfluidics and improve the machine vision segmentation accuracy for cell images. In this study, we demonstrate that the platform can realize this hypothesis using urine cell image segmentation as an example application. We first discuss the reported urine cell droplet microfluidic chip system, which can realize the test conditions in which urine cells are encapsulated in the droplet and isolated from salt crystallization and/or bacteria and other urine-formed elements. Then, based on the analysis conditions set in the aforementioned experiment, the proportions of red blood cells, white blood cells, and squamous epithelial cells covered by various formed elements in the total urine cells in the same urine sample are measured. We simultaneously analyze the percentage of urine cells covered by salt crystallization and the incidence of overlapping between urine cells. Finally, the Otsu algorithm is used to segment the urine cell images encapsulated by the droplet and the urine cell images not encapsulated by the droplet, and the Dice, Jaccard, precision, and recall values are calculated. The results suggest that the method of encapsulating single cells based on droplets can improve the image segmentation effect without optimizing the algorithm.

Automated Detection of Acute Lymphoblastic Leukemia From Microscopic Images Based on Human Visual Perception

Microscopic image analysis plays a significant role in initial leukemia screening and its efficient diagnostics. Since the present conventional methodologies partly rely on manual examination, which is time consuming and depends greatly on the experience of domain experts, automated leukemia detection opens up new possibilities to minimize human intervention and provide more accurate clinical information. This paper proposes a novel approach based on conventional digital image processing techniques and machine learning algorithms to automatically identify acute lymphoblastic leukemia from peripheral blood smear images. To overcome the greatest challenges in the segmentation phase, we implemented extensive pre-processing and introduced a three-phase filtration algorithm to achieve the best segmentation results. Moreover, sixteen robust features were extracted from the images in the way that hematological experts do, which significantly increased the capability of the classifiers to recognize leukemic cells in microscopic images. To perform the classification, we applied two traditional machine learning classifiers, the artificial neural network and the support vector machine. Both methods reached a specificity of 95.31%, and the sensitivity of the support vector machine and artificial neural network reached 98.25 and 100%, respectively.

Accurate and versatile 3D segmentation of plant tissues at cellular resolution

Quantitative analysis of plant and animal morphogenesis requires accurate segmentation of individual cells in volumetric images of growing organs. In the last years, deep learning has provided robust automated algorithms that approach human performance, with applications to bio-image analysis now starting to emerge. Here, we present PlantSeg, a pipeline for volumetric segmentation of plant tissues into cells. PlantSeg employs a convolutional neural network to predict cell boundaries and graph partitioning to segment cells based on the neural network predictions. PlantSeg was trained on fixed and live plant organs imaged with confocal and light sheet microscopes. PlantSeg delivers accurate results and generalizes well across different tissues, scales, acquisition settings even on non plant samples. We present results of PlantSeg applications in diverse developmental contexts. PlantSeg is free and open-source, with both a command line and a user-friendly graphical interface.

UNet++: Redesigning Skip Connections to Exploit Multiscale Features in Image Segmentation

The state-of-the-art models for medical image segmentation are variants of U-Net and fully convolutional networks (FCN). Despite their success, these models have two limitations: (1) their optimal depth is apriori unknown, requiring extensive architecture search or inefficient ensemble of models of varying depths; and (2) their skip connections impose an unnecessarily restrictive fusion scheme, forcing aggregation only at the same-scale feature maps of the encoder and decoder sub-networks. To overcome these two limitations, we propose UNet++, a new neural architecture for semantic and instance segmentation, by (1) alleviating the unknown network depth with an efficient ensemble of U-Nets of varying depths, which partially share an encoder and co-learn simultaneously using deep supervision; (2) redesigning skip connections to aggregate features of varying semantic scales at the decoder sub-networks, leading to a highly flexible feature fusion scheme; and (3) devising a pruning scheme to accelerate the inference speed of UNet++. We have evaluated UNet++ using six different medical image segmentation datasets, covering multiple imaging modalities such as computed tomography (CT), magnetic resonance imaging (MRI), and electron microscopy (EM), and demonstrating that (1) UNet++ consistently outperforms the baseline models for the task of semantic segmentation across different datasets and backbone architectures; (2) UNet++ enhances segmentation quality of varying-size objects-an improvement over the fixed-depth U-Net; (3) Mask RCNN++ (Mask R-CNN with UNet++ design) outperforms the original Mask R-CNN for the task of instance segmentation; and (4) pruned UNet++ models achieve significant speedup while showing only modest performance degradation. Our implementation and pre-trained models are available at https://github.com/MrGiovanni/UNetPlusPlus.

Image-Based Cell Profiling Enables Quantitative Tissue Microscopy in Gastroenterology

Immunofluorescence microscopy is an essential tool for tissue-based research, yet data reporting is almost always qualitative. Quantification of images, at the per-cell level, enables "flow cytometry-type" analyses with intact locational data but achieving this is complex. Gastrointestinal tissue, for example, is highly diverse: from mixed-cell epithelial layers through to discrete lymphoid patches. Moreover, different species (e.g., rat, mouse, and humans) and tissue preparations (paraffin/frozen) are all commonly studied. Here, using field-relevant examples, we develop open, user-friendly methodology that can encompass these variables to provide quantitative tissue microscopy for the field. Antibody-independent cell labeling approaches, compatible across preparation types and species, were optimized. Per-cell data were extracted from routine confocal micrographs, with semantic machine learning employed to tackle densely packed lymphoid tissues. Data analysis was achieved by flow cytometry-type analyses alongside visualization and statistical definition of cell locations, interactions and established microenvironments. First, quantification of Escherichia coli passage into human small bowel tissue, following Ussing chamber incubations exemplified objective quantification of rare events in the context of lumen-tissue crosstalk. Second, in rat jejenum, precise histological context revealed distinct populations of intraepithelial lymphocytes between and directly below enterocytes enabling quantification in context of total epithelial cell numbers. Finally, mouse mononuclear phagocyte-T cell interactions, cell expression and significant spatial cell congregations were mapped to shed light on cell-cell communication in lymphoid Peyer's patch. Accessible, quantitative tissue microscopy provides a new window-of-insight to diverse questions in gastroenterology. It can also help combat some of the data reproducibility crisis associated with antibody technologies and over-reliance on qualitative microscopy. © 2020 The Authors. Cytometry Part A published by Wiley Periodicals LLC. on behalf of International Society for Advancement of Cytometry.

Comparative Study of Contact Repulsion in Control and Mutant Macrophages Using a Novel Interaction Detection

In this paper, a novel method for interaction detection is presented to compare the contact dynamics of macrophages in the Drosophila embryo. The study is carried out by a framework called macrosight, which analyses the movement and interaction of migrating macrophages. The framework incorporates a segmentation and tracking algorithm into analysing the motion characteristics of cells after contact. In this particular study, the interactions between cells is characterised in the case of control embryos and Shot mutants, a candidate protein that is hypothesised to regulate contact dynamics between migrating cells. Statistical significance between control and mutant cells was found when comparing the direction of motion after contact in specific conditions. Such discoveries provide insights for future developments in combining biological experiments with computational analysis.

LeukocyteMask: An automated localization and segmentation method for leukocyte in blood smear images using deep neural networks

Digital pathology and microscope image analysis is widely used in comprehensive studies of cell morphology. Identification and analysis of leukocytes in blood smear images, acquired from bright field microscope, are vital for diagnosing many diseases such as hepatitis, leukaemia and acquired immune deficiency syndrome (AIDS). The major challenge for robust and accurate identification and segmentation of leukocyte in blood smear images lays in the large variations of cell appearance such as size, colour and shape of cells, the adhesion between leukocytes (white blood cells, WBCs) and erythrocytes (red blood cells, RBCs), and the emergence of substantial dyeing impurities in blood smear images. In this paper, an end-to-end leukocyte localization and segmentation method is proposed, named LeukocyteMask, in which pixel-level prior information is utilized for supervisor training of a deep convolutional neural network, which is then employed to locate the region of interests (ROI) of leukocyte, and finally segmentation mask of leukocyte is obtained based on the extracted ROI by forward propagation of the network. Experimental results validate the effectiveness of the propose method and both the quantitative and qualitative comparisons with existing methods indicate that LeukocyteMask achieves a state-of-the-art performance for the segmentation of leukocyte in terms of robustness and accuracy .

Split and Merge Watershed: a two-step method for cell segmentation in fluorescence microscopy images

The development of advanced techniques in medical imaging has allowed scanning of the human body to microscopic levels, making research on cell behavior more complex and more in-depth. Recent studies have focused on cellular heterogeneity since cell-to-cell differences are always present in the cell population and this variability contains valuable information. However, identifying each cell is not an easy task because, in the images acquired from the microscope, there are clusters of cells that are touching one another. Therefore, the segmentation stage is a problem of considerable difficulty in cell image processing. Although several methods for cell segmentation are described in the literature, they have drawbacks in terms of over-segmentation, under-segmentation or misidentification. Consequently, our main motivation in studying cell segmentation was to develop a new method to achieve a good tradeoff between accurately identifying all relevant elements and not inserting segmentation artifacts. This article presents a new method for cell segmentation in fluorescence microscopy images. The proposed approach combines the well-known Marker-Controlled Watershed algorithm (MC-Watershed) with a new, two-step method based on Watershed, Split and Merge Watershed (SM-Watershed): in the first step, or split phase, the algorithm identifies the clusters using inherent characteristics of the cell, such as size and convexity, and separates them using watershed. In the second step, or the merge stage, it identifies the over-segmented regions using proper features of the cells and eliminates the divisions. Before applying our two-step method, the input image is first preprocessed, and the MC-Watershed algorithm is used to generate an initial segmented image. However, this initial result may not be suitable for subsequent tasks, such as cell count or feature extraction, because not all cells are separated, and some cells may be mistakenly confused with the background. Thus, our proposal corrects this issue with its two-step process, reaching a high performance, a suitable tradeoff between over-segmentation and under-segmentation and preserving the shape of the cell, without the need of any labeled data or relying on machine learning processes. The latter is advantageous over state-of-the-art techniques that in order to achieve similar results require labeled data, which may not be available for all of the domains. Two cell datasets were used to validate this approach, and the results were compared with other methods in the literature, using traditional metrics and quality visual assessment. We obtained 90% of average visual accuracy and an F-index higher than 80%. This proposal outperforms other techniques for cell separation, achieving an acceptable balance between over-segmentation and under-segmentation, which makes it suitable for several applications in cell identification, such as virus infection analysis, high-content cell screening, drug discovery, and morphometry.

Separating Touching Cells Using Pixel Replicated Elliptical Shape Models

One of the most important and error-prone tasks in biological image analysis is the segmentation of touching or overlapping cells. Particularly for optical microscopy, including transmitted light and confocal fluorescence microscopy, there is often no consistent discriminative information to separate cells that touch or overlap. It is desired to partition touching foreground pixels into cells using the binary threshold image information only, and optionally incorporating gradient information. The most common approaches for segmenting touching and overlapping cells in these scenarios are based on the watershed transform. We describe a new approach called pixel replication for the task of segmenting elliptical objects that touch or overlap. Pixel replication uses the image Euclidean distance transform in combination with Gaussian mixture models to better exploit practically effective optimization for delineating objects with elliptical decision boundaries. Pixel replication improves significantly on commonly used methods based on watershed transforms, or based on fitting Gaussian mixtures directly to the thresholded image data. Pixel replication works equivalently on both 2-D and 3-D image data, and naturally combines information from multi-channel images. The accuracy of the proposed technique is measured using both the segmentation accuracy on simulated ellipse data and the tracking accuracy on validated stem cell tracking results extracted from hundreds of live-cell microscopy image sequences. Pixel replication is shown to be significantly more accurate compared with other approaches. Variance relationships are derived, allowing a more practically effective Gaussian mixture model to extract cell boundaries for data generated from the threshold image using the uniform elliptical distribution and from the distance transform image using the triangular elliptical distribution.

Robust Cell Detection and Segmentation for Image Cytometry Reveal Th17 Cell Heterogeneity

Image cytometry enables quantitative cell characterization with preserved tissue architecture; thus, it has been highlighted in the advancement of multiplex immunohistochemistry (IHC) and digital image analysis in the context of immune-based biomarker monitoring associated with cancer immunotherapy. However, one of the challenges in the current image cytometry methodology is a technical limitation in the segmentation of nuclei and cellular components particularly in heterogeneously stained cancer tissue images. To improve the detection and specificity of single-cell segmentation in hematoxylin-stained images (which can be utilized for recently reported 12-biomarker chromogenic sequential multiplex IHC), we adapted a segmentation algorithm previously developed for hematoxlin and eosin-stained images, where morphological features are extracted based on Gabor-filtering, followed by stacking of image pixels into n-dimensional feature space and unsupervised clustering of individual pixels. Our proposed method showed improved sensitivity and specificity in comparison with standard segmentation methods. Replacing previously proposed methods with our method in multiplex IHC/image cytometry analysis, we observed higher detection of cell lineages including relatively rare T_H 17 cells, further enabling sub-population analysis into T_H 1-like and T_H 2-like phenotypes based on T-bet and GATA3 expression. Interestingly, predominance of T_H 2-like T_H 17 cells was associated with human papilloma virus (HPV)-negative status of oropharyngeal squamous cell carcinoma of head and neck, known as a poor-prognostic subtype in comparison with HPV-positive status. Furthermore, T_H 2-like T_H 17 cells in HPV-negative head and neck cancer tissues were spatiotemporally correlated with CD66b⁺ granulocytes, presumably associated with an immunosuppressive microenvironment. Our cell segmentation method for multiplex IHC/image cytometry potentially contributes to in-depth immune profiling and spatial association, leading to further tissue-based biomarker exploration. © 2019 International Society for Advancement of Cytometry.

Quality assessment of Ki67 staining using cell line proliferation index and stain intensity features

Breast cancer is the most frequent cancer among women worldwide. Ki67 can be used as an immunohistochemical pseudo marker for cell proliferation to determine how aggressive the cancer is and thereby the treatment of the patient. No standard Ki67 staining protocol exists, resulting in inter-laboratory stain variability. Therefore, it is important to determine the quality control of a staining protocol to ensure correct diagnosis and treatment of patients. Currently, quality control is performed by the organization NordiQC that use an expert panel-based qualitative assessment system. However, no objective method exists to determine the quality of a staining protocol. In this study, we propose an algorithm, to objectively assess staining quality from segmented cell nuclei structures extracted from cell lines. The cell nuclei were classified into either Ki67 positive or negative to determine the Ki67 proliferation index within the cell lines. A Ki67 stain quality model based on ordinal logistic regression was developed to determine the quality of a staining protocol from features extracted from the segmented cell nuclei in the cell lines. The algorithm was able to segment and classify Ki67 positive cell nuclei with a sensitivity and positive predictive value (PPV) of 0.90 and 0.94 and Ki67 negative cell nuclei with a sensitivity and PPV of 0.78 and 0.78. The mean difference between a manual and automatic Ki67 proliferation index was -0.003 with a standard deviation of 0.056. The ordinal logistic regression model found that the stain intensity for both the Ki67 positive and Ki67 negative cell nuclei were statistically significant as parameters determining the stain quality from the cell line cores. The framework shows great promise for using cell nuclei information from cell lines to predict the staining quality of staining protocols. © 2018 International Society for Advancement of Cytometry.

Quantitative analysis of cell shape and the cytoskeleton in developmental biology

Computational approaches that enable quantification of microscopy data have revolutionized the field of developmental biology. Due to its inherent complexity, elucidating mechanisms of development requires sophisticated analysis of the structure, shape, and kinetics of cellular processes. This need has prompted the creation of numerous techniques to visualize, quantify, and merge microscopy data. These approaches have defined the order and structure of developmental events, thus, providing insight into the mechanisms that drive them. This review describes current computational approaches that are being used to answer developmental questions related to morphogenesis and describe how these approaches have impacted the field. Our intent is not to comprehensively review techniques, but to highlight examples of how different approaches have impacted our understanding of development. Specifically, we focus on methods to quantify cell shape and cytoskeleton structure and dynamics in developing tissues. Finally, we speculate on where the future of computational analysis in developmental biology might be headed. This article is categorized under: Technologies > Analysis of Cell, Tissue, and Animal Phenotypes Early Embryonic Development > Gastrulation and Neurulation Early Embryonic Development > Development to the Basic Body Plan.

Automatic Quantitative Segmentation of Myotubes Reveals Single-cell Dynamics of S6 Kinase Activation

Automatic cell segmentation is a powerful method for quantifying signaling dynamics at single-cell resolution in live cell fluorescence imaging. Segmentation methods for mononuclear and round shape cells have been developed extensively. However, a segmentation method for elongated polynuclear cells, such as differentiated C2C12 myotubes, has yet to be developed. In addition, myotubes are surrounded by undifferentiated reserve cells, making it difficult to identify background regions and subsequent quantification. Here we developed an automatic quantitative segmentation method for myotubes using watershed segmentation of summed binary images and a two-component Gaussian mixture model. We used time-lapse fluorescence images of differentiated C2C12 cells stably expressing Eevee-S6K, a fluorescence resonance energy transfer (FRET) biosensor of S6 kinase (S6K). Summation of binary images enhanced the contrast between myotubes and reserve cells, permitting detection of a myotube and a myotube center. Using a myotube center instead of a nucleus, individual myotubes could be detected automatically by watershed segmentation. In addition, a background correction using the two-component Gaussian mixture model permitted automatic signal intensity quantification in individual myotubes. Thus, we provide an automatic quantitative segmentation method by combining automatic myotube detection and background correction. Furthermore, this method allowed us to quantify S6K activity in individual myotubes, demonstrating that some of the temporal properties of S6K activity such as peak time and half-life of adaptation show different dose-dependent changes of insulin between cell population and individuals.Key words: time lapse images, cell segmentation, fluorescence resonance energy transfer, C2C12, myotube.

Automated Cell Segmentation for Quantitative Phase Microscopy

Automated cell segmentation and tracking is essential for dynamic studies of cellular morphology, movement, and interactions as well as other cellular behaviors. However, accurate, automated, and easy-to-use cell segmentation remains a challenge, especially in cases of high cell densities, where discrete boundaries are not easily discernable. Here, we present a fully automated segmentation algorithm that iteratively segments cells based on the observed distribution of optical cell volumes measured by quantitative phase microscopy. By fitting these distributions to known probability density functions, we are able to converge on volumetric thresholds that enable valid segmentation cuts. Since each threshold is determined from the observed data itself, virtually no input is needed from the user. We demonstrate the effectiveness of this approach over time using six cell types that display a range of morphologies, and evaluate these cultures over a range of confluencies. Facile dynamic measures of cell mobility and function revealed unique cellular behaviors that relate to tissue origins, state of differentiation, and real-time signaling. These will improve our understanding of multicellular communication and organization.

Phase contrast cell detection using multilevel classification

In this paper, we propose a fully automated learning-based approach for detecting cells in time-lapse phase contrast images. The proposed system combines 2 machine learning approaches to achieve bottom-up image segmentation. We apply pixel-wise classification using random forests (RF) classifiers to determine the potential location of the cells. Each pixel is classified into 4 categories (cell, mitotic cell, halo effect, and background noise). Various image features are extracted at different scales to train the RF classifier. The resulting probability map is partitioned using the k-means algorithm to form potential cell regions. These regions are expanded into the neighboring areas to recover some missing or broken cell regions. To validate the cell regions, another machine learning method based on the bag-of-features and spatial pyramid encoding is proposed. The result of the second classifier can be a validated cell, a merged cell, or a noncell. In the case that the cell region is classified as a merged cell, it is split by using the seeded watershed method. The proposed method is demonstrated on several phase contrast image datasets, ie, U2OS, HeLa, and NIH 3T3. In comparison to state-of-the-art cell detection techniques, the proposed method shows improved performance, particularly in dealing with noise interference and drastic shape variations.

Deciphering tissue morphodynamics using bioimage informatics

In recent years developmental biology has greatly benefited from the latest advances in fluorescence microscopy techniques. Consequently, quantitative and automated analysis of this data is becoming a vital first step in the quest for novel insights into the various aspects of development. Here we present an introductory overview of the various image analysis methods proposed for developmental biology images, with particular attention to openly available software packages. These tools, as well as others to come, are rapidly paving the way towards standardized and reproducible bioimaging studies at the whole-tissue level. Reflecting on these achievements, we discuss the remaining challenges and the future endeavours lying ahead in the post-image analysis era.This article is part of the themed issue 'Systems morphodynamics: understanding the development of tissue hardware'.

Nuclear spatial and spectral features based evolutionary method for meningioma subtypes classification in histopathology

Meningioma subtypes classification is a real-world multiclass problem from the realm of neuropathology. The major challenge in solving this problem is the inherent complexity due to high intra-class variability and low inter-class variation in tissue samples. The development of computational methods to assist pathologists in characterization of these tissue samples would have great diagnostic and prognostic value. In this article, we proposed an optimized evolutionary framework for the classification of benign meningioma into four subtypes. This framework investigates the imperative role of RGB color channels for discrimination of tumor subtypes and compute structural, statistical and spectral phenotypes. An evolutionary technique, Genetic Algorithm, in combination with Support Vector Machine is applied to tune classifier parameters and to select the best possible combination of extracted phenotypes that improved the classification accuracy (94.88%) on meningioma histology dataset, provided by the Institute of Neuropathology, Bielefeld. These statistics show that computational framework can robustly discriminate four subtypes of benign meningioma and may aid pathologists in the diagnosis and classification of these lesions.

Robust Cell Segmentation for Histological Images of Glioblastoma

Glioblastoma (GBM) is a malignant brain tumor with uniformly dismal prognosis. Quantitative analysis of GBM cells is an important avenue to extract latent histologic disease signatures to correlate with molecular underpinnings and clinical outcomes. As a prerequisite, a robust and accurate cell segmentation is required. In this paper, we present an automated cell segmentation method that can satisfactorily address segmentation of overlapped cells commonly seen in GBM histology specimens. This method first detects cells with seed connectivity, distance constraints, image edge map, and a shape-based voting image. Initialized by identified seeds, cell boundaries are deformed with an improved variational level set method that can handle clumped cells. We test our method on 40 histological images of GBM with human annotations. The validation results suggest that our cell segmentation method is promising and represents an advance in quantitative cancer research.