An approach of separating the overlapped cells or nuclei based on the outer Canny edges and morphological erosion

In biomedicine, the automatic processing of medical microscope images plays a key role in the subsequent analysis and diagnosis. Cell or nucleus segmentation is one of the most challenging tasks for microscope image processing. Due to the frequently occurred overlapping, few segmentation methods can achieve satisfactory segmentation accuracy yet. In this paper, we propose an approach to separate the overlapped cells or nuclei based on the outer Canny edges and morphological erosion. The threshold selection is first used to segment the foreground and background of cell or nucleus images. For each binary connected domain in the segmentation image, an intersection based edge selection method is proposed to choose the outer Canny edges of the overlapped cells or nuclei. The outer Canny edges are used to generate a binary cell or nucleus image that is then used to compute the cell or nucleus seeds by the proposed morphological erosion method. The nuclei of the Human U2OS cells, the mouse NIH3T3 cells and the synthetic cells are used for evaluating our proposed approach. The quantitative quantification accuracy is computed by the Dice score and 95.53% is achieved by the proposed approach. Both the quantitative and the qualitative comparisons show that the accuracy of the proposed approach is better than those of the area constrained morphological erosion (ACME) method, the iterative erosion (IE) method, the morphology and watershed (MW) method, the Generalized Laplacian of Gaussian filters (GLGF) method and ellipse fitting (EF) method in separating the cells or nuclei in three publicly available datasets.

A Deep Learning Approach for Histology-Based Nucleus Segmentation and Tumor Microenvironment Characterization

Microscopic examination of pathology slides is essential to disease diagnosis and biomedical research. However, traditional manual examination of tissue slides is laborious and subjective. Tumor whole-slide image (WSI) scanning is becoming part of routine clinical procedures and produces massive data that capture tumor histologic details at high resolution. Furthermore, the rapid development of deep learning algorithms has significantly increased the efficiency and accuracy of pathology image analysis. In light of this progress, digital pathology is fast becoming a powerful tool to assist pathologists. Studying tumor tissue and its surrounding microenvironment provides critical insight into tumor initiation, progression, metastasis, and potential therapeutic targets. Nucleus segmentation and classification are critical to pathology image analysis, especially in characterizing and quantifying the tumor microenvironment (TME). Computational algorithms have been developed for nucleus segmentation and TME quantification within image patches. However, existing algorithms are computationally intensive and time consuming for WSI analysis. This study presents Histology-based Detection using Yolo (HD-Yolo), a new method that significantly accelerates nucleus segmentation and TME quantification. We demonstrate that HD-Yolo outperforms existing WSI analysis methods in nucleus detection, classification accuracy, and computation time. We validated the advantages of the system on 3 different tissue types: lung cancer, liver cancer, and breast cancer. For breast cancer, nucleus features by HD-Yolo were more prognostically significant than both the estrogen receptor status by immunohistochemistry and the progesterone receptor status by immunohistochemistry. The WSI analysis pipeline and a real-time nucleus segmentation viewer are available at https://github.com/impromptuRong/hd_wsi.

NuKit: A deep learning platform for fast nucleus segmentation of histopathological images

Nucleus segmentation represents the initial step for histopathological image analysis pipelines, and it remains a challenge in many quantitative analysis methods in terms of accuracy and speed. Recently, deep learning nucleus segmentation methods have demonstrated to outperform previous intensity- or pattern-based methods. However, the heavy computation of deep learning provides impression of lagging response in real time and hampered the adoptability of these models in routine research. We developed and implemented NuKit a deep learning platform, which accelerates nucleus segmentation and provides prompt results to the users. NuKit platform consists of two deep learning models coupled with an interactive graphical user interface (GUI) to provide fast and automatic nucleus segmentation "on the fly". Both deep learning models provide complementary tasks in nucleus segmentation. The whole image segmentation model performs whole image nucleus whereas the click segmentation model supplements the nucleus segmentation with user-driven input to edits the segmented nuclei. We trained the NuKit whole image segmentation model on a large public training data set and tested its performance in seven independent public image data sets. The whole image segmentation model achieves average [Formula: see text] and [Formula: see text]. The outputs could be exported into different file formats, as well as provides seamless integration with other image analysis tools such as QuPath. NuKit can be executed on Windows, Mac, and Linux using personal computers.

NuCLS: A scalable crowdsourcing approach and dataset for nucleus classification and segmentation in breast cancer

Background

Deep learning enables accurate high-resolution mapping of cells and tissue structures that can serve as the foundation of interpretable machine-learning models for computational pathology. However, generating adequate labels for these structures is a critical barrier, given the time and effort required from pathologists.

Results

This article describes a novel collaborative framework for engaging crowds of medical students and pathologists to produce quality labels for cell nuclei. We used this approach to produce the NuCLS dataset, containing >220,000 annotations of cell nuclei in breast cancers. This builds on prior work labeling tissue regions to produce an integrated tissue region- and cell-level annotation dataset for training that is the largest such resource for multi-scale analysis of breast cancer histology. This article presents data and analysis results for single and multi-rater annotations from both non-experts and pathologists. We present a novel workflow that uses algorithmic suggestions to collect accurate segmentation data without the need for laborious manual tracing of nuclei. Our results indicate that even noisy algorithmic suggestions do not adversely affect pathologist accuracy and can help non-experts improve annotation quality. We also present a new approach for inferring truth from multiple raters and show that non-experts can produce accurate annotations for visually distinctive classes.

Conclusions

This study is the most extensive systematic exploration of the large-scale use of wisdom-of-the-crowd approaches to generate data for computational pathology applications.

Convolutional Blur Attention Network for Cell Nuclei Segmentation

Accurately segmented nuclei are important, not only for cancer classification, but also for predicting treatment effectiveness and other biomedical applications. However, the diversity of cell types, various external factors, and illumination conditions make nucleus segmentation a challenging task. In this work, we present a new deep learning-based method for cell nucleus segmentation. The proposed convolutional blur attention (CBA) network consists of downsampling and upsampling procedures. A blur attention module and a blur pooling operation are used to retain the feature salience and avoid noise generation in the downsampling procedure. A pyramid blur pooling (PBP) module is proposed to capture the multi-scale information in the upsampling procedure. The superiority of the proposed method has been compared with a few prior segmentation models, namely U-Net, ENet, SegNet, LinkNet, and Mask RCNN on the 2018 Data Science Bowl (DSB) challenge dataset and the multi-organ nucleus segmentation (MoNuSeg) at MICCAI 2018. The Dice similarity coefficient and some evaluation matrices, such as F1 score, recall, precision, and average Jaccard index (AJI) were used to evaluate the segmentation efficiency of these models. Overall, the proposal method in this paper has the best performance, the AJI indicator on the DSB dataset and MoNuSeg is 0.8429, 0.7985, respectively.

Improved Quantification of Cell Density in the Arterial Wall-A Novel Nucleus Splitting Approach Applied to 3D Two-Photon Laser-Scanning Microscopy

Accurate information on vascular smooth muscle cell (VSMC) content, orientation, and distribution in blood vessels is indispensable to increase understanding of arterial remodeling and to improve modeling of vascular biomechanics. We have previously proposed an analysis method to automatically characterize VSMC orientation and transmural distribution in murine carotid arteries under well-controlled biomechanical conditions. However, coincident nuclei, erroneously detected as one large nucleus, were excluded from the analysis, hampering accurate VSMC content characterization and distorting transmural distributions. In the present study, therefore, we aim to (1) improve the previous method by adding a "nucleus splitting" procedure to split coinciding nuclei, (2) evaluate the accuracy of this novel method, and (3) test this method in a mouse model of VSMC apoptosis. After euthanasia, carotid arteries from SM22α-hDTR Apoe -/- and control Apoe -/- mice were bluntly dissected, excised, mounted in a biaxial biomechanical tester and brought to in vivo axial stretch and a pressure of 100 mmHg. Nuclei and elastin fibers were then stained using Syto-41 and Eosin-Y, respectively, and imaged using 3D two-photon laser scanning microscopy. Nuclei were segmented from images and coincident nuclei were split. The nucleus splitting procedure determines the likelihood that voxel pairs within coincident nuclei belong to the same nucleus and utilizes these likelihoods to identify individual nuclei using spectral clustering. Manual nucleus counts were used as a reference to assess the performance of our splitting procedure. Before and after splitting, automatic nucleus counts differed -26.6 ± 9.90% (p < 0.001) and -1.44 ± 7.05% (p = 0.467) from the manual reference, respectively. Whereas the slope of the relative difference between the manual and automated counts as a function of the manual count was significantly negative before splitting (p = 0.008), this slope became insignificant after splitting (p = 0.653). Smooth muscle apoptosis led to a 33.7% decrease in VSMC density (p = 0.008). Nucleus splitting improves the accuracy of automated cell content quantification in murine carotid arteries and overcomes the progressively worsening problem of coincident nuclei with increasing cell content in vessels. The presented image analysis framework provides a robust tool to quantify cell content, orientation, shape, and distribution in vessels to inform experimental and advanced computational studies on vascular structure and function.

A Deep Learning Pipeline for Nucleus Segmentation

Deep learning is rapidly becoming the technique of choice for automated segmentation of nuclei in biological image analysis workflows. In order to evaluate the feasibility of training nuclear segmentation models on small, custom annotated image datasets that have been augmented, we have designed a computational pipeline to systematically compare different nuclear segmentation model architectures and model training strategies. Using this approach, we demonstrate that transfer learning and tuning of training parameters, such as the composition, size, and preprocessing of the training image dataset, can lead to robust nuclear segmentation models, which match, and often exceed, the performance of existing, off-the-shelf deep learning models pretrained on large image datasets. We envision a practical scenario where deep learning nuclear segmentation models trained in this way can be shared across a laboratory, facility, or institution, and continuously improved by training them on progressively larger and varied image datasets. Our work provides computational tools and a practical framework for deep learning-based biological image segmentation using small annotated image datasets. Published [2020]. This article is a U.S. Government work and is in the public domain in the USA.

A Multi-Organ Nucleus Segmentation Challenge

Generalized nucleus segmentation techniques can contribute greatly to reducing the time to develop and validate visual biomarkers for new digital pathology datasets. We summarize the results of MoNuSeg 2018 Challenge whose objective was to develop generalizable nuclei segmentation techniques in digital pathology. The challenge was an official satellite event of the MICCAI 2018 conference in which 32 teams with more than 80 participants from geographically diverse institutes participated. Contestants were given a training set with 30 images from seven organs with annotations of 21,623 individual nuclei. A test dataset with 14 images taken from seven organs, including two organs that did not appear in the training set was released without annotations. Entries were evaluated based on average aggregated Jaccard index (AJI) on the test set to prioritize accurate instance segmentation as opposed to mere semantic segmentation. More than half the teams that completed the challenge outperformed a previous baseline. Among the trends observed that contributed to increased accuracy were the use of color normalization as well as heavy data augmentation. Additionally, fully convolutional networks inspired by variants of U-Net, FCN, and Mask-RCNN were popularly used, typically based on ResNet or VGG base architectures. Watershed segmentation on predicted semantic segmentation maps was a popular post-processing strategy. Several of the top techniques compared favorably to an individual human annotator and can be used with confidence for nuclear morphometrics.

Nucleus segmentation of cervical cytology images based on multi-scale fuzzy clustering algorithm

In the screening of cervical cancer cells, accurate identification and segmentation of nucleus in cell images is a key part in the early diagnosis of cervical cancer. Overlapping, uneven staining, poor contrast, and other reasons present challenges to cervical nucleus segmentation. We propose a segmentation method for cervical nuclei based on a multi-scale fuzzy clustering algorithm, which segments cervical cell clump images at different scales. We adopt a novel interesting degree based on area prior to measure the interesting degree of the node. The application of these two methods not only solves the problem of selecting the categories number of the clustering algorithm but also greatly improves the nucleus recognition performance. The method is evaluated by the IBSI2014 and IBSI2015 public datasets. Experiments show that the proposed algorithm has greater advantages than the state-of-the-art cervical nucleus segmentation algorithms and accomplishes high accuracy nucleus segmentation results.

Object-Oriented Segmentation of Cell Nuclei in Fluorescence Microscopy Images

Cell nucleus segmentation remains an open and challenging problem especially to segment nuclei in cell clumps. Splitting a cell clump would be straightforward if the gradients of boundary pixels in-between the nuclei were always higher than the others. However, imperfections may exist: inhomogeneities of pixel intensities in a nucleus may cause to define spurious boundaries whereas insufficient pixel intensity differences at the border of overlapping nuclei may cause to miss some true boundary pixels. In contrast, these imperfections are typically observed at the pixel-level, causing local changes in pixel values without changing the semantics on a large scale. In response to these issues, this article introduces a new nucleus segmentation method that relies on using gradient information not at the pixel level but at the object level. To this end, it proposes to decompose an image into smaller homogeneous subregions, define edge-objects at four different orientations to encode the gradient information at the object level, and devise a merging algorithm, in which the edge-objects vote for subregion pairs along their orientations and the pairs are iteratively merged if they get sufficient votes from multiple orientations. Our experiments on fluorescence microscopy images reveal that this high-level representation and the design of a merging algorithm using edge-objects (gradients at the object level) improve the segmentation results.

An Automatic Method for Nucleus Boundary Segmentation Based on a Closed Cubic Spline

The recognition of brain nuclei is the basis for localizing brain functions. Traditional histological research, represented by atlas illustration, achieves the goal of nucleus boundary recognition by manual delineation, but it has become increasingly difficult to extend this handmade method to delineating brain regions and nuclei from large datasets acquired by the recently developed single-cell-resolution imaging techniques for the whole brain. Here, we propose a method based on a closed cubic spline (CCS), which can automatically segment the boundaries of nuclei that differ to a relatively high degree in cell density from the surrounding areas and has been validated on model images and Nissl-stained microimages of mouse brain. It may even be extended to the segmentation of target outlines on MRI or CT images. The proposed method for the automatic extraction of nucleus boundaries would greatly accelerate the illustration of high-resolution brain atlases.

Hierarchical nucleus segmentation in digital pathology images

Extracting nuclei is one of the most actively studied topic in the digital pathology researches. Most of the studies directly search the nuclei (or seeds for the nuclei) from the finest resolution available. While the richest information has been utilized by such approaches, it is sometimes difficult to address the heterogeneity of nuclei in different tissues. In this work, we propose a hierarchical approach which starts from the lower resolution level and adaptively adjusts the parameters while progressing into finer and finer resolution. The algorithm is tested on brain and lung cancers images from The Cancer Genome Atlas data set.

Iterative h-minima-based marker-controlled watershed for cell nucleus segmentation

Automated microscopy imaging systems facilitate high-throughput screening in molecular cellular biology research. The first step of these systems is cell nucleus segmentation, which has a great impact on the success of the overall system. The marker-controlled watershed is a technique commonly used by the previous studies for nucleus segmentation. These studies define their markers finding regional minima on the intensity/gradient and/or distance transform maps. They typically use the h-minima transform beforehand to suppress noise on these maps. The selection of the h value is critical; unnecessarily small values do not sufficiently suppress the noise, resulting in false and oversegmented markers, and unnecessarily large ones suppress too many pixels, causing missing and undersegmented markers. Because cell nuclei show different characteristics within an image, the same h value may not work to define correct markers for all the nuclei. To address this issue, in this work, we propose a new watershed algorithm that iteratively identifies its markers, considering a set of different h values. In each iteration, the proposed algorithm defines a set of candidates using a particular h value and selects the markers from those candidates provided that they fulfill the size requirement. Working with widefield fluorescence microscopy images, our experiments reveal that the use of multiple h values in our iterative algorithm leads to better segmentation results, compared to its counterparts. © 2016 International Society for Advancement of Cytometry.