A deep learning algorithm for one-step contour aware nuclei segmentation of histopathology images

This paper addresses the task of nuclei segmentation in high-resolution histopathology images. We propose an automatic end-to-end deep neural network algorithm for segmentation of individual nuclei. A nucleus-boundary model is introduced to predict nuclei and their boundaries simultaneously using a fully convolutional neural network. Given a color-normalized image, the model directly outputs an estimated nuclei map and a boundary map. A simple, fast, and parameter-free post-processing procedure is performed on the estimated nuclei map to produce the final segmented nuclei. An overlapped patch extraction and assembling method is also designed for seamless prediction of nuclei in large whole-slide images. We also show the effectiveness of data augmentation methods for nuclei segmentation task. Our experiments showed our method outperforms prior state-of-the-art methods. Moreover, it is efficient that one 1000×1000 image can be segmented in less than 5 s. This makes it possible to precisely segment the whole-slide image in acceptable time. The source code is available at https://github.com/easycui/nuclei_segmentation . Graphical Abstract The neural network for nuclei segmentation.

NucleiSegNet: Robust deep learning architecture for the nuclei segmentation of liver cancer histopathology images

The nuclei segmentation of hematoxylin and eosin (H&E) stained histopathology images is an important prerequisite in designing a computer-aided diagnostics (CAD) system for cancer diagnosis and prognosis. Automated nuclei segmentation methods enable the qualitative and quantitative analysis of tens of thousands of nuclei within H&E stained histopathology images. However, a major challenge during nuclei segmentation is the segmentation of variable sized, touching nuclei. To address this challenge, we present NucleiSegNet - a robust deep learning network architecture for the nuclei segmentation of H&E stained liver cancer histopathology images. Our proposed architecture includes three blocks: a robust residual block, a bottleneck block, and an attention decoder block. The robust residual block is a newly proposed block for the efficient extraction of high-level semantic maps. The attention decoder block uses a new attention mechanism for efficient object localization, and it improves the proposed architecture's performance by reducing false positives. When applied to nuclei segmentation tasks, the proposed deep-learning architecture yielded superior results compared to state-of-the-art nuclei segmentation methods. We applied our proposed deep learning architecture for nuclei segmentation to a set of H&E stained histopathology images from two datasets, and our comprehensive results show that our proposed architecture outperforms state-of-the-art methods. As part of this work, we also introduced a new liver dataset (KMC liver dataset) of H&E stained liver cancer histopathology image tiles, containing 80 images with annotated nuclei procured from Kasturba Medical College (KMC), Mangalore, Manipal Academy of Higher Education (MAHE), Manipal, Karnataka, India. The proposed model's source code is available at https://github.com/shyamfec/NucleiSegNet.

Triple U-net: Hematoxylin-aware nuclei segmentation with progressive dense feature aggregation

Nuclei segmentation is a vital step for pathological cancer research. It is still an open problem due to some difficulties, such as color inconsistency introduced by non-uniform manual operations, blurry tumor nucleus boundaries and overlapping tumor cells. In this paper, we aim to leverage the unique optical characteristic of H&E staining images that hematoxylin always stains cell nuclei blue, and eosin always stains the extracellular matrix and cytoplasm pink. Therefore, we extract the Hematoxylin component from RGB images by Beer-Lambert's Law. According to the optical attribute, the extracted Hematoxylin component is robust to color inconsistency. With the Hematoxylin component, we propose a Hematoxylin-aware CNN model for nuclei segmentation without the necessity of color normalization. Our proposed network is formulated as a Triple U-net structure which includes an RGB branch, a Hematoxylin branch and a Segmentation branch. Then we propose a novel feature aggregation strategy to allow the network to fuse features progressively and to learn better feature representations from different branches. Extensive experiments are performed to qualitatively and quantitatively evaluate the effectiveness of our proposed method. In the meanwhile, it outperforms state-of-the-art methods on three different nuclei segmentation datasets.

Multi-tissue and multi-scale approach for nuclei segmentation in H&E stained images

Background

Accurate nuclei detection and segmentation in histological images is essential for many clinical purposes. While manual annotations are time-consuming and operator-dependent, full automated segmentation remains a challenging task due to the high variability of cells intensity, size and morphology. Most of the proposed algorithms for the automated segmentation of nuclei were designed for specific organ or tissues.

Results

The aim of this study was to develop and validate a fully multiscale method, named MANA (Multiscale Adaptive Nuclei Analysis), for nuclei segmentation in different tissues and magnifications. MANA was tested on a dataset of H&E stained tissue images with more than 59,000 annotated nuclei, taken from six organs (colon, liver, bone, prostate, adrenal gland and thyroid) and three magnifications (10×, 20×, 40×). Automatic results were compared with manual segmentations and three open-source software designed for nuclei detection. For each organ, MANA obtained always an F1-score higher than 0.91, with an average F1 of 0.9305 ± 0.0161. The average computational time was about 20 s independently of the number of nuclei to be detected (anyway, higher than 1000), indicating the efficiency of the proposed technique.

Conclusion

To the best of our knowledge, MANA is the first fully automated multi-scale and multi-tissue algorithm for nuclei detection. Overall, the robustness and versatility of MANA allowed to achieve, on different organs and magnifications, performances in line or better than those of state-of-art algorithms optimized for single tissues.

An automatic nuclei segmentation method based on deep convolutional neural networks for histopathology images

BACKGROUND

Since nuclei segmentation in histopathology images can provide key information for identifying the presence or stage of a disease, the images need to be assessed carefully. However, color variation in histopathology images, and various structures of nuclei are two major obstacles in accurately segmenting and analyzing histopathology images. Several machine learning methods heavily rely on hand-crafted features which have limitations due to manual thresholding.

RESULTS

To obtain robust results, deep learning based methods have been proposed. Deep convolutional neural networks (DCNN) used for automatically extracting features from raw image data have been proven to achieve great performance. Inspired by such achievements, we propose a nuclei segmentation method based on DCNNs. To normalize the color of histopathology images, we use a deep convolutional Gaussian mixture color normalization model which is able to cluster pixels while considering the structures of nuclei. To segment nuclei, we use Mask R-CNN which achieves state-of-the-art object segmentation performance in the field of computer vision. In addition, we perform multiple inference as a post-processing step to boost segmentation performance. We evaluate our segmentation method on two different datasets. The first dataset consists of histopathology images of various organ while the other consists histopathology images of the same organ. Performance of our segmentation method is measured in various experimental setups at the object-level and the pixel-level. In addition, we compare the performance of our method with that of existing state-of-the-art methods. The experimental results show that our nuclei segmentation method outperforms the existing methods.

CONCLUSIONS

We propose a nuclei segmentation method based on DCNNs for histopathology images. The proposed method which uses Mask R-CNN with color normalization and multiple inference post-processing provides robust nuclei segmentation results. Our method also can facilitate downstream nuclei morphological analyses as it provides high-quality features extracted from histopathology images.

Cell Nuclei Segmentation in Cytological Images Using Convolutional Neural Network and Seeded Watershed Algorithm

Morphometric analysis of nuclei is crucial in cytological examinations. Unfortunately, nuclei segmentation presents many challenges because they usually create complex clusters in cytological samples. To deal with this problem, we are proposing an approach, which combines convolutional neural network and watershed transform to segment nuclei in cytological images of breast cancer. The method initially is preprocessing images using color deconvolution to highlight hematoxylin-stained objects (nuclei). Next, convolutional neural network is applied to perform semantic segmentation of preprocessed image. It finds nuclei areas, cytoplasm areas, edges of nuclei, and background. All connected components in the binary mask of nuclei are treated as potential nuclei. However, some objects actually are clusters of overlapping nuclei. They are detected by their outlying values of morphometric features. Then an attempt is made to separate them using the seeded watershed segmentation. If the attempt is successful, they are included in the nuclei set. The accuracy of this approach is evaluated with the help of referenced, manually segmented images. The degree of matching between reference nuclei and discovered objects is measured with the help of Jaccard distance and Hausdorff distance. As part of the study, we verified how the use of a convolutional neural network instead of the intensity thresholding to generate a topographical map for the watershed improves segmentation outcomes. Our results show that convolutional neural network outperforms Otsu thresholding and adaptive thresholding in most cases, especially in scenarios with many overlapping nuclei.

CryoNuSeg: A dataset for nuclei instance segmentation of cryosectioned H&E-stained histological images

Nuclei instance segmentation plays an important role in the analysis of hematoxylin and eosin (H&E)-stained images. While supervised deep learning (DL)-based approaches represent the state-of-the-art in automatic nuclei instance segmentation, annotated datasets are required to train these models. There are two main types of tissue processing protocols resulting in formalin-fixed paraffin-embedded samples (FFPE) and frozen tissue samples (FS), respectively. Although FFPE-derived H&E stained tissue sections are the most widely used samples, H&E staining of frozen sections derived from FS samples is a relevant method in intra-operative surgical sessions as it can be performed more rapidly. Due to differences in the preparation of these two types of samples, the derived images and in particular the nuclei appearance may be different in the acquired whole slide images. Analysis of FS-derived H&E stained images can be more challenging as rapid preparation, staining, and scanning of FS sections may lead to deterioration in image quality. In this paper, we introduce CryoNuSeg, the first fully annotated FS-derived cryosectioned and H&E-stained nuclei instance segmentation dataset. The dataset contains images from 10 human organs that were not exploited in other publicly available datasets, and is provided with three manual mark-ups to allow measuring intra-observer and inter-observer variabilities. Moreover, we investigate the effects of tissue fixation/embedding protocol (i.e., FS or FFPE) on the automatic nuclei instance segmentation performance and provide a baseline segmentation benchmark for the dataset that can be used in future research. A step-by-step guide to generate the dataset as well as the full dataset and other detailed information are made available to fellow researchers at https://github.com/masih4/CryoNuSeg.

DenseRes-Unet: Segmentation of overlapped/clustered nuclei from multi organ histopathology images

Cancer is the second deadliest disease globally that can affect any human body organ. Early detection of cancer can increase the chances of survival in humans. Morphometric appearances of histopathology images make it difficult to segment nuclei effectively. We proposed a model to segment overlapped nuclei from H&E stained images. U-Net model achieved state-of-the-art performance in many medical image segmentation tasks; however, we modified the U-Net to learn a distinct set of consistent features. In this paper, we proposed the DenseRes-Unet model by integrating dense blocks in the last layers of the encoder block of U-Net, focused on relevant features from previous layers of the model. Moreover, we take advantage of residual connections with Atrous blocks instead of conventional skip connections, which helps to reduce the semantic gap between encoder and decoder paths. The distance map and binary threshold techniques intensify the nuclei interior and contour information in the images, respectively. The distance map is used to detect the center point of nuclei; moreover, it differentiates among nuclei interior boundary and core area. The distance map lacks a contour problem, which is resolved by using a binary threshold. Binary threshold helps to enhance the pixels around nuclei. Afterward, we fed images into the proposed DenseRes-Unet model, a deep, fully convolutional network to segment nuclei in the images. We have evaluated our model on four publicly available datasets for Nuclei segmentation to validate the model's performance. Our proposed model achieves 89.77% accuracy 90.36% F1-score, and 78.61% Aggregated Jaccard Index (AJI) on Multi organ Nucleus Segmentation (MoNuSeg).

BENDING LOSS REGULARIZED NETWORK FOR NUCLEI SEGMENTATION IN HISTOPATHOLOGY IMAGES

Separating overlapped nuclei is a major challenge in histopathology image analysis. Recently published approaches have achieved promising overall performance on public datasets; however, their performance in segmenting overlapped nuclei are limited. To address the issue, we propose the bending loss regularized network for nuclei segmentation. The proposed bending loss defines high penalties to contour points with large curvatures, and applies small penalties to contour points with small curvature. Minimizing the bending loss can avoid generating contours that encompass multiple nuclei. The proposed approach is validated on the MoNuSeg dataset using five quantitative metrics. It outperforms six state-of-the-art approaches on the following metrics: Aggregate Jaccard Index, Dice, Recognition Quality, and Panoptic Quality.

Fast unsupervised nuclear segmentation and classification scheme for automatic allred cancer scoring in immunohistochemical breast tissue images

BACKGROUND AND OBJECTIVE

This paper presents an improved scheme able to perform accurate segmentation and classification of cancer nuclei in immunohistochemical (IHC) breast tissue images in order to provide quantitative evaluation of estrogen or progesterone (ER/PR) receptor status that will assist pathologists in cancer diagnostic process.

METHODS

The proposed segmentation method is based on adaptive local thresholding and an enhanced morphological procedure, which are applied to extract all stained nuclei regions and to split overlapping nuclei. In fact, a new segmentation approach is presented here for cell nuclei detection from the IHC image using a modified Laplacian filter and an improved watershed algorithm. Stromal cells are then removed from the segmented image using an adaptive criterion in order to get fast tumor nuclei recognition. Finally, unsupervised classification of cancer nuclei is obtained by the combination of four common color separation techniques for a subsequent Allred cancer scoring.

RESULTS

Experimental results on various IHC tissue images of different cancer affected patients, demonstrate the effectiveness of the proposed scheme when compared to the manual scoring of pathological experts. A statistical analysis is performed on the whole image database between immuno-score of manual and automatic method, and compared with the scores that have reached using other state-of-art segmentation and classification strategies. According to the performance evaluation, we recorded more than 98% for both accuracy of detected nuclei and image cancer scoring over the truths provided by experienced pathologists which shows the best correlation with the expert's score (Pearson's correlation coefficient = 0.993, p-value < 0.005) and the lowest computational total time of 72.3 s/image (±1.9) compared to recent studied methods.

CONCLUSIONS

The proposed scheme can be easily applied for any histopathological diagnostic process that needs stained nuclear quantification and cancer grading. Moreover, the reduced processing time and manual interactions of our procedure can facilitate its implementation in a real-time device to construct a fully online evaluation system of IHC tissue images.

Nuclei segmentation with point annotations from pathology images via self-supervised learning and co-training

Nuclei segmentation is a crucial task for whole slide image analysis in digital pathology. Generally, the segmentation performance of fully-supervised learning heavily depends on the amount and quality of the annotated data. However, it is time-consuming and expensive for professional pathologists to provide accurate pixel-level ground truth, while it is much easier to get coarse labels such as point annotations. In this paper, we propose a weakly-supervised learning method for nuclei segmentation that only requires point annotations for training. First, coarse pixel-level labels are derived from the point annotations based on the Voronoi diagram and the k-means clustering method to avoid overfitting. Second, a co-training strategy with an exponential moving average method is designed to refine the incomplete supervision of the coarse labels. Third, a self-supervised visual representation learning method is tailored for nuclei segmentation of pathology images that transforms the hematoxylin component images into the H&E stained images to gain better understanding of the relationship between the nuclei and cytoplasm. We comprehensively evaluate the proposed method using two public datasets. Both visual and quantitative results demonstrate the superiority of our method to the state-of-the-art methods, and its competitive performance compared to the fully-supervised methods. Codes are available at https://github.com/hust-linyi/SC-Net.

Efficient deep learning architecture with dimension-wise pyramid pooling for nuclei segmentation of histopathology images

Image segmentation remains to be one of the most vital tasks in the area of computer vision and more so in the case of medical image processing. Image segmentation quality is the main metric that is often considered with memory and computation efficiency overlooked, limiting the use of power hungry models for practical use. In this paper, we propose a novel framework (Kidney-SegNet) that combines the effectiveness of an attention based encoder-decoder architecture with atrous spatial pyramid pooling with highly efficient dimension-wise convolutions. The segmentation results of the proposed Kidney-SegNet architecture have been shown to outperform existing state-of-the-art deep learning methods by evaluating them on two publicly available kidney and TNBC breast H&E stained histopathology image datasets. Further, our simulation experiments also reveal that the computational complexity and memory requirement of our proposed architecture is very efficient compared to existing deep learning state-of-the-art methods for the task of nuclei segmentation of H&E stained histopathology images. The source code of our implementation will be available at https://github.com/Aaatresh/Kidney-SegNet.

Application of convolutional neural networks towards nuclei segmentation in localization-based super-resolution fluorescence microscopy images

BACKGROUND

Automated segmentation of nuclei in microscopic images has been conducted to enhance throughput in pathological diagnostics and biological research. Segmentation accuracy and speed has been significantly enhanced with the advent of convolutional neural networks. A barrier in the broad application of neural networks to nuclei segmentation is the necessity to train the network using a set of application specific images and image labels. Previous works have attempted to create broadly trained networks for universal nuclei segmentation; however, such networks do not work on all imaging modalities, and best results are still commonly found when the network is retrained on user specific data. Stochastic optical reconstruction microscopy (STORM) based super-resolution fluorescence microscopy has opened a new avenue to image nuclear architecture at nanoscale resolutions. Due to the large size and discontinuous features typical of super-resolution images, automatic nuclei segmentation can be difficult. In this study, we apply commonly used networks (Mask R-CNN and UNet architectures) towards the task of segmenting super-resolution images of nuclei. First, we assess whether networks broadly trained on conventional fluorescence microscopy datasets can accurately segment super-resolution images. Then, we compare the resultant segmentations with results obtained using networks trained directly on our super-resolution data. We next attempt to optimize and compare segmentation accuracy using three different neural network architectures.

RESULTS

Results indicate that super-resolution images are not broadly compatible with neural networks trained on conventional bright-field or fluorescence microscopy images. When the networks were trained on super-resolution data, however, we attained nuclei segmentation accuracies (F1-Score) in excess of 0.8, comparable to past results found when conducting nuclei segmentation on conventional fluorescence microscopy images. Overall, we achieved the best results utilizing the Mask R-CNN architecture.

CONCLUSIONS

We found that convolutional neural networks are powerful tools capable of accurately and quickly segmenting localization-based super-resolution microscopy images of nuclei. While broadly trained and widely applicable segmentation algorithms are desirable for quick use with minimal input, optimal results are still found when the network is both trained and tested on visually similar images. We provide a set of Colab notebooks to disseminate the software into the broad scientific community ( https://github.com/YangLiuLab/Super-Resolution-Nuclei-Segmentation ).

CAS: Cell Annotation Software - Research on Neuronal Tissue Has Never Been so Transparent

CAS (Cell Annotation Software) is a novel tool for analysis of microscopic images and selection of the cell soma or nucleus, depending on the research objectives in medicine, biology, bioinformatics, etc. It replaces time-consuming and tiresome manual analysis of single images not only with automatic methods for object segmentation based on the Statistical Dominance Algorithm, but also semi-automatic tools for object selection within a marked region of interest. For each image, a broad set of object parameters is computed, including shape features and optical and topographic characteristics, thus giving additional insight into data. Our solution for cell detection and analysis has been verified by microscopic data and its application in the annotation of the lateral geniculate nucleus has been examined in a case study.

SHARP-GAN: SHARPNESS LOSS REGULARIZED GAN FOR HISTOPATHOLOGY IMAGE SYNTHESIS

Existing deep learning-based approaches for histopathology image analysis require large annotated training sets to achieve good performance; but annotating histopathology images is slow and resource-intensive. Conditional generative adversarial networks have been applied to generate synthetic histopathology images to alleviate this issue, but current approaches fail to generate clear contours for overlapped and touching nuclei. In this study, We propose a sharpness loss regularized generative adversarial network to synthesize realistic histopathology images. The proposed network uses normalized nucleus distance map rather than the binary mask to encode nuclei contour information. The proposed sharpness loss enhances the contrast of nuclei contour pixels. The proposed method is evaluated using four image quality metrics and segmentation results on two public datasets. Both quantitative and qualitative results demonstrate that the proposed approach can generate realistic histopathology images with clear nuclei contours.

Multifractal-based nuclei segmentation in fish images

The method for nuclei segmentation in fluorescence in-situ hybridization (FISH) images, based on the inverse multifractal analysis (IMFA) is proposed. From the blue channel of the FISH image in RGB format, the matrix of Holder exponents, with one-by-one correspondence with the image pixels, is determined first. The following semi-automatic procedure is proposed: initial nuclei segmentation is performed automatically from the matrix of Holder exponents by applying predefined hard thresholding; then the user evaluates the result and is able to refine the segmentation by changing the threshold, if necessary. After successful nuclei segmentation, the HER2 (human epidermal growth factor receptor 2) scoring can be determined in usual way: by counting red and green dots within segmented nuclei, and finding their ratio. The IMFA segmentation method is tested over 100 clinical cases, evaluated by skilled pathologist. Testing results show that the new method has advantages compared to already reported methods.

Automated proliferation index calculation for skin melanoma biopsy images using machine learning

The Proliferation Index (PI) is an important diagnostic, predictive and prognostic parameter used for evaluating different types of cancer. This paper presents an automated technique to measure the PI values for skin melanoma images using machine learning algorithms. The proposed technique first analyzes a Mart-1 stained histology image and generates a region of interest (ROI) mask for the tumor. The ROI mask is then used to locate the tumor regions in the corresponding Ki-67 stained image. The nuclei in the Ki-67 ROI are then segmented and classified using a Convolutional Neural Network (CNN), and the PI value is calculated based on the number of the active and the passive nuclei. Experimental results show that the proposed technique can robustly segment (with 94 % accuracy) and classify the nuclei with a low computational complexity and the calculated PI values have less than 4 % average error.

Deep structured residual encoder-decoder network with a novel loss function for nuclei segmentation of kidney and breast histopathology images

To improve the process of diagnosis and treatment of cancer disease, automatic segmentation of haematoxylin and eosin (H & E) stained cell nuclei from histopathology images is the first step in digital pathology. The proposed deep structured residual encoder-decoder network (DSREDN) focuses on two aspects: first, it effectively utilized residual connections throughout the network and provides a wide and deep encoder-decoder path, which results to capture relevant context and more localized features. Second, vanished boundary of detected nuclei is addressed by proposing an efficient loss function that better train our proposed model and reduces the false prediction which is undesirable especially in healthcare applications. The proposed architecture experimented on three different publicly available H&E stained histopathological datasets namely: (I) Kidney (RCC) (II) Triple Negative Breast Cancer (TNBC) (III) MoNuSeg-2018. We have considered F1-score, Aggregated Jaccard Index (AJI), the total number of parameters, and FLOPs (Floating point operations), which are mostly preferred performance measure metrics for comparison of nuclei segmentation. The evaluated score of nuclei segmentation indicated that the proposed architecture achieved a considerable margin over five state-of-the-art deep learning models on three different histopathology datasets. Visual segmentation results show that the proposed DSREDN model accurately segment the nuclear regions than those of the state-of-the-art methods.

Automated Macro Approach to Quantify Synapse Density in 2D Confocal Images from Fixed Immunolabeled Neural Tissue Sections

This chapter describes an ImageJ/Fiji automated macro approach to estimate synapse densities in 2D fluorescence confocal microscopy images. The main step-by-step imaging workflow is explained, including example macro language scripts that perform all steps automatically for multiple images. Such tool provides a straightforward method for exploratory synapse screenings where hundreds to thousands of images need to be analyzed in order to render significant statistical information. The method can be adapted to any particular set of images where fixed brain slices have been immunolabeled against validated presynaptic and postsynaptic markers.

Improving generalization capability of deep learning-based nuclei instance segmentation by non-deterministic train time and deterministic test time stain normalization

With the advent of digital pathology and microscopic systems that can scan and save whole slide histological images automatically, there is a growing trend to use computerized methods to analyze acquired images. Among different histopathological image analysis tasks, nuclei instance segmentation plays a fundamental role in a wide range of clinical and research applications. While many semi- and fully-automatic computerized methods have been proposed for nuclei instance segmentation, deep learning (DL)-based approaches have been shown to deliver the best performances. However, the performance of such approaches usually degrades when tested on unseen datasets. In this work, we propose a novel method to improve the generalization capability of a DL-based automatic segmentation approach. Besides utilizing one of the state-of-the-art DL-based models as a baseline, our method incorporates non-deterministic train time and deterministic test time stain normalization, and ensembling to boost the segmentation performance. We trained the model with one single training set and evaluated its segmentation performance on seven test datasets. Our results show that the proposed method provides up to 4.9%, 5.4%, and 5.9% better average performance in segmenting nuclei based on Dice score, aggregated Jaccard index, and panoptic quality score, respectively, compared to the baseline segmentation model.

Microscopic nuclei classification, segmentation, and detection with improved deep convolutional neural networks (DCNN)

BACKGROUND

Nuclei classification, segmentation, and detection from pathological images are challenging tasks due to cellular heterogeneity in the Whole Slide Images (WSI).

METHODS

In this work, we propose advanced DCNN models for nuclei classification, segmentation, and detection tasks. The Densely Connected Neural Network (DCNN) and Densely Connected Recurrent Convolutional Network (DCRN) models are applied for the nuclei classification tasks. The Recurrent Residual U-Net (R2U-Net) and the R2UNet-based regression model named the University of Dayton Net (UD-Net) are applied for nuclei segmentation and detection tasks respectively. The experiments are conducted on publicly available datasets, including Routine Colon Cancer (RCC) classification and detection and the Nuclei Segmentation Challenge 2018 datasets for segmentation tasks. The experimental results were evaluated with a five-fold cross-validation method, and the average testing results are compared against the existing approaches in terms of precision, recall, Dice Coefficient (DC), Mean Squared Error (MSE), F1-score, and overall testing accuracy by calculating pixels and cell-level analysis.

RESULTS

The results demonstrate around 2.6% and 1.7% higher performance in terms of F1-score for nuclei classification and detection tasks when compared to the recently published DCNN based method. Also, for nuclei segmentation, the R2U-Net shows around 91.90% average testing accuracy in terms of DC, which is around 1.54% higher than the U-Net model.

CONCLUSION

The proposed methods demonstrate robustness with better quantitative and qualitative results in three different tasks for analyzing the WSI.

High-resolution deep transferred ASPPU-Net for nuclei segmentation of histopathology images

PURPOSE

Increasing cancer disease incidence worldwide has become a major public health issue. Manual histopathological analysis is a common diagnostic method for cancer detection. Due to the complex structure and wide variability in the texture of histopathology images, it has been challenging for pathologists to diagnose manually those images. Automatic segmentation of histopathology images to diagnose cancer disease is a continuous exploration field in recent times. Segmentation and analysis for diagnosis of histopathology images by using an efficient deep learning algorithm are the purpose of the proposed method.

METHOD

To improve the segmentation performance, we proposed a deep learning framework that consists of a high-resolution encoder path, an atrous spatial pyramid pooling bottleneck module, and a powerful decoder. Compared to the benchmark segmentation models having a deep and thin path, our network is wide and deep that effectively leverages the strength of residual learning as well as encoder-decoder architecture.

RESULTS

We performed careful experimentation and analysis on three publically available datasets namely kidney dataset, Triple Negative Breast Cancer (TNBC) dataset, and MoNuSeg histopathology image dataset. We have used the two most preferred performance metrics called F1 score and aggregated Jaccard index (AJI) to evaluate the performance of the proposed model. The measured values of F1 score and AJI score are (0.9684, 0.9394), (0.8419, 0.7282), and (0.8344, 0.7169) on the kidney dataset, TNBC histopathology dataset, and MoNuSeg dataset, respectively.

CONCLUSION

Our proposed method yields better results as compared to benchmark segmentation methods on three histopathology datasets. Visual segmentation results justify the high value of the F1 score and AJI scores which indicated that it is a very good prediction by our proposed model.

SAC-Net: Learning with weak and noisy labels in histopathology image segmentation

Deep convolutional neural networks have been highly effective in segmentation tasks. However, segmentation becomes more difficult when training images include many complex instances to segment, such as the task of nuclei segmentation in histopathology images. Weakly supervised learning can reduce the need for large-scale, high-quality ground truth annotations by involving non-expert annotators or algorithms to generate supervision information for segmentation. However, there is still a significant performance gap between weakly supervised learning and fully supervised learning approaches. In this work, we propose a weakly-supervised nuclei segmentation method in a two-stage training manner that only requires annotation of the nuclear centroids. First, we generate boundary and superpixel-based masks as pseudo ground truth labels to train our SAC-Net, which is a segmentation network enhanced by a constraint network and an attention network to effectively address the problems caused by noisy labels. Then, we refine the pseudo labels at the pixel level based on Confident Learning to train the network again. Our method shows highly competitive performance of cell nuclei segmentation in histopathology images on three public datasets. Code will be available at: https://github.com/RuoyuGuo/MaskGA_Net.

Image quilting and wavelet fusion for creation of synthetic microscopy nuclei images

BACKGROUND AND OBJECTIVE

In this study a texture simulation methodology is proposed for composing synthetic tissue microscopy images that could serve as a quantitative gold standard for the evaluation of the reliability, accuracy and performance of segmentation algorithms in computer-aided diagnosis.

METHODS

A library of background and nuclei regions was generated using pre-segmented Haematoxylin and Eosin images of brain tumours. Background image samples were used as input to an image quilting algorithm that produced the synthetic background image. Randomly selected pre-segmented nuclei were randomly fused on the synthetic background using a wavelet-based fusion approach. To investigate whether the produced synthetic images are meaningful and similar to real world images, two different tests were performed, one qualitative by an experienced histopathologist and one quantitative using the normalized mutual information and the Kullback-Leibler tests. To illustrate the challenges that synthetic images may pose to object recognition algorithms, two segmentation methodologies were utilized for nuclei detection, one based on the Otsu thresholding and another based on the seeded region growing approach.

RESULTS

Results showed a satisfactory to good resemblance of the synthetic with the real world images according to both qualitative and quantitative tests. The segmentation accuracy was slightly higher for the seeded region growing algorithm (87.2%) than the Otsu's algorithm (86.3%).

CONCLUSIONS

Since we know the exact coordinates of the regions of interest within the synthesised images, these images could then serve as a 'gold standard' for evaluation of segmentation algorithms in computer-aided diagnosis in tissue microscopy.

An artificial intelligence-based ecological index for prognostic evaluation of colorectal cancer

BACKGROUND AND OBJECTIVE

In the tumor microenvironment (TME), the dynamic interaction between tumor cells and immune cells plays a critical role in predicting the prognosis of colorectal cancer. This study introduces a novel approach based on artificial intelligence (AI) and immunohistochemistry (IHC)-stained whole-slide images (WSIs) of colorectal cancer (CRC) patients to quantitatively assess the spatial associations between tumor cells and immune cells. To achieve this, we employ the Morisita-Horn ecological index (Mor-index), which allows for a comprehensive analysis of the spatial distribution patterns between tumor cells and immune cells within the TME.

MATERIALS AND METHODS

In this study, we employed a combination of deep learning technology and traditional computer segmentation methods to accurately segment the tumor nuclei, immune nuclei, and stroma nuclei within the tumor regions of IHC-stained WSIs. The Mor-index was used to assess the spatial association between tumor cells and immune cells in TME of CRC patients by obtaining the results of cell nuclei segmentation. A discovery cohort (N = 432) and validation cohort (N = 137) were used to evaluate the prognostic value of the Mor-index for overall survival (OS).

RESULTS

The efficacy of our method was demonstrated through experiments conducted on two datasets comprising a total of 569 patients. Compared to other studies, our method is not only superior to the QuPath tool but also produces better segmentation results with an accuracy of 0.85. Mor-index was quantified automatically by our method. Survival analysis indicated that the higher Mor-index correlated with better OS in the discovery cohorts (HR for high vs. low 0.49, 95% CI 0.27-0.77, P = 0.0014) and validation cohort (0.21, 0.10-0.46, < 0.0001).

CONCLUSION

This study provided a novel AI-based approach to segmenting various nuclei in the TME. The Mor-index can reflect the immune status of CRC patients and is associated with favorable survival. Thus, Mor-index can potentially make a significant role in aiding clinical prognosis and decision-making.