Multi-tissue and multi-scale approach for nuclei segmentation in H&E stained images

The need for clinical, genetic and radiological characterization of atypical polycystic kidney disease

BACKGROUND

Autosomal Dominant Polycystic Kidney Disease (ADPKD) is a monogenic disease having a prevalence of 1:400-1000 live births. Depending on kidney imaging, patients can be subdivided into Class 1 (typical) and Class 2 (atypical). The present study aims to provide better assessment of Class 2 patients to help define their family history, together with their clinical and radiological characteristics.

METHODS

One hundred twenty-four PKD patients with abdominal Magnetic Resonance Imaging (MRI) for the staging of ADPKD, were retrospectively analyzed, aiming to focus on Class 2 ADPKD patients. Total kidney volume and total cyst volume were evaluated, while also assessing their clinical and genetic characteristics.

RESULTS

Twelve patients fulfilled the Mayo criteria for Class 2 ADPKD (two Class 2B and ten Class 2A). Extrarenal involvement was observed in 66.7% of cases, but only two subjects presented an estimated Glomerular Filtration Rate (eGFR) < 60 mL/min/1.73 m2. A positive family history for cystic disease was more frequent compared to other published cohorts. Only 8.3% tested positive for a likely pathogenic mutation in the PKD1 gene. Class 2B patients showed a lower height-adjusted total kidney volume, with a lower percentage of total cyst volume.

CONCLUSION

Based on our results, atypical ADPKD does not represent an uncommon condition, being present in about 10% of MRI-evaluated patients diagnosed with ADPKD. Genetic tests are frequently negative for PKD1/PKD2, and total cyst volume and residual tissue volume do not increase the prognostic value of MRI in patients with these radiological characteristics. Other tools are needed to better characterize their kidney prognosis.

Data-driven nucleus subclassification on colon hematoxylin and eosin using style-transferred digital pathology

Purpose

Cells are building blocks for human physiology; consequently, understanding the way cells communicate, co-locate, and interrelate is essential to furthering our understanding of how the body functions in both health and disease. Hematoxylin and eosin (H&E) is the standard stain used in histological analysis of tissues in both clinical and research settings. Although H&E is ubiquitous and reveals tissue microanatomy, the classification and mapping of cell subtypes often require the use of specialized stains. The recent CoNIC Challenge focused on artificial intelligence classification of six types of cells on colon H&E but was unable to classify epithelial subtypes (progenitor, enteroendocrine, goblet), lymphocyte subtypes (B, helper T, cytotoxic T), and connective subtypes (fibroblasts). We propose to use inter-modality learning to label previously un-labelable cell types on H&E.

Approach

We took advantage of the cell classification information inherent in multiplexed immunofluorescence (MxIF) histology to create cell-level annotations for 14 subclasses. Then, we performed style transfer on the MxIF to synthesize realistic virtual H&E. We assessed the efficacy of a supervised learning scheme using the virtual H&E and 14 subclass labels. We evaluated our model on virtual H&E and real H&E.

Results

On virtual H&E, we were able to classify helper T cells and epithelial progenitors with positive predictive values of 0.34 ± 0.15 (prevalence 0.03 ± 0.01 ) and 0.47 ± 0.1 (prevalence 0.07 ± 0.02 ), respectively, when using ground truth centroid information. On real H&E, we needed to compute bounded metrics instead of direct metrics because our fine-grained virtual H&E predicted classes had to be matched to the closest available parent classes in the coarser labels from the real H&E dataset. For the real H&E, we could classify bounded metrics for the helper T cells and epithelial progenitors with upper bound positive predictive values of 0.43 ± 0.03 (parent class prevalence 0.21) and 0.94 ± 0.02 (parent class prevalence 0.49) when using ground truth centroid information.

Conclusions

This is the first work to provide cell type classification for helper T and epithelial progenitor nuclei on H&E.

Optimized detection and segmentation of nuclei in gastric cancer images using stain normalization and blurred artifact removal

Histological analysis with microscopy is the gold standard to diagnose and stage cancer, where slides or whole slide images are analyzed for cell morphological and spatial features by pathologists. The nuclei of cancerous cells are characterized by nonuniform chromatin distribution, irregular shapes, and varying size. As nucleus area and shape alone carry prognostic value, detection and segmentation of nuclei are among the most important steps in disease grading. However, evaluation of nuclei is a laborious, time-consuming, and subjective process with large variation among pathologists. Recent advances in digital pathology have allowed significant applications in nuclei detection, segmentation, and classification, but automated image analysis is greatly affected by staining factors, scanner variability, and imaging artifacts, requiring robust image preprocessing, normalization, and segmentation methods for clinically satisfactory results. In this paper, we aimed to evaluate and compare the digital image analysis techniques used in clinical pathology and research in the setting of gastric cancer. A literature review was conducted to evaluate potential methods of improving nuclei detection. Digitized images of 35 patients from a retrospective cohort of gastric adenocarcinoma at Oulu University Hospital in 1987-2016 were annotated for nuclei (n = 9085) by expert pathologists and 14 images of different cancer types from public TCGA dataset with annotated nuclei (n = 7000) were used as a comparison to evaluate applicability in other cancer types. The detection and segmentation accuracy with the selected color normalization and stain separation techniques were compared between the methods. The extracted information can be supplemented by patient's medical data and fed to the existing statistical clinical tools or subjected to subsequent AI-assisted classification and prediction models. The performance of each method is evaluated by several metrics against the annotations done by expert pathologists. The F1-measure of 0.854 ± 0.068 is achieved with color normalization for the gastric cancer dataset, and 0.907 ± 0.044 with color deconvolution for the public dataset, showing comparable results to the earlier state-of-the-art works. The developed techniques serve as a basis for further research on application and interpretability of AI-assisted tools for gastric cancer diagnosis.

A pathomic approach for tumor-infiltrating lymphocytes classification on breast cancer digital pathology images

Background and objectives

The detection of tumor-infiltrating lymphocytes (TILs) could aid in the development of objective measures of the infiltration grade and can support decision-making in breast cancer (BC). However, manual quantification of TILs in BC histopathological whole slide images (WSI) is currently based on a visual assessment, thus resulting not standardized, not reproducible, and time-consuming for pathologists. In this work, a novel pathomic approach, aimed to apply high-throughput image feature extraction techniques to analyze the microscopic patterns in WSI, is proposed. In fact, pathomic features provide additional information concerning the underlying biological processes compared to the WSI visual interpretation, thus providing more easily interpretable and explainable results than the most frequently investigated Deep Learning based methods in the literature.

Methods

A dataset containing 1037 regions of interest with tissue compartments and TILs annotated on 195 TNBC and HER2+ BC hematoxylin and eosin (H&E)-stained WSI was used. After segmenting nuclei within tumor-associated stroma using a watershed-based approach, 71 pathomic features were extracted from each nucleus and reduced using a Spearman's correlation filter followed by a nonparametric Wilcoxon rank-sum test and least absolute shrinkage and selection operator. The relevant features were used to classify each candidate nucleus as either TILs or non-TILs using 5 multivariable machine learning classification models trained using 5-fold cross-validation (1) without resampling, (2) with the synthetic minority over-sampling technique and (3) with downsampling. The prediction performance of the models was assessed using ROC curves.

Results

21 features were selected, with most of them related to the well-known TILs properties of having regular shape, clearer margins, high peak intensity, more homogeneous enhancement and different textural pattern than other cells. The best performance was obtained by Random-Forest with ROC AUC of 0.86, regardless of resampling technique.

Conclusions

The presented approach holds promise for the classification of TILs in BC H&E-stained WSI and could provide support to pathologists for a reliable, rapid and interpretable clinical assessment of TILs in BC.

A review: The detection of cancer cells in histopathology based on machine vision

Machine vision is being employed in defect detection, size measurement, pattern recognition, image fusion, target tracking and 3D reconstruction. Traditional cancer detection methods are dominated by manual detection, which wastes time and manpower, and heavily relies on the pathologists' skill and work experience. Therefore, these manual detection approaches are not convenient for the inheritance of domain knowledge, and are not suitable for the rapid development of medical care in the future. The emergence of machine vision can iteratively update and learn the domain knowledge of cancer cell pathology detection to achieve automated, high-precision, and consistent detection. Consequently, this paper reviews the use of machine vision to detect cancer cells in histopathology images, as well as the benefits and drawbacks of various detection approaches. First, we review the application of image preprocessing and image segmentation in histopathology for the detection of cancer cells, and compare the benefits and drawbacks of different algorithms. Secondly, for the characteristics of histopathological cancer cell images, the research progress of shape, color and texture features and other methods is mainly reviewed. Furthermore, for the classification methods of histopathological cancer cell images, the benefits and drawbacks of traditional machine vision approaches and deep learning methods are compared and analyzed. Finally, the above research is discussed and forecasted, with the expected future development tendency serving as a guide for future research.

Reduction of Cardiac Fibrosis by Interference With YAP-Dependent Transactivation

BACKGROUND

Conversion of cardiac stromal cells into myofibroblasts is typically associated with hypoxia conditions, metabolic insults, and/or inflammation, all of which are predisposing factors to cardiac fibrosis and heart failure. We hypothesized that this conversion could be also mediated by response of these cells to mechanical cues through activation of the Hippo transcriptional pathway. The objective of the present study was to assess the role of cellular/nuclear straining forces acting in myofibroblast differentiation of cardiac stromal cells under the control of YAP (yes-associated protein) transcription factor and to validate this finding using a pharmacological agent that interferes with the interactions of the YAP/TAZ (transcriptional coactivator with PDZ-binding motif) complex with their cognate transcription factors TEADs (TEA domain transcription factors), under high-strain and profibrotic stimulation.

METHODS

We employed high content imaging, 2-dimensional/3-dimensional culture, atomic force microscopy mapping, and molecular methods to prove the role of cell/nuclear straining in YAP-dependent fibrotic programming in a mouse model of ischemia-dependent cardiac fibrosis and in human-derived primitive cardiac stromal cells. We also tested treatment of cells with Verteporfin, a drug known to prevent the association of the YAP/TAZ complex with their cognate transcription factors TEADs.

RESULTS

Our experiments suggested that pharmacologically targeting the YAP-dependent pathway overrides the profibrotic activation of cardiac stromal cells by mechanical cues in vitro, and that this occurs even in the presence of profibrotic signaling mediated by TGF-β1 (transforming growth factor beta-1). In vivo administration of Verteporfin in mice with permanent cardiac ischemia reduced significantly fibrosis and morphometric remodeling but did not improve cardiac performance.

CONCLUSIONS

Our study indicates that preventing molecular translation of mechanical cues in cardiac stromal cells reduces the impact of cardiac maladaptive remodeling with a positive effect on fibrosis.

Role of tumor infiltrating lymphocytes and spatial immune heterogeneity in sensitivity to PD-1 axis blockers in non-small cell lung cancer

BACKGROUND

Tumor infiltrating lymphocytes (TILs) reflect adaptive antitumor immune responses in cancer and are generally associated with favorable prognosis. However, the relationships between TILs subsets and their spatial arrangement with clinical benefit from immune checkpoint inhibitors (ICI) in non-small cell lung cancer (NSCLC) remains less explored.

METHODS

We used multiplexed quantitative immunofluorescence panels to determine the association of major TILs subpopulations, CD8+ cytotoxic T cells, CD4+ helper T cells and CD20+ B cells, and T cell exhaustion markers, programmed cell death protein-1 (PD-1),lymphocyte-activation gene 3 (LAG-3) and T cell immunoglobulin mucin-3 (TIM-3) with outcomes in a multi-institutional cohort of baseline tumor samples from 179 patients with NSCLC treated with ICI. The analysis of full-face tumor biopsies including numerous fields of view allowed a detailed spatial analysis and assessment of tumor immune heterogeneity using a multiparametric quadratic entropy metric (Rao's Q Index (RQI)).

RESULTS

TILs were preferentially located in the stromal tissue areas surrounding tumor-cell nests and CD8+ T cells were the most abundant subset. Higher density of stromal CD8+ cytotoxic T cells was significantly associated with longer survival, and this effect was more prominent in programmed death ligand-1 (PD-L1) positive cases. The role of baseline T cell infiltration to stratify PD-L1 expressing cases was confirmed measuring the T cell receptor-burden in an independent NSCLC cohort studied with whole-exome DNA sequencing. High levels of LAG-3 on T cells or elevated RQI heterogeneity index were associated with worse survival in the cohort.

CONCLUSION

Baseline T cell density and T cell exhaustion marker expression can stratify outcomes in PD-L1 positive patients with NSCLC treated with ICI. Spatial immune heterogeneity can be measured using the RQI and is associated with survival in NSCLC.

Automated Cell Foreground–Background Segmentation with Phase-Contrast Microscopy Images: An Alternative to Machine Learning Segmentation Methods with Small-Scale Data

Cell segmentation is a critical step for image-based experimental analysis. Existing cell segmentation methods are neither entirely automated nor perform well under basic laboratory microscopy. This study proposes an efficient and automated cell segmentation method involving morphological operations to automatically achieve cell segmentation for phase-contrast microscopes. Manual/visual counting of cell segmentation serves as the control group (156 images as ground truth) to evaluate the proposed method’s performance. The proposed technology’s adaptive performance is assessed at varying conditions, including artificial blurriness, illumination, and image size. Compared to the Trainable Weka Segmentation method, the Empirical Gradient Threshold method, and the ilastik segmentation software, the proposed method achieved better segmentation accuracy (dice coefficient: 90.07, IoU: 82.16%, and 6.51% as the average relative error on measuring cell area). The proposed method also has good reliability, even under unfavored imaging conditions at which manual labeling or human intervention is inefficient. Additionally, similar degrees of segmentation accuracy were confirmed when the ground truth data and the generated data from the proposed method were applied individually to train modified U-Net models (16848 images). These results demonstrated good accuracy and high practicality of the proposed cell segmentation method with phase-contrast microscopy image data.

Nuclei Segmentation on Histopathology Images of Breast Carcinoma

With the use of computer-aided diagnostic systems, the automatic detection and segmentation of the cell nuclei have become essential in pathology due to cellular nuclei counting and nuclear pleomorphism analysis are critical for the classification and grading of breast cancer histopathology. This work describes a methodology for automatic detection and segmentation of cellular nuclei in breast cancer histopathology images obtained from the BreakHis database, the Standford tissue microarray database, and the Breast Cancer Cell Segmentation database. The proposed scheme is based on the characterization of Hematoxylin and Eosin (H&E) staining, size, and shape features. In addition, we use the information obtained from morphological transformations and adaptive intensity adjustments to detect and separate each cell nucleus detected in the image. The segmentation was carried out by testing the proposed methodology in a histological breast cancer database that provides the associated groundtruth segmentation. Subsequently, the Sørensen-Dice similarity coefficient was calculated to analyze the suitability of the results.Clinical relevance- In this work, the detection and segmentation of cell nuclei in breast cancer histological images are carried out automatically. The method can identify cell nuclei regardless of variations in the level of staining and image magnification. Moreover, a granulometric analysis of the components allows identifying cell clumps and segment them into individual cell nuclei. Improved identification of cell nuclei under different image conditions was demonstrated to reach a sensitivity average of 0.76 ± 0.12. The results provide a base for further and complex processes such as cell counting, feature analysis, and nuclear pleomorphism, which are relevant tasks in the evaluation and diagnostic performed by the expert pathologist.

Local Integral Regression Network for Cell Nuclei Detection

Nuclei detection is a fundamental task in the field of histopathology image analysis and remains challenging due to cellular heterogeneity. Recent studies explore convolutional neural networks to either isolate them with sophisticated boundaries (segmentation-based methods) or locate the centroids of the nuclei (counting-based approaches). Although these two methods have demonstrated superior success, their fully supervised training demands considerable and laborious pixel-wise annotations manually labeled by pathology experts. To alleviate such tedious effort and reduce the annotation cost, we propose a novel local integral regression network (LIRNet) that allows both fully and weakly supervised learning (FSL/WSL) frameworks for nuclei detection. Furthermore, the LIRNet can output an exquisite density map of nuclei, in which the localization of each nucleus is barely affected by the post-processing algorithms. The quantitative experimental results demonstrate that the FSL version of the LIRNet achieves a state-of-the-art performance compared to other counterparts. In addition, the WSL version has exhibited a competitive detection performance and an effortless data annotation that requires only 17.5% of the annotation effort.

Multi_Scale_Tools: A Python Library to Exploit Multi-Scale Whole Slide Images

Algorithms proposed in computational pathology can allow to automatically analyze digitized tissue samples of histopathological images to help diagnosing diseases. Tissue samples are scanned at a high-resolution and usually saved as images with several magnification levels, namely whole slide images (WSIs). Convolutional neural networks (CNNs) represent the state-of-the-art computer vision methods targeting the analysis of histopathology images, aiming for detection, classification and segmentation. However, the development of CNNs that work with multi-scale images such as WSIs is still an open challenge. The image characteristics and the CNN properties impose architecture designs that are not trivial. Therefore, single scale CNN architectures are still often used. This paper presents Multi_Scale_Tools, a library aiming to facilitate exploiting the multi-scale structure of WSIs. Multi_Scale_Tools currently include four components: a pre-processing component, a scale detector, a multi-scale CNN for classification and a multi-scale CNN for segmentation of the images. The pre-processing component includes methods to extract patches at several magnification levels. The scale detector allows to identify the magnification level of images that do not contain this information, such as images from the scientific literature. The multi-scale CNNs are trained combining features and predictions that originate from different magnification levels. The components are developed using private datasets, including colon and breast cancer tissue samples. They are tested on private and public external data sources, such as The Cancer Genome Atlas (TCGA). The results of the library demonstrate its effectiveness and applicability. The scale detector accurately predicts multiple levels of image magnification and generalizes well to independent external data. The multi-scale CNNs outperform the single-magnification CNN for both classification and segmentation tasks. The code is developed in Python and it will be made publicly available upon publication. It aims to be easy to use and easy to be improved with additional functions.

Deep Learning Enables Individual Xenograft Cell Classification in Histological Images by Analysis of Contextual Features

Patient-Derived Xenografts (PDXs) are the preclinical models which best recapitulate inter- and intra-patient complexity of human breast malignancies, and are also emerging as useful tools to study the normal breast epithelium. However, data analysis generated with such models is often confounded by the presence of host cells and can give rise to data misinterpretation. For instance, it is important to discriminate between xenografted and host cells in histological sections prior to performing immunostainings. We developed Single Cell Classifier (SCC), a data-driven deep learning-based computational tool that provides an innovative approach for automated cell species discrimination based on a multi-step process entailing nuclei segmentation and single cell classification. We show that human and murine cell contextual features, more than cell-intrinsic ones, can be exploited to discriminate between cell species in both normal and malignant tissues, yielding up to 96% classification accuracy. SCC will facilitate the interpretation of H&E- and DAPI-stained histological sections of xenografted human-in-mouse tissues and it is open to new in-house built models for further applications. SCC is released as an open-source plugin in ImageJ/Fiji available at the following link: https://github.com/Biomedical-Imaging-Group/SingleCellClassifier .

Ex Vivo MR Histology and Cytometric Feature Mapping Connect Three-dimensional in Vivo MR Images to Two-dimensional Histopathologic Images of Murine Sarcomas

Purpose To establish a platform for quantitative tissue-based interpretation of cytoarchitecture features from tumor MRI measurements. Materials and Methods In a pilot preclinical study, multicontrast in vivo MRI of murine soft-tissue sarcomas in 10 mice, followed by ex vivo MRI of fixed tissues (termed MR histology), was performed. Paraffin-embedded limb cross-sections were stained with hematoxylin-eosin, digitized, and registered with MRI. Registration was assessed by using binarized tumor maps and Dice similarity coefficients (DSCs). Quantitative cytometric feature maps from histologic slides were derived by using nuclear segmentation and compared with registered MRI, including apparent diffusion coefficients and transverse relaxation times as affected by magnetic field heterogeneity (T2* maps). Cytometric features were compared with each MR image individually by using simple linear regression analysis to identify the features of interest, and the goodness of fit was assessed on the basis of R² values. Results Registration of MR images to histopathologic slide images resulted in mean DSCs of 0.912 for ex vivo MR histology and 0.881 for in vivo MRI. Triplicate repeats showed high registration repeatability (mean DSC, >0.9). Whole-slide nuclear segmentations were automated to detect nuclei on histopathologic slides (DSC = 0.8), and feature maps were generated for correlative analysis with MR images. Notable trends were observed between cell density and in vivo apparent diffusion coefficients (best line fit: R² = 0.96, P < .001). Multiple cytoarchitectural features exhibited linear relationships with in vivo T2* maps, including nuclear circularity (best line fit: R² = 0.99, P < .001) and variance in nuclear circularity (best line fit: R² = 0.98, P < .001). Conclusion An infrastructure for registering and quantitatively comparing in vivo tumor MRI with traditional histologic analysis was successfully implemented in a preclinical pilot study of soft-tissue sarcomas. Keywords: MRI, Pathology, Animal Studies, Tissue Characterization Supplemental material is available for this article. © RSNA, 2021.

A hybrid deep learning approach for gland segmentation in prostate histopathological images

BACKGROUND

In digital pathology, the morphology and architecture of prostate glands have been routinely adopted by pathologists to evaluate the presence of cancer tissue. The manual annotations are operator-dependent, error-prone and time-consuming. The automated segmentation of prostate glands can be very challenging too due to large appearance variation and serious degeneration of these histological structures.

METHOD

A new image segmentation method, called RINGS (Rapid IdentificatioN of Glandural Structures), is presented to segment prostate glands in histopathological images. We designed a novel glands segmentation strategy using a multi-channel algorithm that exploits and fuses both traditional and deep learning techniques. Specifically, the proposed approach employs a hybrid segmentation strategy based on stroma detection to accurately detect and delineate the prostate glands contours.

RESULTS

Automated results are compared with manual annotations and seven state-of-the-art techniques designed for glands segmentation. Being based on stroma segmentation, no performance degradation is observed when segmenting healthy or pathological structures. Our method is able to delineate the prostate gland of the unknown histopathological image with a dice score of 90.16 % and outperforms all the compared state-of-the-art methods.

CONCLUSIONS

To the best of our knowledge, the RINGS algorithm is the first fully automated method capable of maintaining a high sensitivity even in the presence of severe glandular degeneration. The proposed method will help to detect the prostate glands accurately and assist the pathologists to make accurate diagnosis and treatment. The developed model can be used to support prostate cancer diagnosis in polyclinics and community care centres.

Multilevel thresholding using a modified ant lion optimizer with opposition-based learning for color image segmentation

Multilevel thresholding has important research value in image segmentation and can effectively solve region analysis problems of complex images. In this paper, Otsu and Kapur's entropy are adopted among thresholding segmentation methods. They are used as the objective functions. When the number of threshold increases, the time complexity increases exponentially. In order to overcome this drawback, a modified ant lion optimizer algorithm based on opposition-based learning (MALO) is proposed to determine the optimum threshold values by the maximization of the objective functions. By introducing the opposition-based learning strategy, the search accuracy and convergence performance are increased. In addition to IEEE CEC 2017 benchmark functions validation, 11 state-of-the-art algorithms are selected for comparison. A series of experiments are conducted to evaluate the segmentation performance of the algorithm. The evaluation metrics include: fitness value, peak signal-to-noise ratio, structural similarity index, feature similarity index, and computational time. The experimental data are analyzed and discussed in details. The experimental results significantly demonstrate that the proposed method is superior over others, which can be considered as a powerful and efficient thresholding technique.

Histopathological Classification of Canine Cutaneous Round Cell Tumors Using Deep Learning: A Multi-Center Study

Canine cutaneous round cell tumors (RCT) represent one of the routine diagnostic challenges for veterinary pathologists. Computer-aided approaches are developed to overcome these restrictions and to increase accuracy and consistency of diagnosis. These systems are also of high benefit reducing errors when a large number of cases are screened daily. In this study we describe ARCTA (Automated Round Cell Tumors Assessment), a fully automated algorithm for cutaneous RCT classification and mast cell tumors grading in canine histopathological images. ARCTA employs a deep learning strategy and was developed on 416 RCT images and 213 mast cell tumors images. In the test set, our algorithm exhibited an excellent classification performance in both RCT classification (accuracy: 91.66%) and mast cell tumors grading (accuracy: 100%). Misdiagnoses were encountered for histiocytomas in the train set and for melanomas in the test set. For mast cell tumors the reduction of a grade was observed in the train set, but not in the test set. To the best of our knowledge, the proposed model is the first fully automated algorithm in histological images specifically developed for veterinary medicine. Being very fast (average computational time 2.63 s), this algorithm paves the way for an automated and effective evaluation of canine tumors.

Deep Adversarial Training for Multi-Organ Nuclei Segmentation in Histopathology Images

Nuclei mymargin segmentation is a fundamental task for various computational pathology applications including nuclei morphology analysis, cell type classification, and cancer grading. Deep learning has emerged as a powerful approach to segmenting nuclei but the accuracy of convolutional neural networks (CNNs) depends on the volume and the quality of labeled histopathology data for training. In particular, conventional CNN-based approaches lack structured prediction capabilities, which are required to distinguish overlapping and clumped nuclei. Here, we present an approach to nuclei segmentation that overcomes these challenges by utilizing a conditional generative adversarial network (cGAN) trained with synthetic and real data. We generate a large dataset of H&E training images with perfect nuclei segmentation labels using an unpaired GAN framework. This synthetic data along with real histopathology data from six different organs are used to train a conditional GAN with spectral normalization and gradient penalty for nuclei segmentation. This adversarial regression framework enforces higher-order spacial-consistency when compared to conventional CNN models. We demonstrate that this nuclei segmentation approach generalizes across different organs, sites, patients and disease states, and outperforms conventional approaches, especially in isolating individual and overlapping nuclei.

Karpinski Score under Digital Investigation: A Fully Automated Segmentation Algorithm to Identify Vascular and Stromal Injury of Donors’ Kidneys

In kidney transplantations, the evaluation of the vascular structures and stromal areas is crucial for determining kidney acceptance, which is currently based on the pathologist’s visual evaluation. In this context, an accurate assessment of the vascular and stromal injury is fundamental to assessing the nephron status. In the present paper, the authors present a fully automated algorithm, called RENFAST (Rapid EvaluatioN of Fibrosis And vesselS Thickness), for the segmentation of kidney blood vessels and fibrosis in histopathological images. The proposed method employs a novel strategy based on deep learning to accurately segment blood vessels, while interstitial fibrosis is assessed using an adaptive stain separation method. The RENFAST algorithm is developed and tested on 350 periodic acid–Schiff (PAS) images for blood vessel segmentation and on 300 Massone’s trichrome (TRIC) stained images for the detection of renal fibrosis. In the TEST set, the algorithm exhibits excellent segmentation performance in both blood vessels (accuracy: 0.8936) and fibrosis (accuracy: 0.9227) and outperforms all the compared methods. To the best of our knowledge, the RENFAST algorithm is the first fully automated method capable of detecting both blood vessels and fibrosis in digital histological images. Being very fast (average computational time 2.91 s), this algorithm paves the way for automated, quantitative, and real-time kidney graft assessments.

Stain Color Adaptive Normalization (SCAN) algorithm: Separation and standardization of histological stains in digital pathology

BACKGROUND AND OBJECTIVE

The diagnosis of histopathological images is based on the visual analysis of tissue slices under a light microscope. However, the histological tissue appearance may assume different color intensities depending on the staining process, operator ability and scanner specifications. This stain variability affects the diagnosis of the pathologist and decreases the accuracy of computer-aided diagnosis systems. In this context, the stain normalization process has proved to be a powerful tool to cope with this issue, allowing to standardize the stain color appearance of a source image respect to a reference image.

METHODS

In this paper, novel fully automated stain separation and normalization approaches for hematoxylin and eosin stained histological slides are presented. The proposed algorithm, named SCAN (Stain Color Adaptive Normalization), is based on segmentation and clustering strategies for cellular structures detection. The SCAN algorithm is able to improve the contrast between histological tissue and background and preserve local structures without changing the color of the lumen and the background.

RESULTS

Both stain separation and normalization techniques were qualitatively and quantitively validated on a multi-tissue and multiscale dataset, with highly satisfactory results, outperforming the state-of-the-art approaches. SCAN was also tested on whole-slide images with high performances and low computational times.

CONCLUSIONS

The potential contribution of the proposed standardization approach is twofold: the improvement of visual diagnosis in digital histopathology and the development of powerful pre-processing strategies to automated classification techniques for cancer detection.

Fully automated quantitative assessment of hepatic steatosis in liver transplants

BACKGROUND

The presence of macro- and microvesicular steatosis is one of the major risk factors for liver transplantation. An accurate assessment of the steatosis percentage is crucial for determining liver graft transplantability, which is currently based on the pathologists' visual evaluations on liver histology specimens.

METHOD

The aim of this study was to develop and validate a fully automated algorithm, called HEPASS (HEPatic Adaptive Steatosis Segmentation), for both micro- and macro-steatosis detection in digital liver histological images. The proposed method employs a hybrid deep learning framework, combining the accuracy of an adaptive threshold with the semantic segmentation of a deep convolutional neural network. Starting from all white regions, the HEPASS algorithm was able to detect lipid droplets and classify them into micro- or macrosteatosis.

RESULTS

The proposed method was developed and tested on 385 hematoxylin and eosin (H&E) stained images coming from 77 liver donors. Automated results were compared with manual annotations and nine state-of-the-art techniques designed for steatosis segmentation. In the TEST set, the algorithm was characterized by 97.27% accuracy in steatosis quantification (average error 1.07%, maximum average error 5.62%) and outperformed all the compared methods.

CONCLUSIONS

To the best of our knowledge, the proposed algorithm is the first fully automated algorithm for the assessment of both micro- and macrosteatosis in H&E stained liver tissue images. Being very fast (average computational time 0.72 s), this algorithm paves the way for automated, quantitative and real-time liver graft assessments.

Enhancing Multi-tissue and Multi-scale Cell Nuclei Segmentation with Deep Metric Learning

(1) Background: The segmentation of cell nuclei is an essential task in a wide range of biomedical studies and clinical practices. The full automation of this process remains a challenge due to intra- and internuclear variations across a wide range of tissue morphologies, differences in staining protocols and imaging procedures. (2) Methods: A deep learning model with metric embeddings such as contrastive loss and triplet loss with semi-hard negative mining is proposed in order to accurately segment cell nuclei in a diverse set of microscopy images. The effectiveness of the proposed model was tested on a large-scale multi-tissue collection of microscopy image sets. (3) Results: The use of deep metric learning increased the overall segmentation prediction by 3.12% in the average value of Dice similarity coefficients as compared to no metric learning. In particular, the largest gain was observed for segmenting cell nuclei in H&E -stained images when deep learning network and triplet loss with semi-hard negative mining were considered for the task. (4) Conclusion: We conclude that deep metric learning gives an additional boost to the overall learning process and consequently improves the segmentation performance. Notably, the improvement ranges approximately between 0.13% and 22.31% for different types of images in the terms of Dice coefficients when compared to no metric deep learning.

Intraoperative assessment of skull base tumors using stimulated Raman scattering microscopy

Intraoperative consultations, used to guide tumor resection, can present histopathological findings that are challenging to interpret due to artefacts from tissue cryosectioning and conventional staining. Stimulated Raman histology (SRH), a label-free imaging technique for unprocessed biospecimens, has demonstrated promise in a limited subset of tumors. Here, we target unexplored skull base tumors using a fast simultaneous two-channel stimulated Raman scattering (SRS) imaging technique and a new pseudo-hematoxylin and eosin (H&E) recoloring methodology. To quantitatively evaluate the efficacy of our approach, we use modularized assessment of diagnostic accuracy beyond cancer/non-cancer determination and neuropathologist confidence for SRH images contrasted to H&E-stained frozen and formalin-fixed paraffin-embedded (FFPE) tissue sections. Our results reveal that SRH is effective for establishing a diagnosis using fresh tissue in most cases with 87% accuracy relative to H&E-stained FFPE sections. Further analysis of discrepant case interpretation suggests that pseudo-H&E recoloring underutilizes the rich chemical information offered by SRS imaging, and an improved diagnosis can be achieved if full SRS information is used. In summary, our findings show that pseudo-H&E recolored SRS images in combination with lipid and protein chemical information can maximize the use of SRS during intraoperative pathologic consultation with implications for tissue preservation and augmented diagnostic utility.

Automatic skin lesion area determination of basal cell carcinoma using optical coherence tomography angiography and a skeletonization approach: Preliminary results

Cutaneous blood flow plays a key role in numerous physiological and pathological processes and has significant potential to be used as a biomarker to diagnose skin diseases such as basal cell carcinoma (BCC). The determination of the lesion area and vascular parameters within it, such as vessel density, is essential for diagnosis, surgical treatment and follow-up procedures. Here, an automatic skin lesion area determination algorithm based on optical coherence tomography angiography (OCTA) images is presented for the first time. The blood vessels are segmented within the OCTA images and then skeletonized. Subsequently, the skeleton is searched over the volume and numerous quantitative vascular parameters are calculated. The vascular density is then used to segment the lesion area. The algorithm is tested on both nodular and superficial BCC, and comparing with dermatological and histological results, the proposed method provides an accurate, non-invasive, quantitative and automatic tool for BCC lesion area determination.

Automated segmentation of brain cells for clonal analyses in fluorescence microscopy images

The understanding of how cell diversity within and across distinct brain regions is ontogenetically achieved is a pivotal topic in neuroscience. Clonal analyses based on multicolor cell labeling represent a powerful tool to tackle this issue and disclose lineage relationships, but produce enormous sets of fluorescence images, leading to time consuming analyses that may be biased by the operator's subjectivity. Thus, time-efficient automated software are needed to analyze images easily, accurately and without subjective bias. In this paper, we present a fully automated method, named FAST ('Fluorescent cell Analysis Segmentation Tool'), for the segmentation of neural cells labeled by multicolor combinations of fluorophores and for their classification into clones. The proposed method was tested on 77 high-magnification fluorescence images of adult mouse cerebellar tissues acquired using a confocal microscope. Automatic results were compared with manual annotations and two open-source software designed for cell detection in microscopic imaging. The algorithm showed very good performance in the cellular detection and in the assignment of the clonal identity. To the best of our knowledge, FAST is the first fully automated technique for the analysis of cellular clones based on combinatorial expression of fluorescent proteins. The proposed approach allows to perform clonal analyses easily, accurately and objectively, overcoming those biases and errors that may result from manual annotations. Moreover, it can be broadly applied to the quantification and colocalization within cells of fluorescent markers, therefore representing a versatile and powerful tool for automated quantitative analyses in fluorescence microscopy.

Multimodal T2w and DWI Prostate Gland Automated Registration

Multiparametric magnetic resonance imaging (mpMRI) is emerging as a promising tool in the clinical pathway of prostate cancer (PCa). The registration between a structural and a functional imaging modality, such as T2-weighted (T2w) and diffusion-weighted imaging (DWI) is fundamental in the development of a mpMRI-based computer aided diagnosis (CAD) system for PCa. Here, we propose an automated method to register the prostate gland in T2w and DWI image sequences by a landmark-based affine registration and a non-parametric diffeomorphic registration. An expert operator manually segmented the prostate gland in both modalities on a dataset of 20 patients. Target registration error and Jaccard index, which measures the overlap between masks, were evaluated pre- and post- registration resulting in an improvement of 44% and 21%, respectively. In the future, the proposed method could be useful in the framework of a CAD system for PCa detection and characterization in mpMRI.

Self-Learning Microfluidic Platform for Single-Cell Imaging and Classification in Flow

Single-cell analysis commonly requires the confinement of cell suspensions in an analysis chamber or the precise positioning of single cells in small channels. Hydrodynamic flow focusing has been broadly utilized to achieve stream confinement in microchannels for such applications. As imaging flow cytometry gains popularity, the need for imaging-compatible microfluidic devices that allow for precise confinement of single cells in small volumes becomes increasingly important. At the same time, high-throughput single-cell imaging of cell populations produces vast amounts of complex data, which gives rise to the need for versatile algorithms for image analysis. In this work, we present a microfluidics-based platform for single-cell imaging in-flow and subsequent image analysis using variational autoencoders for unsupervised characterization of cellular mixtures. We use simple and robust Y-shaped microfluidic devices and demonstrate precise 3D particle confinement towards the microscope slide for high-resolution imaging. To demonstrate applicability, we use these devices to confine heterogeneous mixtures of yeast species, brightfield-image them in-flow and demonstrate fully unsupervised, as well as few-shot classification of single-cell images with 88% accuracy.

Automated Segmentation of Fluorescence Microscopy Images for 3D Cell Detection in human-derived Cardiospheres

The ‘cardiosphere’ is a 3D cluster of cardiac progenitor cells recapitulating a stem cell niche-like microenvironment with a potential for disease and regeneration modelling of the failing human myocardium. In this multicellular 3D context, it is extremely important to decrypt the spatial distribution of cell markers for dissecting the evolution of cellular phenotypes by direct quantification of fluorescent signals in confocal microscopy. In this study, we present a fully automated method, named CARE (‘CARdiosphere Evaluation’), for the segmentation of membranes and cell nuclei in human-derived cardiospheres. The proposed method is tested on twenty 3D-stacks of cardiospheres, for a total of 1160 images. Automatic results are compared with manual annotations and two open-source software designed for fluorescence microscopy. CARE performance was excellent in cardiospheres membrane segmentation and, in cell nuclei detection, the algorithm achieved the same performance as two expert operators. To the best of our knowledge, CARE is the first fully automated algorithm for segmentation inside in vitro 3D cell spheroids, including cardiospheres. The proposed approach will provide, in the future, automated quantitative analysis of markers distribution within the cardiac niche-like environment, enabling predictive associations between cell mechanical stresses and dynamic phenotypic changes.