Experimental Assessment of Color Deconvolution and Color Normalization for Automated Classification of Histology Images Stained with Hematoxylin and Eosin

Beneficial role of rosemary extract on oxidative stress-mediated neuronal apoptosis in rotenone-induced attention deficit hyperactivity disease in juvenile rat model

BACKGROUND AND AIM

Attention deficit hyperactivity disorder (ADHD) is heterogeneous neurobehavioral disorders that co-exist with cognitive and learning deficits affecting 3-7% of children. We study the role of rosemary in the protection of the prefrontal cortical neurons against rotenone-induced ADHD in juvenile rats.

METHODS

Twenty-four juvenile rats were divided into four groups (n=6): control group, received olive oil 0.5 ml/kg/day/ I.P. for 4 weeks, rosemary group received rosemary 75 mg/kg/day/ I.P. for 4 weeks, rotenone group received rotenone 1 mg/kg/day/ I.P. dissolved in olive oil for 4 days and combined group received rotenone 1 mg/kg/day/ I.P. for 4 days and rosemary 75 mg/kg/day/ I.P. for 4 weeks.

RESULTS

Rotenone group showed higher impulsivity with reduction in the recognition index and total locomotor activity. However, combined group showed significant improvement in the recognition index and the total locomotor activity. Neurochemical analysis disclosed that rotenone decreased levels of GSH and significantly increased lipid peroxidation and oxidative stress. The administration of rosemary amended these neurochemical changes. Rotenone caused a significant increase in serum amyloid protein A and C-reactive protein levels indicating a marked state of inflammation. Rosemary ameliorated these biochemical changes. The immunohistochemical expression of tyrosine hydroxylase was decreased in the rotenone group. On the other hand, caspase-3 was increased in the rotenone group. PCR confirmed immunohistochemical results for gene expression.

CONCLUSIONS

The findings of the behavioral, neurochemical, biochemical, immunohistochemical and molecular outcomes suggested that rosemary could fight oxidative stress, inflammation and apoptosis in the prefrontal cortex of rotenone-induced ADHD in juvenile rats.

Validation of a color deconvolution method to quantify MSC tri-lineage differentiation across species

Mesenchymal stem cells (MSCs) are a promising candidate for both human and veterinary regenerative medicine applications because of their abundance and ability to differentiate into several lineages. Mesenchymal stem cells are however a heterogeneous cell population and as such, it is imperative that they are unequivocally characterized to acquire reproducible results in clinical trials. Although the tri-lineage differentiation potential of MSCs is reported in most veterinary studies, a qualitative evaluation of representative histological images does not always unambiguously confirm tri-lineage differentiation. Moreover, potential differences in differentiation capacity are not identified. Therefore, quantification of tri-lineage differentiation would greatly enhance proper characterization of MSCs. In this study, a method to quantify the tri-lineage differentiation potential of MSCs is described using digital image analysis, based on the color deconvolution plug-in (ImageJ). Mesenchymal stem cells from three species, i.e., bovine, equine, and porcine, were differentiated toward adipocytes, chondrocytes, and osteocytes. Subsequently, differentiated MSCs were stained with Oil Red O, Alcian Blue, and Alizarin Red S, respectively. Next, a differentiation ratio (DR) was obtained by dividing the area % of the differentiation signal by the area % of the nuclear signal. Although MSCs isolated from all donors in all species were capable of tri-lineage differentiation, differences were demonstrated between donors using this quantitative DR. Our straightforward, simple but robust method represents an elegant approach to determine the degree of MSC tri-lineage differentiation across species. As such, differences in differentiation potential within the heterogeneous MSC population and between different MSC sources can easily be identified, which will support further optimization of regenerative therapies.

Multiclass classification of breast cancer histopathology images using multilevel features of deep convolutional neural network

Breast cancer is a common malignancy and a leading cause of cancer-related deaths in women worldwide. Its early diagnosis can significantly reduce the morbidity and mortality rates in women. To this end, histopathological diagnosis is usually followed as the gold standard approach. However, this process is tedious, labor-intensive, and may be subject to inter-reader variability. Accordingly, an automatic diagnostic system can assist to improve the quality of diagnosis. This paper presents a deep learning approach to automatically classify hematoxylin-eosin-stained breast cancer microscopy images into normal tissue, benign lesion, in situ carcinoma, and invasive carcinoma using our collected dataset. Our proposed model exploited six intermediate layers of the Xception (Extreme Inception) network to retrieve robust and abstract features from input images. First, we optimized the proposed model on the original (unnormalized) dataset using 5-fold cross-validation. Then, we investigated its performance on four normalized datasets resulting from Reinhard, Ruifrok, Macenko, and Vahadane stain normalization. For original images, our proposed framework yielded an accuracy of 98% along with a kappa score of 0.969. Also, it achieved an average AUC-ROC score of 0.998 as well as a mean AUC-PR value of 0.995. Specifically, for in situ carcinoma and invasive carcinoma, it offered sensitivity of 96% and 99%, respectively. For normalized images, the proposed architecture performed better for Makenko normalization compared to the other three techniques. In this case, the proposed model achieved an accuracy of 97.79% together with a kappa score of 0.965. Also, it attained an average AUC-ROC score of 0.997 and a mean AUC-PR value of 0.991. Especially, for in situ carcinoma and invasive carcinoma, it offered sensitivity of 96% and 99%, respectively. These results demonstrate that our proposed model outperformed the baseline AlexNet as well as state-of-the-art VGG16, VGG19, Inception-v3, and Xception models with their default settings. Furthermore, it can be inferred that although stain normalization techniques offered competitive performance, they could not surpass the results of the original dataset.

H&E Multi-Laboratory Staining Variance Exploration with Machine Learning

In diagnostic histopathology, hematoxylin and eosin (H&E) staining is a critical process that highlights salient histological features. Staining results vary between laboratories regardless of the histopathological task, although the method does not change. This variance can impair the accuracy of algorithms and histopathologists’ time-to-insight. Investigating this variance can help calibrate stain normalization tasks to reverse this negative potential. With machine learning, this study evaluated the staining variance between different laboratories on three tissue types. We received H&E-stained slides from 66 different laboratories. Each slide contained kidney, skin, and colon tissue samples stained by the method routinely used in each laboratory. The samples were digitized and summarized as red, green, and blue channel histograms. Dimensions were reduced using principal component analysis. The data projected by principal components were inserted into the k-means clustering algorithm and the k-nearest neighbors classifier with the laboratories as the target. The k-means silhouette index indicated that K = 2 clusters had the best separability in all tissue types. The supervised classification result showed laboratory effects and tissue-type bias. Both supervised and unsupervised approaches suggested that tissue type also affected inter-laboratory variance. We suggest tissue type to also be considered upon choosing the staining and color-normalization approach.

Digital Pathology Enables Automated and Quantitative Assessment of Inflammatory Activity in Patients with Chronic Liver Disease

Traditional histological evaluation for grading liver disease severity is based on subjective and semi-quantitative scores. We examined the relationship between digital pathology analysis and corresponding scoring systems for the assessment of hepatic necroinflammatory activity. A prospective, multicenter study including 156 patients with chronic liver disease (74% nonalcoholic fatty liver disease-NAFLD, 26% chronic hepatitis-CH etiologies) was performed. Inflammation was graded according to the Nonalcoholic Steatohepatitis (NASH) Clinical Research Network system and METAVIR score. Whole-slide digital image analysis based on quantitative (I-score: inflammation ratio) and morphometric (C-score: proportionate area of staining intensities clusters) measurements were independently performed. Our data show that I-scores and C-scores increase with inflammation grades (p < 0.001). High correlation was seen for CH (ρ = 0.85–0.88), but only moderate for NAFLD (ρ = 0.5–0.53). I-score (p = 0.008) and C-score (p = 0.002) were higher for CH than NAFLD. Our MATLAB algorithm performed better than QuPath software for the diagnosis of low-moderate inflammation (p < 0.05). C-score AUC for classifying NASH was 0.75 (95%CI, 0.65–0.84) and for moderate/severe CH was 0.99 (95%CI, 0.97–1.00). Digital pathology measurements increased with fibrosis stages (p < 0.001). In conclusion, quantitative and morphometric metrics of inflammatory burden obtained by digital pathology correlate well with pathologists’ scores, showing a higher accuracy for the evaluation of CH than NAFLD.

The utility of color normalization for AI-based diagnosis of hematoxylin and eosin-stained pathology images

The color variation of hematoxylin and eosin (H&E)-stained tissues has presented a challenge for applications of artificial intelligence (AI) in digital pathology. Many color normalization algorithms have been developed in recent years in order to reduce the color variation between H&E images. However, previous efforts in benchmarking these algorithms have produced conflicting results and none have sufficiently assessed the efficacy of the various color normalization methods for improving diagnostic performance of AI systems. In this study, we systematically investigated eight color normalization algorithms for AI-based classification of H&E-stained histopathology slides, in the context of using images both from one center and from multiple centers. Our results show that color normalization does not consistently improve classification performance when both training and testing data are from a single center. However, using four multi-center datasets of two cancer types (ovarian and pleural) and objective functions, we show that color normalization can significantly improve the classification accuracy of images from external datasets (ovarian cancer: 0.25 AUC increase, p = 1.6 e-05; pleural cancer: 0.21 AUC increase, p = 1.4 e-10). Furthermore, we introduce a novel augmentation strategy by mixing color-normalized images using three easily accessible algorithms that consistently improves the diagnosis of test images from external centers, even when the individual normalization methods had varied results. We anticipate our study to be a starting point for reliable use of color normalization to improve AI-based, digital pathology-empowered diagnosis of cancers sourced from multiple centers. © 2021 The Pathological Society of Great Britain and Ireland. Published by John Wiley & Sons, Ltd.

Normalization of HE-stained histological images using cycle consistent generative adversarial networks

Background

Histological images show strong variance (e.g. illumination, color, staining quality) due to differences in image acquisition, tissue processing, staining, etc. This can impede downstream image analysis such as staining intensity evaluation or classification. Methods to reduce these variances are called image normalization techniques.

Methods

In this paper, we investigate the potential of CycleGAN (cycle consistent Generative Adversarial Network) for color normalization in hematoxylin-eosin stained histological images using daily clinical data with consideration of the variability of internal staining protocol variations. The network consists of a generator network G_B that learns to map an image X from a source domain A to a target domain B, i.e. G_B:X_A→X_B. In addition, a discriminator network D_B is trained to distinguish whether an image from domain B is real or generated. The same process is applied to another generator-discriminator pair (G_A,D_A), for the inverse mapping G_A:X_B→X_A. Cycle consistency ensures that a generated image is close to its original when being mapped backwards (G_A(G_B(X_A))≈X_A and vice versa). We validate the CycleGAN approach on a breast cancer challenge and a follicular thyroid carcinoma data set for various stain variations. We evaluate the quality of the generated images compared to the original images using similarity measures. In addition, we apply stain normalization on pathological lymph node data from our institute and test the gain from normalization on a ResNet classifier pre-trained on the Camelyon16 data set.

Results

Qualitative results of the images generated by our network are compared to original color distributions. Our evaluation indicates that by mapping images to a target domain, the similarity training images from that domain improves up to 96%. We also achieve a high cycle consistency for the generator networks by obtaining similarity indices greater than 0.9. When applying the CycleGAN normalization to HE-stain images from our institute the kappa-value of the ResNet-model that is only trained on Camelyon16 data is increased more than 50%.

Conclusions

CycleGANs have proven to efficiently normalize HE-stained images. The approach compensates for deviations resulting from image acquisition (e.g. different scanning devices) as well as from tissue staining (e.g. different staining protocols), and thus overcomes the staining variations in images from various institutions.The code is publicly available at https://github.com/m4ln/stainTransfer_CycleGAN_pytorch. The data set supporting the solutions is available at 10.11588/data/8LKEZF.

Comparison of texture-based classification and deep learning for plantar soft tissue histology segmentation

Histomorphological measurements can be used to identify microstructural changes related to disease pathomechanics, in particular, plantar soft tissue changes with diabetes. However, these measurements are time-consuming and susceptible to sampling and human measurement error. We investigated two approaches to automate segmentation of plantar soft tissue stained with modified Hart's stain for elastin with the eventual goal of subsequent morphological analysis. The first approach used multiple texture- and color-based features with tile-wise classification. The second approach used a convolutional neural network modified from the U-Net architecture with fewer channel dimensions and additional downsampling steps. A hybrid color and texture feature, Fourier reduced histogram of uniform improved opponent color local binary patterns (f-IOCLBP), yielded the best feature-based segmentation, but still performed 3.6% worse on average than the modified U-Net. The texture-based method was sensitive to changes in illumination and stain intensity, and segmentation errors were often in large regions of single tissues or at tissue boundaries. The U-Net was able to segment small, few-pixel tissue boundaries, and errors were often trivial to clean up with post-processing. A U-Net approach outperforms hand-crafted features for segmentation of plantar soft tissue stained with modified Hart's stain for elastin.

Deep Learning Prediction of Metastasis in Locally Advanced Colon Cancer Using Binary Histologic Tumor Images

In this study, we developed the Binary ImaGe Colon Metastasis classifier (BIg-CoMet), a semi-guided approach for the stratification of colon cancer patients into two risk groups for the occurrence of distant metastasis, using an InceptionResNetV2-based deep learning model trained on binary images. We enrolled 291 colon cancer patients with pT3 and pT4 adenocarcinomas and converted one cytokeratin-stained representative tumor section per case into a binary image. Image augmentation and dropout layers were incorporated to avoid overfitting. In a validation collective (n = 128), BIg-CoMet was able to discriminate well between patients with and without metastasis (AUC: 0.842, 95% CI: 0.774-0.911). Further, the Kaplan-Meier curves of the metastasis-free survival showed a highly significant worse clinical course for the high-risk group (log-rank test: p < 0.001), and we demonstrated superiority over other established risk factors. A multivariable Cox regression analysis adjusted for confounders supported the use of risk groups as a prognostic factor for the occurrence of metastasis (hazard ratio (HR): 5.4, 95% CI: 2.5-11.7, p < 0.001). BIg-CoMet achieved good performance for both UICC subgroups, especially for UICC III (n = 53), with a positive predictive value of 80%. Our study demonstrates the ability to stratify colon cancer patients via a semi-guided process on images that primarily reflect tumor architecture.

Spatial immunoprofiling of the intratumoral and peritumoral tissue of renal cell carcinoma patients

While the abundance and phenotype of tumor-infiltrating lymphocytes are linked with clinical survival, their spatial coordination and its clinical significance remain unclear. Here, we investigated the immune profile of intratumoral and peritumoral tissue of clear cell renal cell carcinoma patients (n = 64). We trained a cell classifier to detect lymphocytes from hematoxylin and eosin stained tissue slides. Using unsupervised classification, patients were further classified into immune cold, hot and excluded topographies reflecting lymphocyte abundance and localization. The immune topography distribution was further validated with The Cancer Genome Atlas digital image dataset. We showed association between PBRM1 mutation and immune cold topography, STAG1 mutation and immune hot topography and BAP1 mutation and immune excluded topography. With quantitative multiplex immunohistochemistry we analyzed the expression of 23 lymphocyte markers in intratumoral and peritumoral tissue regions. To study spatial interactions, we developed an algorithm quantifying the proportion of adjacent immune cell pairs and their immunophenotypes. Immune excluded tumors were associated with superior overall survival (HR 0.19, p = 0.02) and less extensive metastasis. Intratumoral T cells were characterized with pronounced expression of immunological activation and exhaustion markers such as granzyme B, PD1, and LAG3. Immune cell interaction occurred most frequently in the intratumoral region and correlated with CD45RO expression. Moreover, high proportion of peritumoral CD45RO+ T cells predicted poor overall survival. In summary, intratumoral and peritumoral tissue regions represent distinct immunospatial profiles and are associated with clinicopathologic characteristics.