1
|
Hetz MJ, Bucher TC, Brinker TJ. Multi-domain stain normalization for digital pathology: A cycle-consistent adversarial network for whole slide images. Med Image Anal 2024; 94:103149. [PMID: 38574542 DOI: 10.1016/j.media.2024.103149] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/21/2022] [Revised: 12/11/2023] [Accepted: 03/20/2024] [Indexed: 04/06/2024]
Abstract
The variation in histologic staining between different medical centers is one of the most profound challenges in the field of computer-aided diagnosis. The appearance disparity of pathological whole slide images causes algorithms to become less reliable, which in turn impedes the wide-spread applicability of downstream tasks like cancer diagnosis. Furthermore, different stainings lead to biases in the training which in case of domain shifts negatively affect the test performance. Therefore, in this paper we propose MultiStain-CycleGAN, a multi-domain approach to stain normalization based on CycleGAN. Our modifications to CycleGAN allow us to normalize images of different origins without retraining or using different models. We perform an extensive evaluation of our method using various metrics and compare it to commonly used methods that are multi-domain capable. First, we evaluate how well our method fools a domain classifier that tries to assign a medical center to an image. Then, we test our normalization on the tumor classification performance of a downstream classifier. Furthermore, we evaluate the image quality of the normalized images using the Structural similarity index and the ability to reduce the domain shift using the Fréchet inception distance. We show that our method proves to be multi-domain capable, provides a very high image quality among the compared methods, and can most reliably fool the domain classifier while keeping the tumor classifier performance high. By reducing the domain influence, biases in the data can be removed on the one hand and the origin of the whole slide image can be disguised on the other, thus enhancing patient data privacy.
Collapse
Affiliation(s)
- Martin J Hetz
- Division of Digital Biomarkers for Oncology, German Cancer Research Center (DKFZ), Heidelberg, Germany
| | - Tabea-Clara Bucher
- Division of Digital Biomarkers for Oncology, German Cancer Research Center (DKFZ), Heidelberg, Germany
| | - Titus J Brinker
- Division of Digital Biomarkers for Oncology, German Cancer Research Center (DKFZ), Heidelberg, Germany.
| |
Collapse
|
2
|
Aalam SW, Ahanger AB, Masoodi TA, Bhat AA, Akil ASAS, Khan MA, Assad A, Macha MA, Bhat MR. Deep learning-based identification of esophageal cancer subtypes through analysis of high-resolution histopathology images. Front Mol Biosci 2024; 11:1346242. [PMID: 38567100 PMCID: PMC10985197 DOI: 10.3389/fmolb.2024.1346242] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/29/2023] [Accepted: 02/23/2024] [Indexed: 04/04/2024] Open
Abstract
Esophageal cancer (EC) remains a significant health challenge globally, with increasing incidence and high mortality rates. Despite advances in treatment, there remains a need for improved diagnostic methods and understanding of disease progression. This study addresses the significant challenges in the automatic classification of EC, particularly in distinguishing its primary subtypes: adenocarcinoma and squamous cell carcinoma, using histopathology images. Traditional histopathological diagnosis, while being the gold standard, is subject to subjectivity and human error and imposes a substantial burden on pathologists. This study proposes a binary class classification system for detecting EC subtypes in response to these challenges. The system leverages deep learning techniques and tissue-level labels for enhanced accuracy. We utilized 59 high-resolution histopathological images from The Cancer Genome Atlas (TCGA) Esophageal Carcinoma dataset (TCGA-ESCA). These images were preprocessed, segmented into patches, and analyzed using a pre-trained ResNet101 model for feature extraction. For classification, we employed five machine learning classifiers: Support Vector Classifier (SVC), Logistic Regression (LR), Decision Tree (DT), AdaBoost (AD), Random Forest (RF), and a Feed-Forward Neural Network (FFNN). The classifiers were evaluated based on their prediction accuracy on the test dataset, yielding results of 0.88 (SVC and LR), 0.64 (DT and AD), 0.82 (RF), and 0.94 (FFNN). Notably, the FFNN classifier achieved the highest Area Under the Curve (AUC) score of 0.92, indicating its superior performance, followed closely by SVC and LR, with a score of 0.87. This suggested approach holds promising potential as a decision-support tool for pathologists, particularly in regions with limited resources and expertise. The timely and precise detection of EC subtypes through this system can substantially enhance the likelihood of successful treatment, ultimately leading to reduced mortality rates in patients with this aggressive cancer.
Collapse
Affiliation(s)
- Syed Wajid Aalam
- Department of Computer Science, Islamic University of Science and Technology, Awantipora, India
| | - Abdul Basit Ahanger
- Department of Computer Science, Islamic University of Science and Technology, Awantipora, India
| | - Tariq A. Masoodi
- Human Immunology Department, Research Branch, Sidra Medicine, Doha, Qatar
| | - Ajaz A. Bhat
- Department of Human Genetics-Precision Medicine in Diabetes, Obesity and Cancer Program, Sidra Medicine, Doha, Qatar
| | - Ammira S. Al-Shabeeb Akil
- Department of Human Genetics-Precision Medicine in Diabetes, Obesity and Cancer Program, Sidra Medicine, Doha, Qatar
| | | | - Assif Assad
- Department of Computer Science and Engineering, Islamic University of Science and Technology, Awantipora, India
| | - Muzafar A. Macha
- Watson-Crick Centre for Molecular Medicine, Islamic University of Science and Technology, Awantipora, India
| | - Muzafar Rasool Bhat
- Department of Computer Science, Islamic University of Science and Technology, Awantipora, India
| |
Collapse
|
3
|
Dunn C, Brettle D, Cockroft M, Keating E, Revie C, Treanor D. Quantitative assessment of H&E staining for pathology: development and clinical evaluation of a novel system. Diagn Pathol 2024; 19:42. [PMID: 38395890 PMCID: PMC10885446 DOI: 10.1186/s13000-024-01461-w] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/08/2023] [Accepted: 02/03/2024] [Indexed: 02/25/2024] Open
Abstract
BACKGROUND Staining tissue samples to visualise cellular detail and tissue structure is at the core of pathology diagnosis, but variations in staining can result in significantly different appearances of the tissue sample. While the human visual system is adept at compensating for stain variation, with the growth of digital imaging in pathology, the impact of this variation can be more profound. Despite the ubiquity of haematoxylin and eosin staining in clinical practice worldwide, objective quantification is not yet available. We propose a method for quantitative haematoxylin and eosin stain assessment to facilitate quality assurance of histopathology staining, enabling truly quantitative quality control and improved standardisation. METHODS The stain quantification method comprises conventional microscope slides with a stain-responsive biopolymer film affixed to one side, called stain assessment slides. The stain assessment slides were characterised with haematoxylin and eosin, and implemented in one clinical laboratory to quantify variation levels. RESULTS Stain assessment slide stain uptake increased linearly with duration of haematoxylin and eosin staining (r = 0.99), and demonstrated linearly comparable staining to samples of human liver tissue (r values 0.98-0.99). Laboratory implementation of this technique quantified intra- and inter-instrument variation of staining instruments at one point in time and across a five-day period. CONCLUSION The proposed method has been shown to reliably quantify stain uptake, providing an effective laboratory quality control method for stain variation. This is especially important for whole slide imaging and the future development of artificial intelligence in digital pathology.
Collapse
Affiliation(s)
- Catriona Dunn
- National Pathology Imaging Co-operative, Leeds Teaching Hospitals NHS Trust, Leeds, UK.
- Department of Pathology and Data Analytics, University of Leeds, Leeds, UK.
| | - David Brettle
- National Pathology Imaging Co-operative, Leeds Teaching Hospitals NHS Trust, Leeds, UK
| | - Martin Cockroft
- New Technology Group, Futamura Chemical UK Limited, Wigton, UK
| | | | | | - Darren Treanor
- National Pathology Imaging Co-operative, Leeds Teaching Hospitals NHS Trust, Leeds, UK
- Department of Pathology and Data Analytics, University of Leeds, Leeds, UK
- Department of Histopathology, Leeds Teaching Hospitals NHS Trust, Leeds, UK
- Department of Clinical Pathology and Clinical and Experimental Medicine, Linköping University, Linköping, Sweden
- Centre for Medical Image Science and Visualisation, Linköping University, Linköping, Sweden
| |
Collapse
|
4
|
Azadi Moghadam P, Bashashati A, Goldenberg SL. Artificial Intelligence and Pathomics: Prostate Cancer. Urol Clin North Am 2024; 51:15-26. [PMID: 37945099 DOI: 10.1016/j.ucl.2023.06.001] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/12/2023]
Abstract
Artificial intelligence (AI) has the potential to transform pathologic diagnosis and cancer patient management as a predictive and prognostic biomarker. AI-based systems can be used to examine digitally scanned histopathology slides and differentiate benign from malignant cells and low from high grade. Deep learning models can analyze patient data from individual or multimodal combinations and identify patterns to be used to predict the response to different therapeutic options, the risk of recurrence or progression, and the prognosis of the newly diagnosed patient. AI-based models will improve treatment planning for patients with prostate cancer and improve the efficiency and cost-effectiveness of the pathology laboratory.
Collapse
Affiliation(s)
- Puria Azadi Moghadam
- Department of Electrical and Computer Engineering, University of British Columbia, 2332 Main Mall, Vancouver, British Columbia V6T 1Z4, Canada
| | - Ali Bashashati
- School of Biomedical Engineering, University of British Columbia, 2222 Health Sciences Mall, Vancouver, British Columbia V6T 1Z3, Canada; Department of Pathology and Laboratory Medicine, University of British Columbia, 2211 Wesbrook Mall, Vancouver, BC V6T 1Z7, Canada
| | - S Larry Goldenberg
- Department of Urologic Sciences, University of British Columbia, 2775 Laurel Street, Vancouver British Columbia V5Z 1M9, Canada.
| |
Collapse
|
5
|
Barrera K, Rodellar J, Alférez S, Merino A. Automatic normalized digital color staining in the recognition of abnormal blood cells using generative adversarial networks. COMPUTER METHODS AND PROGRAMS IN BIOMEDICINE 2023; 240:107629. [PMID: 37301181 DOI: 10.1016/j.cmpb.2023.107629] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 03/07/2023] [Revised: 05/23/2023] [Accepted: 05/28/2023] [Indexed: 06/12/2023]
Abstract
BACKGROUND AND OBJECTIVES Combining knowledge of clinical pathologists and deep learning models is a growing trend in morphological analysis of cells circulating in blood to add objectivity, accuracy, and speed in diagnosing hematological and non-hematological diseases. However, the variability in staining protocols across different laboratories can affect the color of images and performance of automatic recognition models. The objective of this work is to develop, train and evaluate a new system for the normalization of color staining of peripheral blood cell images, so that it transforms images from different centers to map the color staining of a reference center (RC) while preserving the structural morphological features. METHODS The system has two modules, GAN1 and GAN2. GAN1 uses the PIX2PIX technique to fade original color images to an adaptive gray, while GAN2 transforms them into RGB normalized images. Both GANs have a similar structure, where the generator is a U-NET convolutional neural network with ResNet and the discriminator is a classifier with ResNet34 structure. Digitally stained images were evaluated using GAN metrics and histograms to assess the ability to modify color without altering cell morphology. The system was also evaluated as a pre-processing tool before cells undergo a classification process. For this purpose, a CNN classifier was designed for three classes: abnormal lymphocytes, blasts and reactive lymphocytes. RESULTS Training of all GANs and the classifier was performed using RC images, while evaluations were conducted using images from four other centers. Classification tests were performed before and after applying the stain normalization system. The overall accuracy reached a similar value around 96% in both cases for the RC images, indicating the neutrality of the normalization model for the reference images. On the contrary, it was a significant improvement in the classification performance when applying the stain normalization to the other centers. Reactive lymphocytes were the most sensitive to stain normalization, with true positive rates (TPR) increasing from 46.3% - 66% for the original images to 81.2% - 97.2% after digital staining. Abnormal lymphocytes TPR ranged from 31.9% - 95.7% with original images to 83% - 100% with digitally stained images. Blast class showed TPR ranges of 90.3% - 94.4% and 94.4% - 100%, for original and stained images, respectively. CONCLUSIONS The proposed GAN-based normalization staining approach improves the performance of classifiers with multicenter data sets by generating digitally stained images with a quality similar to the original images and adaptability to a reference staining standard. The system requires low computation cost and can help improve the performance of automatic recognition models in clinical settings.
Collapse
Affiliation(s)
- Kevin Barrera
- Universitat Politècnica de Catalunya - BarcelonaTech (UPC), Barcelona East Engineering School, Department of Mathematics, Barcelona, Spain.
| | - José Rodellar
- Universitat Politècnica de Catalunya - BarcelonaTech (UPC), Barcelona East Engineering School, Department of Mathematics, Barcelona, Spain.
| | - Santiago Alférez
- Universitat Politècnica de Catalunya - BarcelonaTech (UPC), Barcelona East Engineering School, Department of Mathematics, Barcelona, Spain.
| | - Anna Merino
- Hospital Clínic of Barcelona-IDIBAPS, Biochemistry and Molecular Genetics Department, CORE Laboratory, Biomedical Diagnostic, Barcelona, Spain.
| |
Collapse
|
6
|
Impact of Stain Normalization on Pathologist Assessment of Prostate Cancer: A Comparative Study. Cancers (Basel) 2023; 15:cancers15051503. [PMID: 36900293 PMCID: PMC10000688 DOI: 10.3390/cancers15051503] [Citation(s) in RCA: 6] [Impact Index Per Article: 6.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/24/2023] [Revised: 02/17/2023] [Accepted: 02/23/2023] [Indexed: 03/04/2023] Open
Abstract
In clinical routine, the quality of whole-slide images plays a key role in the pathologist's diagnosis, and suboptimal staining may be a limiting factor. The stain normalization process helps to solve this problem through the standardization of color appearance of a source image with respect to a target image with optimal chromatic features. The analysis is focused on the evaluation of the following parameters assessed by two experts on original and normalized slides: (i) perceived color quality, (ii) diagnosis for the patient, (iii) diagnostic confidence and (iv) time required for diagnosis. Results show a statistically significant increase in color quality in the normalized images for both experts (p < 0.0001). Regarding prostate cancer assessment, the average times for diagnosis are significantly lower for normalized images than original ones (first expert: 69.9 s vs. 77.9 s with p < 0.0001; second expert: 37.4 s vs. 52.7 s with p < 0.0001), and at the same time, a statistically significant increase in diagnostic confidence is proven. The improvement of poor-quality images and greater clarity of diagnostically important details in normalized slides demonstrate the potential of stain normalization in the routine practice of prostate cancer assessment.
Collapse
|
7
|
Deep learning-based histotype diagnosis of ovarian carcinoma whole-slide pathology images. Mod Pathol 2022; 35:1983-1990. [PMID: 36065012 DOI: 10.1038/s41379-022-01146-z] [Citation(s) in RCA: 12] [Impact Index Per Article: 6.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/09/2022] [Revised: 07/19/2022] [Accepted: 07/20/2022] [Indexed: 12/24/2022]
Abstract
Ovarian carcinoma has the highest mortality of all female reproductive cancers and current treatment has become histotype-specific. Pathologists diagnose five common histotypes by microscopic examination, however, histotype determination is not straightforward, with only moderate interobserver agreement between general pathologists (Cohen's kappa 0.54-0.67). We hypothesized that machine learning (ML)-based image classification models may be able to recognize ovarian carcinoma histotype sufficiently well that they could aid pathologists in diagnosis. We trained four different artificial intelligence (AI) algorithms based on deep convolutional neural networks to automatically classify hematoxylin and eosin-stained whole slide images. Performance was assessed through cross-validation on the training set (948 slides corresponding to 485 patients), and on an independent test set of 60 patients from another institution. The best-performing model achieved a diagnostic concordance of 81.38% (Cohen's kappa of 0.7378) in our training set, and 80.97% concordance (Cohen's kappa 0.7547) on the external dataset. Eight cases misclassified by ML in the external set were reviewed by two subspecialty pathologists blinded to the diagnoses, molecular and immunophenotype data, and ML-based predictions. Interestingly, in 4 of 8 cases from the external dataset, the expert review pathologists rendered diagnoses, based on blind review of the whole section slides classified by AI, that were in agreement with AI rather than the integrated reference diagnosis. The performance characteristics of our classifiers indicate potential for improved diagnostic performance if used as an adjunct to conventional histopathology.
Collapse
|
8
|
Hameed Z, Garcia-Zapirain B, Aguirre JJ, Isaza-Ruget MA. Multiclass classification of breast cancer histopathology images using multilevel features of deep convolutional neural network. Sci Rep 2022; 12:15600. [PMID: 36114214 PMCID: PMC9649689 DOI: 10.1038/s41598-022-19278-2] [Citation(s) in RCA: 11] [Impact Index Per Article: 5.5] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/24/2021] [Accepted: 08/26/2022] [Indexed: 12/03/2022] Open
Abstract
Breast cancer is a common malignancy and a leading cause of cancer-related deaths in women worldwide. Its early diagnosis can significantly reduce the morbidity and mortality rates in women. To this end, histopathological diagnosis is usually followed as the gold standard approach. However, this process is tedious, labor-intensive, and may be subject to inter-reader variability. Accordingly, an automatic diagnostic system can assist to improve the quality of diagnosis. This paper presents a deep learning approach to automatically classify hematoxylin-eosin-stained breast cancer microscopy images into normal tissue, benign lesion, in situ carcinoma, and invasive carcinoma using our collected dataset. Our proposed model exploited six intermediate layers of the Xception (Extreme Inception) network to retrieve robust and abstract features from input images. First, we optimized the proposed model on the original (unnormalized) dataset using 5-fold cross-validation. Then, we investigated its performance on four normalized datasets resulting from Reinhard, Ruifrok, Macenko, and Vahadane stain normalization. For original images, our proposed framework yielded an accuracy of 98% along with a kappa score of 0.969. Also, it achieved an average AUC-ROC score of 0.998 as well as a mean AUC-PR value of 0.995. Specifically, for in situ carcinoma and invasive carcinoma, it offered sensitivity of 96% and 99%, respectively. For normalized images, the proposed architecture performed better for Makenko normalization compared to the other three techniques. In this case, the proposed model achieved an accuracy of 97.79% together with a kappa score of 0.965. Also, it attained an average AUC-ROC score of 0.997 and a mean AUC-PR value of 0.991. Especially, for in situ carcinoma and invasive carcinoma, it offered sensitivity of 96% and 99%, respectively. These results demonstrate that our proposed model outperformed the baseline AlexNet as well as state-of-the-art VGG16, VGG19, Inception-v3, and Xception models with their default settings. Furthermore, it can be inferred that although stain normalization techniques offered competitive performance, they could not surpass the results of the original dataset.
Collapse
|
9
|
Adamo A, Bruno A, Menallo G, Francipane MG, Fazzari M, Pirrone R, Ardizzone E, Wagner WR, D'Amore A. Blood Vessel Detection Algorithm for Tissue Engineering and Quantitative Histology. Ann Biomed Eng 2022; 50:387-400. [PMID: 35171393 PMCID: PMC8917109 DOI: 10.1007/s10439-022-02923-2] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/14/2021] [Accepted: 01/17/2022] [Indexed: 11/29/2022]
Abstract
Immunohistochemistry for vascular network analysis plays a fundamental role in basic science, translational research and clinical practice. However, identifying vascularization in histological tissue images is time consuming and markedly depends on the operator’s experience. In this study, we present “blood vessel detection—BVD”, an automatic algorithm for quantitative analysis of blood vessels in immunohistochemical images. BVD is based on extraction and analysis of low-level image features and spatial filtering techniques, which do not require a training phase. BVD algorithm performance was comparatively evaluated on histological sections from three different in vivo experiments. Collectively, 173 independent images were analyzed, and the algorithm's results were compared to those obtained by human operators. The developed BVD algorithm proved to be a robust and versatile tool, being able to quantify number, area, and spatial distribution of blood vessels within all three considered histologic datasets. BVD is provided as an open-source application working on different operating systems. BVD is supported by a user-friendly graphical interface designed to facilitate large-scale analysis.
Collapse
Affiliation(s)
- A Adamo
- Department of Health Promotion, Mother and Child Care, Internal Medicine and Medical Specialties, University of Palermo, 90100, Palermo, Italy.,McGowan Institute for Regenerative Medicine, University of Pittsburgh, Pittsburgh, PA, 15219, USA.,Fondazione Ri.MED, 90133, Palermo, Italy
| | - A Bruno
- Department of Computing and Informatics in the Faculty of Science and Technology, Bournemouth University, Poole, BH12 5BB, UK
| | - G Menallo
- McGowan Institute for Regenerative Medicine, University of Pittsburgh, Pittsburgh, PA, 15219, USA.,Department of Biomedical Engineering, Worcester Polytechnic Institute, Worcester, MA, 01605, USA
| | - M G Francipane
- McGowan Institute for Regenerative Medicine, University of Pittsburgh, Pittsburgh, PA, 15219, USA.,Fondazione Ri.MED, 90133, Palermo, Italy.,Department of Pathology, University of Pittsburgh, Pittsburgh, PA, 15206, USA
| | - M Fazzari
- Department of Pharmacology & Chemical Biology, University of Pittsburgh, Pittsburgh, PA, 15261, USA
| | - R Pirrone
- Department of Industrial and Digital Innovation, University of Palermo, 90100, Palermo, Italy
| | - E Ardizzone
- Department of Industrial and Digital Innovation, University of Palermo, 90100, Palermo, Italy
| | - W R Wagner
- McGowan Institute for Regenerative Medicine, University of Pittsburgh, Pittsburgh, PA, 15219, USA.,Department of Surgery, University of Pittsburgh, Pittsburgh, PA, 15219, USA
| | - A D'Amore
- McGowan Institute for Regenerative Medicine, University of Pittsburgh, Pittsburgh, PA, 15219, USA. .,Fondazione Ri.MED, 90133, Palermo, Italy. .,Department of Surgery, University of Pittsburgh, Pittsburgh, PA, 15219, USA. .,Department of Surgery and Bioengineering, McGowan Institute for Regenerative Medicine, University of Pittsburgh, 450 Technology Drive, Pittsburgh, PA, 15219, USA.
| |
Collapse
|
10
|
Boschman J, Farahani H, Darbandsari A, Ahmadvand P, Van Spankeren A, Farnell D, Levine AB, Naso JR, Churg A, Jones SJ, Yip S, Köbel M, Huntsman DG, Gilks CB, Bashashati A. The utility of color normalization for AI-based diagnosis of hematoxylin and eosin-stained pathology images. J Pathol 2021; 256:15-24. [PMID: 34543435 DOI: 10.1002/path.5797] [Citation(s) in RCA: 21] [Impact Index Per Article: 7.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/11/2021] [Revised: 08/11/2021] [Accepted: 09/16/2021] [Indexed: 12/17/2022]
Abstract
The color variation of hematoxylin and eosin (H&E)-stained tissues has presented a challenge for applications of artificial intelligence (AI) in digital pathology. Many color normalization algorithms have been developed in recent years in order to reduce the color variation between H&E images. However, previous efforts in benchmarking these algorithms have produced conflicting results and none have sufficiently assessed the efficacy of the various color normalization methods for improving diagnostic performance of AI systems. In this study, we systematically investigated eight color normalization algorithms for AI-based classification of H&E-stained histopathology slides, in the context of using images both from one center and from multiple centers. Our results show that color normalization does not consistently improve classification performance when both training and testing data are from a single center. However, using four multi-center datasets of two cancer types (ovarian and pleural) and objective functions, we show that color normalization can significantly improve the classification accuracy of images from external datasets (ovarian cancer: 0.25 AUC increase, p = 1.6 e-05; pleural cancer: 0.21 AUC increase, p = 1.4 e-10). Furthermore, we introduce a novel augmentation strategy by mixing color-normalized images using three easily accessible algorithms that consistently improves the diagnosis of test images from external centers, even when the individual normalization methods had varied results. We anticipate our study to be a starting point for reliable use of color normalization to improve AI-based, digital pathology-empowered diagnosis of cancers sourced from multiple centers. © 2021 The Pathological Society of Great Britain and Ireland. Published by John Wiley & Sons, Ltd.
Collapse
Affiliation(s)
- Jeffrey Boschman
- School of Biomedical Engineering, University of British Columbia, Vancouver, BC, Canada
| | - Hossein Farahani
- School of Biomedical Engineering, University of British Columbia, Vancouver, BC, Canada.,Department of Pathology and Laboratory Medicine, University of British Columbia, Vancouver, BC, Canada
| | - Amirali Darbandsari
- Department of Electrical and Computer Engineering, University of British Columbia, Vancouver, BC, Canada
| | - Pouya Ahmadvand
- School of Biomedical Engineering, University of British Columbia, Vancouver, BC, Canada
| | - Ashley Van Spankeren
- School of Biomedical Engineering, University of British Columbia, Vancouver, BC, Canada
| | - David Farnell
- Department of Pathology and Laboratory Medicine, University of British Columbia, Vancouver, BC, Canada.,Vancouver General Hospital, Vancouver, BC, Canada
| | - Adrian B Levine
- Department of Pathology and Laboratory Medicine, University of British Columbia, Vancouver, BC, Canada.,Vancouver General Hospital, Vancouver, BC, Canada
| | - Julia R Naso
- Department of Pathology and Laboratory Medicine, University of British Columbia, Vancouver, BC, Canada.,Vancouver General Hospital, Vancouver, BC, Canada
| | - Andrew Churg
- Department of Pathology and Laboratory Medicine, University of British Columbia, Vancouver, BC, Canada.,Vancouver General Hospital, Vancouver, BC, Canada
| | - Steven Jm Jones
- British Columbia Cancer Research Center, Vancouver, BC, Canada
| | - Stephen Yip
- Department of Pathology and Laboratory Medicine, University of British Columbia, Vancouver, BC, Canada.,Vancouver General Hospital, Vancouver, BC, Canada
| | - Martin Köbel
- Department of Pathology and Laboratory Medicine, University of Calgary, Calgary, BC, Canada
| | - David G Huntsman
- Department of Pathology and Laboratory Medicine, University of British Columbia, Vancouver, BC, Canada.,British Columbia Cancer Research Center, Vancouver, BC, Canada
| | - C Blake Gilks
- Department of Pathology and Laboratory Medicine, University of British Columbia, Vancouver, BC, Canada.,Vancouver General Hospital, Vancouver, BC, Canada
| | - Ali Bashashati
- School of Biomedical Engineering, University of British Columbia, Vancouver, BC, Canada.,Department of Pathology and Laboratory Medicine, University of British Columbia, Vancouver, BC, Canada
| |
Collapse
|
11
|
Das A, Nair MS, Peter SD. Computer-Aided Histopathological Image Analysis Techniques for Automated Nuclear Atypia Scoring of Breast Cancer: a Review. J Digit Imaging 2021; 33:1091-1121. [PMID: 31989390 DOI: 10.1007/s10278-019-00295-z] [Citation(s) in RCA: 33] [Impact Index Per Article: 11.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/14/2022] Open
Abstract
Breast cancer is the most common type of malignancy diagnosed in women. Through early detection and diagnosis, there is a great chance of recovery and thereby reduce the mortality rate. Many preliminary tests like non-invasive radiological diagnosis using ultrasound, mammography, and MRI are widely used for the diagnosis of breast cancer. However, histopathological analysis of breast biopsy specimen is inevitable and is considered to be the golden standard for the affirmation of cancer. With the advancements in the digital computing capabilities, memory capacity, and imaging modalities, the development of computer-aided powerful analytical techniques for histopathological data has increased dramatically. These automated techniques help to alleviate the laborious work of the pathologist and to improve the reproducibility and reliability of the interpretation. This paper reviews and summarizes digital image computational algorithms applied on histopathological breast cancer images for nuclear atypia scoring and explores the future possibilities. The algorithms for nuclear pleomorphism scoring of breast cancer can be widely grouped into two categories: handcrafted feature-based and learned feature-based. Handcrafted feature-based algorithms mainly include the computational steps like pre-processing the images, segmenting the nuclei, extracting unique features, feature selection, and machine learning-based classification. However, most of the recent algorithms are based on learned features, that extract high-level abstractions directly from the histopathological images utilizing deep learning techniques. In this paper, we discuss the various algorithms applied for the nuclear pleomorphism scoring of breast cancer, discourse the challenges to be dealt with, and outline the importance of benchmark datasets. A comparative analysis of some prominent works on breast cancer nuclear atypia scoring is done using a benchmark dataset which enables to quantitatively measure and compare the different features and algorithms used for breast cancer grading. Results show that improvements are still required, to have an automated cancer grading system suitable for clinical applications.
Collapse
Affiliation(s)
- Asha Das
- Artificial Intelligence & Computer Vision Lab, Department of Computer Science, Cochin University of Science and Technology, Kochi, Kerala, 682022, India.
| | - Madhu S Nair
- Artificial Intelligence & Computer Vision Lab, Department of Computer Science, Cochin University of Science and Technology, Kochi, Kerala, 682022, India
| | - S David Peter
- Artificial Intelligence & Computer Vision Lab, Department of Computer Science, Cochin University of Science and Technology, Kochi, Kerala, 682022, India
| |
Collapse
|
12
|
Doherty T, McKeever S, Al-Attar N, Murphy T, Aura C, Rahman A, O'Neill A, Finn SP, Kay E, Gallagher WM, Watson RWG, Gowen A, Jackman P. Feature fusion of Raman chemical imaging and digital histopathology using machine learning for prostate cancer detection. Analyst 2021; 146:4195-4211. [PMID: 34060548 DOI: 10.1039/d1an00075f] [Citation(s) in RCA: 8] [Impact Index Per Article: 2.7] [Reference Citation Analysis] [Abstract] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/13/2022]
Abstract
The diagnosis of prostate cancer is challenging due to the heterogeneity of its presentations, leading to the over diagnosis and treatment of non-clinically important disease. Accurate diagnosis can directly benefit a patient's quality of life and prognosis. Towards addressing this issue, we present a learning model for the automatic identification of prostate cancer. While many prostate cancer studies have adopted Raman spectroscopy approaches, none have utilised the combination of Raman Chemical Imaging (RCI) and other imaging modalities. This study uses multimodal images formed from stained Digital Histopathology (DP) and unstained RCI. The approach was developed and tested on a set of 178 clinical samples from 32 patients, containing a range of non-cancerous, Gleason grade 3 (G3) and grade 4 (G4) tissue microarray samples. For each histological sample, there is a pathologist labelled DP-RCI image pair. The hypothesis tested was whether multimodal image models can outperform single modality baseline models in terms of diagnostic accuracy. Binary non-cancer/cancer models and the more challenging G3/G4 differentiation were investigated. Regarding G3/G4 classification, the multimodal approach achieved a sensitivity of 73.8% and specificity of 88.1% while the baseline DP model showed a sensitivity and specificity of 54.1% and 84.7% respectively. The multimodal approach demonstrated a statistically significant 12.7% AUC advantage over the baseline with a value of 85.8% compared to 73.1%, also outperforming models based solely on RCI and mean and median Raman spectra. Feature fusion of DP and RCI does not improve the more trivial task of tumour identification but does deliver an observed advantage in G3/G4 discrimination. Building on these promising findings, future work could include the acquisition of larger datasets for enhanced model generalization.
Collapse
Affiliation(s)
- Trevor Doherty
- Technological University Dublin, School of Computer Science, City Campus, Grangegorman Lower, Dublin 7, Ireland.
| | | | | | | | | | | | | | | | | | | | | | | | | |
Collapse
|
13
|
Chen X, Yu J, Cheng S, Geng X, Liu S, Han W, Hu J, Chen L, Liu X, Zeng S. An unsupervised style normalization method for cytopathology images. Comput Struct Biotechnol J 2021; 19:3852-3863. [PMID: 34285783 PMCID: PMC8273362 DOI: 10.1016/j.csbj.2021.06.025] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/22/2020] [Revised: 06/10/2021] [Accepted: 06/15/2021] [Indexed: 11/17/2022] Open
Abstract
Diverse styles of cytopathology images have a negative effect on the generalization ability of automated image analysis algorithms. This article proposes an unsupervised method to normalize cytopathology image styles. We design a two-stage style normalization framework with a style removal module to convert the colorful cytopathology image into a gray-scale image with a color-encoding mask and a domain adversarial style reconstruction module to map them back to a colorful image with user-selected style. Our method enforces both hue and structure consistency before and after normalization by using the color-encoding mask and per-pixel regression. Intra-domain and inter-domain adversarial learning are applied to ensure the style of normalized images consistent with the user-selected for input images of different domains. Our method shows superior results against current unsupervised color normalization methods on six cervical cell datasets from different hospitals and scanners. We further demonstrate that our normalization method greatly improves the recognition accuracy of lesion cells on unseen cytopathology images, which is meaningful for model generalization.
Collapse
Affiliation(s)
- Xihao Chen
- Britton Chance Center for Biomedical Photonics, Wuhan National Laboratory for Optoelectronics-Huazhong University of Science and Technology, Wuhan, Hubei, China
| | - Jingya Yu
- Britton Chance Center for Biomedical Photonics, Wuhan National Laboratory for Optoelectronics-Huazhong University of Science and Technology, Wuhan, Hubei, China
| | - Shenghua Cheng
- Britton Chance Center for Biomedical Photonics, Wuhan National Laboratory for Optoelectronics-Huazhong University of Science and Technology, Wuhan, Hubei, China
| | - Xiebo Geng
- Britton Chance Center for Biomedical Photonics, Wuhan National Laboratory for Optoelectronics-Huazhong University of Science and Technology, Wuhan, Hubei, China
| | - Sibo Liu
- Britton Chance Center for Biomedical Photonics, Wuhan National Laboratory for Optoelectronics-Huazhong University of Science and Technology, Wuhan, Hubei, China
| | - Wei Han
- Britton Chance Center for Biomedical Photonics, Wuhan National Laboratory for Optoelectronics-Huazhong University of Science and Technology, Wuhan, Hubei, China
| | - Junbo Hu
- Women and Children Hospital of Hubei Province, Wuhan, Hubei, China
| | - Li Chen
- Department of Clinical Laboratory, Tongji Hospital, Huazhong University of Science and Technology, Wuhan, Hubei, China
| | - Xiuli Liu
- Britton Chance Center for Biomedical Photonics, Wuhan National Laboratory for Optoelectronics-Huazhong University of Science and Technology, Wuhan, Hubei, China
| | - Shaoqun Zeng
- Britton Chance Center for Biomedical Photonics, Wuhan National Laboratory for Optoelectronics-Huazhong University of Science and Technology, Wuhan, Hubei, China
| |
Collapse
|
14
|
Salvi M, Acharya UR, Molinari F, Meiburger KM. The impact of pre- and post-image processing techniques on deep learning frameworks: A comprehensive review for digital pathology image analysis. Comput Biol Med 2021; 128:104129. [DOI: 10.1016/j.compbiomed.2020.104129] [Citation(s) in RCA: 94] [Impact Index Per Article: 31.3] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/17/2020] [Accepted: 11/13/2020] [Indexed: 12/12/2022]
|
15
|
Objective Diagnosis for Histopathological Images Based on Machine Learning Techniques: Classical Approaches and New Trends. MATHEMATICS 2020. [DOI: 10.3390/math8111863] [Citation(s) in RCA: 8] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 02/02/2023]
Abstract
Histopathology refers to the examination by a pathologist of biopsy samples. Histopathology images are captured by a microscope to locate, examine, and classify many diseases, such as different cancer types. They provide a detailed view of different types of diseases and their tissue status. These images are an essential resource with which to define biological compositions or analyze cell and tissue structures. This imaging modality is very important for diagnostic applications. The analysis of histopathology images is a prolific and relevant research area supporting disease diagnosis. In this paper, the challenges of histopathology image analysis are evaluated. An extensive review of conventional and deep learning techniques which have been applied in histological image analyses is presented. This review summarizes many current datasets and highlights important challenges and constraints with recent deep learning techniques, alongside possible future research avenues. Despite the progress made in this research area so far, it is still a significant area of open research because of the variety of imaging techniques and disease-specific characteristics.
Collapse
|
16
|
Leo P, Elliott R, Shih NNC, Gupta S, Feldman M, Madabhushi A. Stable and discriminating features are predictive of cancer presence and Gleason grade in radical prostatectomy specimens: a multi-site study. Sci Rep 2018; 8:14918. [PMID: 30297720 PMCID: PMC6175913 DOI: 10.1038/s41598-018-33026-5] [Citation(s) in RCA: 24] [Impact Index Per Article: 4.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/18/2017] [Accepted: 09/20/2018] [Indexed: 01/17/2023] Open
Abstract
Site variation in fixation, staining, and scanning can confound automated tissue based image classifiers for disease characterization. In this study we incorporated stability into four feature selection methods for identifying the most robust and discriminating features for two prostate histopathology classification tasks. We evaluated 242 morphology features from N = 212 prostatectomy specimens from four sites for automated cancer detection and grading. We quantified instability as the rate of significant cross-site feature differences. We mapped feature stability and discriminability using 188 non-cancerous and 210 cancerous regions via 3-fold cross validation, then held one site out, creating independent training and testing sets. In training, one feature set was selected only for discriminability, another for discriminability and stability. We trained a classifier with each feature set, testing on the hold out site. Experiments were repeated with 117 Gleason grade 3 and 112 grade 4 regions. Stability was calculated across non-cancerous regions. Gland shape features yielded the best stability and area under the receiver operating curve (AUC) trade-off while co-occurrence texture features were generally unstable. Our stability-informed method produced a cancer detection AUC of 0.98 ± 0.05 and increased average Gleason grading AUC by 4.38%. Color normalization of the images tended to exacerbate feature instability.
Collapse
Affiliation(s)
- Patrick Leo
- Case Western Reserve University, Dept. of Biomedical Engineering, Cleveland, OH, 44106, United States
| | - Robin Elliott
- Case Western Reserve University, Dept. of Pathology, Cleveland, OH, 44106, United States
| | - Natalie N C Shih
- University of Pennsylvania, Dept. of Pathology, Philadelphia, PA, 19104, United States
| | - Sanjay Gupta
- Case Western Reserve University, Dept. of Urology, Cleveland, OH, 44106, United States
| | - Michael Feldman
- University of Pennsylvania, Dept. of Pathology, Philadelphia, PA, 19104, United States
| | - Anant Madabhushi
- Case Western Reserve University, Dept. of Biomedical Engineering, Cleveland, OH, 44106, United States.
| |
Collapse
|
17
|
Bentaieb A, Hamarneh G. Adversarial Stain Transfer for Histopathology Image Analysis. IEEE TRANSACTIONS ON MEDICAL IMAGING 2018; 37:792-802. [PMID: 29533895 DOI: 10.1109/tmi.2017.2781228] [Citation(s) in RCA: 89] [Impact Index Per Article: 14.8] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 05/28/2023]
Abstract
It is generally recognized that color information is central to the automatic and visual analysis of histopathology tissue slides. In practice, pathologists rely on color, which reflects the presence of specific tissue components, to establish a diagnosis. Similarly, automatic histopathology image analysis algorithms rely on color or intensity measures to extract tissue features. With the increasing access to digitized histopathology images, color variation and its implications have become a critical issue. These variations are the result of not only a variety of factors involved in the preparation of tissue slides but also in the digitization process itself. Consequently, different strategies have been proposed to alleviate stain-related tissue inconsistencies in automatic image analysis systems. Such techniques generally rely on collecting color statistics to perform color matching across images. In this work, we propose a different approach for stain normalization that we refer to as stain transfer. We design a discriminative image analysis model equipped with a stain normalization component that transfers stains across datasets. Our model comprises a generative network that learns data set-specific staining properties and image-specific color transformations as well as a task-specific network (e.g., classifier or segmentation network). The model is trained end-to-end using a multi-objective cost function. We evaluate the proposed approach in the context of automatic histopathology image analysis on three data sets and two different analysis tasks: tissue segmentation and classification. The proposed method achieves superior results in terms of accuracy and quality of normalized images compared to various baselines.
Collapse
|
18
|
Abstract
Immunohistochemical (IHC) biomarkers in breast tissue microarray (TMA) samples are used daily in pathology departments. In recent years, automatic methods to evaluate positive staining have been investigated since they may save time and reduce errors in the diagnosis. These errors are mostly due to subjective evaluation. The aim of this work is to develop a density tool able to automatically quantify the positive brown IHC stain in breast TMA for different biomarkers. To avoid the problem of colour variation and make a robust tool independent of the staining process, several colour standardization methods have been analysed. Four colour standardization methods have been compared against colour model segmentation. The standardization methods have been compared by means of NBS colour distance. The use of colour standardization helps to reduce noise due to stain and histological sample preparation. However, the most reliable and robust results have been obtained by combining the HSV and RGB colour models for segmentation with the HSB channels. The segmentation provides three outputs based on three saturation values for weak, medium and strong staining. Each output image can be combined according to the type of biomarker staining. The results with 12 biomarkers were evaluated and compared to the segmentation and density calculation done by expert pathologists. The Hausdorff distance, sensitivity and specificity have been used to quantitative validate the results. The tests carried out with 8000 TMA images provided an average of 95.94% accuracy applied to the total tissue cylinder area. Colour standardization was used only when the tissue core had blurring and fading stain and the expert could not evaluate them without a pre-processing.
Collapse
|
19
|
Ginley B, Tomaszewski JE, Yacoub R, Chen F, Sarder P. Unsupervised labeling of glomerular boundaries using Gabor filters and statistical testing in renal histology. J Med Imaging (Bellingham) 2017; 4:021102. [PMID: 28331889 DOI: 10.1117/1.jmi.4.2.021102] [Citation(s) in RCA: 23] [Impact Index Per Article: 3.3] [Reference Citation Analysis] [Abstract] [Key Words] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/01/2016] [Accepted: 12/14/2016] [Indexed: 11/14/2022] Open
Abstract
The glomerulus is the blood filtering unit of the kidney. Each human kidney contains [Formula: see text] glomeruli. Several renal conditions originate from structural damage to glomerular microcompartments, such as proteinuria, the excessive loss of blood proteins into urine. The gold standard for evaluating structural damage in renal pathology is histopathological and immunofluorescence examination of needle biopsies under a light microscope. This method is limited by qualitative or semiquantitative manual scoring approaches to the evaluation of glomerular structural features. Computational quantification of equivalent features promises to improve the precision of glomerular structural analysis. One large obstacle to the computational quantification of renal tissue is the identification of complex glomerular boundaries automatically. To mitigate this issue, we developed a computational pipeline capable of extracting and exactly defining glomerular boundaries. Our method, composed of Gabor filtering, Gaussian blurring, statistical [Formula: see text]-testing, and distance transform, is able to accurately identify glomerular boundaries with mean sensitivity/specificity of [Formula: see text] and accuracy of 0.92, on [Formula: see text] glomeruli images stained with standard renal histological stains. Our method will simplify computational partitioning of glomerular microcompartments hidden within dense textural boundaries. Automatic quantification of glomeruli will streamline structural analysis in clinic and can pioneer real-time diagnoses and interventions for renal care.
Collapse
Affiliation(s)
- Brandon Ginley
- University at Buffalo-The State University of New York , Departments of Pathology and Anatomical Sciences, 207 Farber Hall, 3435 Main Street Buffalo, New York 14214, United States
| | - John E Tomaszewski
- University at Buffalo-The State University of New York, Departments of Pathology and Anatomical Sciences, 207 Farber Hall, 3435 Main Street Buffalo, New York 14214, United States; University at Buffalo-The State University of New York, Departments of Biomedical Informatics, 207 Farber Hall, 3435 Main Street Buffalo, New York 14214, United States
| | - Rabi Yacoub
- University at Buffalo-The State University of New York , Departments of Medicine-Nephrology, 207 Farber Hall, 3435 Main Street Buffalo, New York 14214, United States
| | - Feng Chen
- Washington University School of Medicine in Saint Louis , Department of Medicine-Renal Division, Campus Box 8126, St. Louis, Missouri 63110, United States
| | - Pinaki Sarder
- University at Buffalo-The State University of New York, Departments of Pathology and Anatomical Sciences, 207 Farber Hall, 3435 Main Street Buffalo, New York 14214, United States; University at Buffalo-The State University of New York, Departments of Biomedical Engineering, 207 Farber Hall, 3435 Main Street Buffalo, New York 14214, United States; University at Buffalo-The State University of New York, Departments of Biostatistics, 207 Farber Hall, 3435 Main Street Buffalo, New York 14214, United States
| |
Collapse
|
20
|
Leo P, Lee G, Shih NNC, Elliott R, Feldman MD, Madabhushi A. Evaluating stability of histomorphometric features across scanner and staining variations: prostate cancer diagnosis from whole slide images. J Med Imaging (Bellingham) 2016; 3:047502. [PMID: 27803941 DOI: 10.1117/1.jmi.3.4.047502] [Citation(s) in RCA: 42] [Impact Index Per Article: 5.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/03/2016] [Accepted: 09/16/2016] [Indexed: 01/04/2023] Open
Abstract
Quantitative histomorphometry (QH) is the process of computerized feature extraction from digitized tissue slide images to predict disease presence, behavior, and outcome. Feature stability between sites may be compromised by laboratory-specific variables including dye batch, slice thickness, and the whole slide scanner used. We present two new measures, preparation-induced instability score and latent instability score, to quantify feature instability across and within datasets. In a use case involving prostate cancer, we examined QH features which may detect cancer on whole slide images. Using our method, we found that five feature families (graph, shape, co-occurring gland tensor, sub-graph, and texture) were different between datasets in 19.7% to 48.6% of comparisons while the values expected without site variation were 4.2% to 4.6%. Color normalizing all images to a template did not reduce instability. Scanning the same 34 slides on three scanners demonstrated that Haralick features were most substantively affected by scanner variation, being unstable in 62% of comparisons. We found that unstable feature families performed significantly worse in inter- than intrasite classification. Our results appear to suggest QH features should be evaluated across sites to assess robustness, and class discriminability alone should not represent the benchmark for digital pathology feature selection.
Collapse
Affiliation(s)
- Patrick Leo
- Case Western Reserve University , Department of Biomedical Engineering, 2071 Martin Luther King Jr. Drive, Cleveland, Ohio 44106, United States
| | - George Lee
- Case Western Reserve University , Department of Biomedical Engineering, 2071 Martin Luther King Jr. Drive, Cleveland, Ohio 44106, United States
| | - Natalie N C Shih
- University of Pennsylvania , Department of Pathology, 3400 Spruce Street, Philadelphia, Pennsylvania 19104, United States
| | - Robin Elliott
- Case Western Reserve University , Department of Pathology, 11100 Euclid Avenue, Cleveland, Ohio 44106, United States
| | - Michael D Feldman
- University of Pennsylvania , Department of Pathology, 3400 Spruce Street, Philadelphia, Pennsylvania 19104, United States
| | - Anant Madabhushi
- Case Western Reserve University , Department of Biomedical Engineering, 2071 Martin Luther King Jr. Drive, Cleveland, Ohio 44106, United States
| |
Collapse
|
21
|
Tam A, Barker J, Rubin D. A method for normalizing pathology images to improve feature extraction for quantitative pathology. Med Phys 2016; 43:528. [PMID: 26745946 DOI: 10.1118/1.4939130] [Citation(s) in RCA: 15] [Impact Index Per Article: 1.9] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/07/2022] Open
Abstract
PURPOSE With the advent of digital slide scanning technologies and the potential proliferation of large repositories of digital pathology images, many research studies can leverage these data for biomedical discovery and to develop clinical applications. However, quantitative analysis of digital pathology images is impeded by batch effects generated by varied staining protocols and staining conditions of pathological slides. METHODS To overcome this problem, this paper proposes a novel, fully automated stain normalization method to reduce batch effects and thus aid research in digital pathology applications. Their method, intensity centering and histogram equalization (ICHE), normalizes a diverse set of pathology images by first scaling the centroids of the intensity histograms to a common point and then applying a modified version of contrast-limited adaptive histogram equalization. Normalization was performed on two datasets of digitized hematoxylin and eosin (H&E) slides of different tissue slices from the same lung tumor, and one immunohistochemistry dataset of digitized slides created by restaining one of the H&E datasets. RESULTS The ICHE method was evaluated based on image intensity values, quantitative features, and the effect on downstream applications, such as a computer aided diagnosis. For comparison, three methods from the literature were reimplemented and evaluated using the same criteria. The authors found that ICHE not only improved performance compared with un-normalized images, but in most cases showed improvement compared with previous methods for correcting batch effects in the literature. CONCLUSIONS ICHE may be a useful preprocessing step a digital pathology image processing pipeline.
Collapse
Affiliation(s)
- Allison Tam
- Stanford Institutes of Medical Research Program, Stanford University School of Medicine, Stanford, California 94305
| | - Jocelyn Barker
- Department of Radiology, Stanford University School of Medicine, Stanford, California 94305
| | - Daniel Rubin
- Department of Radiology, Stanford University School of Medicine, Stanford, California 94305 and Department of Medicine (Biomedical Informatics Research), Stanford University School of Medicine, Stanford, California 94305
| |
Collapse
|
22
|
Khan AM, Rajpoot N, Treanor D, Magee D. A nonlinear mapping approach to stain normalization in digital histopathology images using image-specific color deconvolution. IEEE Trans Biomed Eng 2015; 61:1729-38. [PMID: 24845283 DOI: 10.1109/tbme.2014.2303294] [Citation(s) in RCA: 255] [Impact Index Per Article: 28.3] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/11/2022]
Abstract
Histopathology diagnosis is based on visual examination of the morphology of histological sections under a microscope. With the increasing popularity of digital slide scanners, decision support systems based on the analysis of digital pathology images are in high demand. However, computerized decision support systems are fraught with problems that stem from color variations in tissue appearance due to variation in tissue preparation, variation in stain reactivity from different manufacturers/batches, user or protocol variation, and the use of scanners from different manufacturers. In this paper, we present a novel approach to stain normalization in histopathology images. The method is based on nonlinear mapping of a source image to a target image using a representation derived from color deconvolution. Color deconvolution is a method to obtain stain concentration values when the stain matrix, describing how the color is affected by the stain concentration, is given. Rather than relying on standard stain matrices, which may be inappropriate for a given image, we propose the use of a color-based classifier that incorporates a novel stain color descriptor to calculate image-specific stain matrix. In order to demonstrate the efficacy of the proposed stain matrix estimation and stain normalization methods, they are applied to the problem of tumor segmentation in breast histopathology images. The experimental results suggest that the paradigm of color normalization, as a preprocessing step, can significantly help histological image analysis algorithms to demonstrate stable performance which is insensitive to imaging conditions in general and scanner variations in particular.
Collapse
|
23
|
|
24
|
Lyon HO, Horobin RW. Standardization and standards for dyes and stains used in biology and medicine. Biotech Histochem 2009; 82:1-11. [PMID: 17510808 DOI: 10.1080/10520290601116590] [Citation(s) in RCA: 13] [Impact Index Per Article: 0.9] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/23/2022] Open
Abstract
The reasons for standardization and the preparation of standards for dyes and stains are presented. The national, regional and international standardization agencies are described in detail prior to a consideration of why standards should be prepared for the field of biomedical staining. An outline is given of the standards for information supplied by the manufacturer for in vitro diagnostic reagents for biological staining, published by the European Committee for Standardization (CEN) in 1999, and in a practically unaltered form by the International Organization for Standards (ISO) in 2002. A more detailed description is given in Appendix A of the rules governing the work of the European Committee for Standardization (CEN) and of the technical co-operation between the International Organization for Standardization (ISO) and CEN (The Vienna Agreement). Finally, a short list of abbreviations used by CEN and ISO is included as Appendix B.
Collapse
Affiliation(s)
- H O Lyon
- Department of Pathology 134, H:S, Hvidovre Hospital, Hvidovre, Denmark.
| | | |
Collapse
|
25
|
Das CM, Becker F, Vernon S, Noshari J, Joyce C, Gascoyne PRC. Dielectrophoretic segregation of different human cell types on microscope slides. Anal Chem 2005; 77:2708-19. [PMID: 15859584 PMCID: PMC1542197 DOI: 10.1021/ac048196z] [Citation(s) in RCA: 67] [Impact Index Per Article: 3.5] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/28/2022]
Abstract
A new method for preparing cells for microscopic examination is presented in which cell mixtures are fractionated by dielectrophoretic forces and simultaneously collected into characteristic zones on slides. The method traps cells directly from the suspending medium onto the slide, reducing cell loss. Furthermore, it exploits differences in the dielectric properties of the cells, which sensitively reflect their morphology. Because different cell types are trapped in characteristic zones on the slide, the technique represents an advance over existing methods for slide preparation, such as centrifugation and smears where cells are randomly distributed. In particular, the new method should aid in the detection of rare and anomalous cell subpopulations that might otherwise go unnoticed against a high background of normal cells. As well as being suitable for traditional microscopic examination and automated slide scanning approaches, it is compatible with histochemical and immunochemical techniques, as well as emerging molecular and proteomic methods. This paper describes the rationale and design of this so-called electrosmear instrumentation and shows experimental results that verify the theory and applicability of the method with model cell lines and normal peripheral blood subpopulations.
Collapse
Affiliation(s)
- Chandra M Das
- Department of Molecular Pathology, University of Texas M. D. Anderson Cancer Center, 1515 Holcombe Boulevard, Houston, Texas 77030, USA
| | | | | | | | | | | |
Collapse
|
26
|
Wittekind D. Traditional staining for routine diagnostic pathology including the role of tannic acid. 1. Value and limitations of the hematoxylin-eosin stain. Biotech Histochem 2004; 78:261-70. [PMID: 14989644 DOI: 10.1080/10520290310001633725] [Citation(s) in RCA: 46] [Impact Index Per Article: 2.3] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/26/2022] Open
Abstract
The components of the hematoxylin and eosin (H & E) stain (i.e. hemalum and eosin Y), their contributions to the typical staining pattern, and the reasons why the H & E stains are the preferred oversight stains for routine diagnostic histopathology are discussed. The essential diagnostic significance of effective nuclear staining by hemalum, providing information on nuclear morphology and texture, is emphasized; as is the ironic advantage for routine diagnostic histopathology of the limited range of colors provided by H & E staining, that allows recognition of significant features under low microscopic magnifications. Standardization of hemalum is considered, along with probable reasons why users show resistance to such a concept. Counterstaining with anionic (acid) dyes is discussed, as is the important phenomenon of contrast. The particular advantages and disadvantages of eosin Y and phloxin B as counterstains to hemalum are outlined. The concept of an "ideal routine histological stain" is considered, and H & E is compared to such an ideal case. Finally, deficiencies of H & E staining are discussed, and a program to develop an improved oversight stain is introduced.
Collapse
Affiliation(s)
- D Wittekind
- Institute of Anatomy and Cell Biology, University of Freiburg, P. O. Box 111, D-79001 Freiburg, Germany
| |
Collapse
|
27
|
Abstract
The purpose of biological staining is to obtain specimens of biological material that can be assessed in the microscope. These specimens are influenced by all processes from removal from the intact organism to mounting on the microscopic slide. To achieve comparable results with various techniques for biological staining, standardization of all procedures and reagents is mandatory. In this paper, I focus particularly on dyes and consider the possibilities for obtaining standardized dyes. In general practice, most biological staining takes place with available commercial dyes. These dyes may or may not have been subjected to quality assessment either internally by the producer or vendor or externally by independent investigators or organizations such as the Biological Stain Commission. Concerted attempts at standardization in Europe are discussed. The latest results of this work, the European standard EN 12376, is presented. This standard is concerned with information supplied by the manufacturer with in vitro diagnostic reagents for biological staining. The standard has been prepared by a Working Group on Staining in Biology under Technical Committee 140, In Vitro Medical Devices, of the European committee for standardization, CEN.
Collapse
Affiliation(s)
- H Lyon
- Department of Pathology 134, Hvidovre Hospital, University of Copenhagen, Denmark
| |
Collapse
|
28
|
Haber N. Electromolecular propulsion (EMP): a rapid, simple method for analyzing dyes used in microscopy. Biotech Histochem 1998; 73:59-70. [PMID: 9605620 DOI: 10.3109/10520299809140508] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/13/2022] Open
Abstract
The electrokinetic molecular effect known as electromolecular propulsion (EMP) was used to examine a variety of commonly used dyes that are used as biological stains. The dyes were electrokinetically mobilized using several organic solvent mixtures on a variety of thin layer substrates. Analysis was completed in seconds to minutes. Nearly all dyes tested separated into multiple colored components. The solvents and substrates used were well suited for qualitatively analyzing a broad variety of hydrophilic and lipophilic colorants. Speed, resolving power and operational simplicity of this technique make it convenient for effectively "fingerprinting" for consistency in dye products. EMP may be broadly applied to study quality control of dyes and their stains. Components not ordinarily known to be present can be readily detected using this technology. EMP can complement or offer a versatile alternative to existing analytical methods.
Collapse
Affiliation(s)
- N Haber
- Haber Incorporated, Towaco, New Jersey 07082, USA
| |
Collapse
|
29
|
Abstract
A recently developed preparative technique, pH-zone-refining countercurrent chromatography (CCC), separates organic acids and bases according to their pKa and hydrophobicity. The hydrodynamic mechanism of pH-zone-refining CCC is illustrated in two elution modes along with a simple mathematical analysis. Separations include acidic and basic derivatives of amino acids and peptides, hydroxyxanthene dyes, alkaloids, indole auxins, and structural and geometrical isomers. In addition, recently developed affinity ligand separations of enantiomers, polar catecholamines, and free peptides are also described. Technical guidance is provided for interested users so that they can conduct a systematic search for the optimum solvent system and experimental conditions. Advantages as well as limitations of the present technique are discussed.
Collapse
Affiliation(s)
- Y Ito
- Laboratory of Biophysical Chemistry, National Heart, Lung, and Blood Institute, National Institutes of Health, Bethesda, MD 20892-1676, USA
| | | |
Collapse
|