Perceptual clustering for automatic hotspot detection from Ki-67-stained neuroendocrine tumour images

Exploration of Digital Image Analysis for Ki67 Quantification in the Grading of Medullary Thyroid Carcinoma: A Pilot Study with 85 Cases

BACKGROUND

Although uncommon, medullary thyroid carcinoma (MTC) accounts for a significant proportion of thyroid cancer deaths. Recent studies have validated the two-tier International Medullary Thyroid Carcinoma Grading System (IMTCGS) to predict clinical outcomes. A 5% Ki67 proliferative index (Ki67PI) cut-off separates low-grade from high-grade MTC. In this study, we compared digital image analysis (DIA) to manual counting (MC) for determining the Ki67PI in a MTC cohort, and explored the challenges encountered.

METHODS

Available slides from 85 MTCs were reviewed by two pathologists. The Ki67PI was documented by immunohistochemistry for each case, scanned with the Aperio® slide scanner at 40× magnification, and quantified using the QuPath® DIA platform. The same hotspots were screenshot, printed in color, and blindly counted. For each case, over 500 MTC cells were counted. Each MTC was graded using IMTCGS criteria.

RESULTS

In our MTC cohort (n = 85), 84.7 and 15.3% were low- and high-grade with the IMTCGS. In the entire cohort, QuPath® DIA performed well (R2 = 0.9891) but appeared to undercall compared to MC. QuPath® performed better in high-grade cases (R2 = 0.99) compared to low-grade cases (R2 = 0.7071). Overall, Ki67PI determined with either MC or DIA did not affect IMTCGS grade. Encountered DIA challenges include optimizing cell detection, overlapping nuclei, and tissue artifacts. Encountered MC challenges include background staining, morphologic overlap with normal elements, and counting time.

CONCLUSION

Our study highlights the utility of DIA in quantifying Ki67PI for MTC and can serve as an adjunct for grading in conjunction with the other criteria of mitotic activity and necrosis.

Implementation of deep learning in liver pathology optimizes diagnosis of benign lesions and adenocarcinoma metastasis

INTRODUCTION

Differentiation of histologically similar structures in the liver, including anatomical structures, benign bile duct lesions, or common types of liver metastases, can be challenging with conventional histological tissue sections alone. Accurate histopathological classification is paramount for the diagnosis and adequate treatment of the disease. Deep learning algorithms have been proposed for objective and consistent assessment of digital histopathological images.

MATERIALS AND METHODS

In the present study, we trained and evaluated deep learning algorithms based on the EfficientNetV2 and ResNetRS architectures to discriminate between different histopathological classes. For the required dataset, specialized surgical pathologists annotated seven different histological classes, including different non-neoplastic anatomical structures, benign bile duct lesions, and liver metastases from colorectal and pancreatic adenocarcinoma in a large patient cohort. Annotation resulted in a total of 204.159 image patches, followed by discrimination analysis using our deep learning models. Model performance was evaluated on validation and test data using confusion matrices.

RESULTS

Evaluation of the test set based on tiles and cases revealed overall highly satisfactory prediction capability of our algorithm for the different histological classes, resulting in a tile accuracy of 89% (38 413/43 059) and case accuracy of 94% (198/211). Importantly, the separation of metastasis versus benign lesions was certainly confident on case level, confirming the classification model performed with high diagnostic accuracy. Moreover, the whole curated raw data set is made publically available.

CONCLUSIONS

Deep learning is a promising approach in surgical liver pathology supporting decision making in personalized medicine.

Ki-67 assessment of pancreatic neuroendocrine neoplasms: Systematic review and meta-analysis of manual vs. digital pathology scoring

Ki-67 assessment is a key step in the diagnosis of neuroendocrine neoplasms (NENs) from all anatomic locations. Several challenges exist related to quantifying the Ki-67 proliferation index due to lack of method standardization and inter-reader variability. The application of digital pathology coupled with machine learning has been shown to be highly accurate and reproducible for the evaluation of Ki-67 in NENs. We systematically reviewed all published studies on the subject of Ki-67 assessment in pancreatic NENs (PanNENs) employing digital image analysis (DIA). The most common advantages of DIA were improvement in the standardization and reliability of Ki-67 evaluation, as well as its speed and practicality, compared to the current gold standard approach of manual counts from captured images, which is cumbersome and time consuming. The main limitations were attributed to higher costs, lack of widespread availability (as of yet), operator qualification and training issues (if it is not done by pathologists), and most importantly, the drawback of image algorithms counting contaminating non-neoplastic cells and other signals like hemosiderin. However, solutions are rapidly developing for all of these challenging issues. A comparative meta-analysis for DIA versus manual counting shows very high concordance (global coefficient of concordance: 0.94, 95% CI: 0.83-0.98) between these two modalities. These findings support the widespread adoption of validated DIA methods for Ki-67 assessment in PanNENs, provided that measures are in place to ensure counting of only tumor cells either by software modifications or education of non-pathologist operators, as well as selection of standard regions of interest for analysis. NENs, being cellular and monotonous neoplasms, are naturally more amenable to Ki-67 assessment. However, lessons of this review may be applicable to other neoplasms where proliferation activity has become an integral part of theranostic evaluation including breast, brain, and hematolymphoid neoplasms.

Detection of Ki67 Hot-Spots of Invasive Breast Cancer Based on Convolutional Neural Networks Applied to Mutual Information of H&E and Ki67 Whole Slide Images

Ki67 hot-spot detection and its evaluation in invasive breast cancer regions play a significant role in routine medical practice. The quantification of cellular proliferation assessed by Ki67 immunohistochemistry is an established prognostic and predictive biomarker that determines the choice of therapeutic protocols. In this paper, we present three deep learning-based approaches to automatically detect and quantify Ki67 hot-spot areas by means of the Ki67 labeling index. To this end, a dataset composed of 100 whole slide images (WSIs) belonging to 50 breast cancer cases (Ki67 and H&E WSI pairs) was used. Three methods based on CNN classification were proposed and compared to create the tumor proliferation map. The best results were obtained by applying the CNN to the mutual information acquired from the color deconvolution of both the Ki67 marker and the H&E WSIs. The overall accuracy of this approach was 95%. The agreement between the automatic Ki67 scoring and the manual analysis is promising with a Spearman’s ρ correlation of 0.92. The results illustrate the suitability of this CNN-based approach for detecting hot-spots areas of invasive breast cancer in WSI.

Automated Computational Detection, Quantitation, and Mapping of Mitosis in Whole-Slide Images for Clinically Actionable Surgical Pathology Decision Support

Background

Determining mitotic index by counting mitotic figures (MFs) microscopically from tumor areas with most abundant MF (hotspots [HS]) produces a prognostically useful tumor grading biomarker. However, interobserver concordance identifying MF and HS can be poorly reproducible. Immunolabeling MF, coupled with computer-automated counting by image analysis, can improve reproducibility. A computational system for obtaining MF values across digitized whole-slide images (WSIs) was sought that would minimize impact of artifacts, generate values clinically relatable to counting ten high-power microscopic fields of view typical in conventional microscopy, and that would reproducibly map HS topography.

Materials and Methods

Relatively low-resolution WSI scans (0.50 μm/pixel) were imported in grid-tile format for feature-based MF segmentation, from naturally occurring canine melanomas providing a wide range of proliferative activity. MF feature extraction conformed to anti-phospho-histone H3-immunolabeled mitotic (M) phase cells. Computer vision image processing was established to subtract key artifacts, obtain MF counts, and employ rotationally invariant feature extraction to map MF topography.

Results

The automated topometric HS (TMHS) algorithm identified mitotic HS and mapped select tissue tiles with greatest MF counts back onto WSI thumbnail images to plot HS topographically. Influence of dye, pigment, and extraneous structure artifacts was minimized. TMHS diagnostic decision support included image overlay graphics of HS topography, as well as a spreadsheet and plot of tile-based MF count values. TMHS performance was validated examining both mitotic HS counting and mapping functions. Significantly correlated TMHS MF mapping and metrics were demonstrated using repeat analysis with WSI in different orientation (R ² = 0.9916) and by agreement with a pathologist (R ² = 0.8605) as well as through assessment of counting function using an independently tuned object counting algorithm (OCA) (R ² = 0.9482). Limits of agreement analysis support method interchangeability. MF counts obtained led to accurate patient survival prediction in all (n = 30) except one case. By contrast, more variable performance was documented when several pathologists examined similar cases using microscopy (pair-wise correlations, rho range = 0.7597-0.9286).

Conclusions

Automated TMHS MF segmentation and feature engineering performance were interchangeable with both observer and OCA in digital mode. Moreover, enhanced HS location accuracy and superior method reproducibility were achieved using the automated TMHS algorithm compared to the current practice employing clinical microscopy.

Pathological image compression for big data image analysis: Application to hotspot detection in breast cancer

In this paper, we propose a pathological image compression framework to address the needs of Big Data image analysis in digital pathology. Big Data image analytics require analysis of large databases of high-resolution images using distributed storage and computing resources along with transmission of large amounts of data between the storage and computing nodes that can create a major processing bottleneck. The proposed image compression framework is based on the JPEG2000 Interactive Protocol and aims to minimize the amount of data transfer between the storage and computing nodes as well as to considerably reduce the computational demands of the decompression engine. The proposed framework was integrated into hotspot detection from images of breast biopsies, yielding considerable reduction of data and computing requirements.

Identifying tumor in pancreatic neuroendocrine neoplasms from Ki67 images using transfer learning

The World Health Organization (WHO) has clear guidelines regarding the use of Ki67 index in defining the proliferative rate and assigning grade for pancreatic neuroendocrine tumor (NET). WHO mandates the quantification of Ki67 index by counting at least 500 positive tumor cells in a hotspot. Unfortunately, Ki67 antibody may stain both tumor and non-tumor cells as positive depending on the phase of the cell cycle. Likewise, the counter stain labels both tumor and non-tumor as negative. This non-specific nature of Ki67 stain and counter stain therefore hinders the exact quantification of Ki67 index. To address this problem, we present a deep learning method to automatically differentiate between NET and non-tumor regions based on images of Ki67 stained biopsies. Transfer learning was employed to recognize and apply relevant knowledge from previous learning experiences to differentiate between tumor and non-tumor regions. Transfer learning exploits a rich set of features previously used to successfully categorize non-pathology data into 1,000 classes. The method was trained and validated on a set of whole-slide images including 33 NETs subject to Ki67 immunohistochemical staining using a leave-one-out cross-validation. When applied to 30 high power fields (HPF) and assessed against a gold standard (evaluation by two expert pathologists), the method resulted in a high sensitivity of 97.8% and specificity of 88.8%. The deep learning method developed has the potential to reduce pathologists’ workload by directly identifying tumor boundaries on images of Ki67 stained slides. Moreover, it has the potential to replace sophisticated and expensive imaging methods which are recently developed for identification of tumor boundaries in images of Ki67-stained NETs.

An Advanced Deep Learning Approach for Ki-67 Stained Hotspot Detection and Proliferation Rate Scoring for Prognostic Evaluation of Breast Cancer

Being a non-histone protein, Ki-67 is one of the essential biomarkers for the immunohistochemical assessment of proliferation rate in breast cancer screening and grading. The Ki-67 signature is always sensitive to radiotherapy and chemotherapy. Due to random morphological, color and intensity variations of cell nuclei (immunopositive and immunonegative), manual/subjective assessment of Ki-67 scoring is error-prone and time-consuming. Hence, several machine learning approaches have been reported; nevertheless, none of them had worked on deep learning based hotspots detection and proliferation scoring. In this article, we suggest an advanced deep learning model for computerized recognition of candidate hotspots and subsequent proliferation rate scoring by quantifying Ki-67 appearance in breast cancer immunohistochemical images. Unlike existing Ki-67 scoring techniques, our methodology uses Gamma mixture model (GMM) with Expectation-Maximization for seed point detection and patch selection and deep learning, comprises with decision layer, for hotspots detection and proliferation scoring. Experimental results provide 93% precision, 0.88% recall and 0.91% F-score value. The model performance has also been compared with the pathologists’ manual annotations and recently published articles. In future, the proposed deep learning framework will be highly reliable and beneficial to the junior and senior pathologists for fast and efficient Ki-67 scoring.

An adaptive positivity thresholding method for automated Ki67 hotspot detection (AKHoD) in breast cancer biopsies

The proliferative activity of breast cancer tissue can be estimated using the Ki67 biomarker. The percentage of positivity of such biomarker is correlated with proliferation and consequently with the prognosis of a breast tumor. Ki67 marked tissue samples are analyzed by an experienced pathologist who identifies the most active areas of tumor cell proliferation called hotspots, and estimates the positivity of each case. A method for the Automated Ki67 Hotspot Detection (AKHoD) is presented in this work. The main objective of the AKHoD method is to automatically and efficiently provide the pathologist with suggestions about Ki67 hotspot areas as a decision support. The input of AKHoD is a digital slide that is divided in tiles. For each tile, AKHoD provides a rough estimate of positivity and cellularity, summarized in very low resolution positivity and cellularity images. In a second step, an adaptive thresholding is applied to such positivity image to identify the most positive connected and convex areas, within cellularity limits set by current guidelines (that is, 500-2000). The method has been preliminarily validated on 50 digital slides for which three expert pathologists provided gold standard hotspots. 82% of the gold standard hotspots have been successfully recognized by the system, spending an average of 54s per slide. While further validation is needed taking into account also patients follow-up, this first experimentation suggests that the proposed method could be adequate for supporting the pathologist in hotspot detection.

Spatial intratumoral heterogeneity of proliferation in immunohistochemical images of solid tumors

PURPOSE

The interactions of neoplastic cells with each other and the microenvironment are complex. To understand intratumoral heterogeneity, subtle differences should be quantified. Main factors contributing to heterogeneity include the gradient ischemic level within neoplasms, action of microenvironment, mechanisms of intercellular transfer of genetic information, and differential mechanisms of modifications of genetic material/proteins. This may reflect on the expression of biomarkers in the context of prognosis/stratification. Hence, a rigorous approach for assessing the spatial intratumoral heterogeneity of histological biomarker expression with accuracy and reproducibility is required, since patterns in immunohistochemical images can be challenging to identify and describe.

METHODS

A quantitative method that is useful for characterizing complex irregular structures is lacunarity; it is a multiscale technique that exhaustively samples the image, while the decay of its index as a function of window size follows characteristic patterns for different spatial arrangements. In histological images, lacunarity provides a useful measure for the spatial organization of a biomarker when a sampling scheme is employed and relevant features are computed. The proposed approach quantifies the segmented proliferative cells and not the textural content of the histological slide, thus providing a more realistic measure of heterogeneity within the sample space of the tumor region. The aim is to investigate in whole sections of primary pancreatic neuroendocrine neoplasms (pNENs), using whole-slide imaging and image analysis, the spatial intratumoral heterogeneity of Ki-67 immunostains. Unsupervised learning is employed to verify that the approach can partition the tissue sections according to distributional heterogeneity.

RESULTS

The architectural complexity of histological images has shown that single measurements are often insufficient. Inhomogeneity of distribution depends not only on percentage content of proliferation phase but also on how the phase fills the space. Lacunarity curves demonstrate variations in the sampled image sections. Since the spatial distribution of proliferation in each case is different, the width of the curves changes too. Image sections that have smaller numerical variations in the computed features correspond to neoplasms with spatially homogeneous proliferation, while larger variations correspond to cases where proliferation shows various degrees of clumping. Grade 1 (uniform/nonuniform: 74%/26%) and grade 3 (uniform: 100%) pNENs demonstrate a more homogeneous proliferation with grade 1 neoplasms being more variant, while grade 2 tumor regions render a more diverse landscape (50%/50%). Hence, some cases show an increased degree of spatial heterogeneity comparing to others with similar grade. Whether this is a sign of different tumor biology and an association with a more benign/malignant clinical course needs to be investigated further. The extent and range of spatial heterogeneity has the potential to be evaluated as a prognostic marker.

CONCLUSIONS

The association with tumor grade as well as the rationale that the methodology reflects true tumor architecture supports the technical soundness of the method. This reflects a general approach which is relevant to other solid tumors and biomarkers. Drawing upon the merits of computational biomedicine, the approach uncovers salient features for use in future studies of clinical relevance.

A Computational Framework to Detect Normal and Tuberculosis Infected Lung from H&E-stained Whole Slide Images

Accurate detection and quantification of normal lung tissue in the context of Mycobacterium tuberculosis infection is of interest from a biological perspective. The automatic detection and quantification of normal lung will allow the biologists to focus more intensely on regions of interest within normal and infected tissues. We present a computational framework to extract individual tissue sections from whole slide images having multiple tissue sections. It automatically detects the background, red blood cells and handwritten digits to bring efficiency as well as accuracy in quantification of tissue sections. For efficiency, we model our framework with logical and morphological operations as they can be performed in linear time. We further divide these individual tissue sections into normal and infected areas using deep neural network. The computational framework was trained on 60 whole slide images. The proposed computational framework resulted in an overall accuracy of 99.2% when extracting individual tissue sections from 120 whole slide images in the test dataset. The framework resulted in a relatively higher accuracy (99.7%) while classifying individual lung sections into normal and infected areas. Our preliminary findings suggest that the proposed framework has good agreement with biologists on how define normal and infected lung areas.

Quantitative tumor heterogeneity assessment on a nuclear population basis

Immunohistochemistry Ki-67 stain is widely used for visualizing cell proliferation. The common method for scoring the proliferation is to manually select and score a hot spot. This method is time-consuming and will often not give reproducible results due to subjective selection of the hotspots and subjective scoring. An automatic hotspot detection and proliferative index scoring would be time-saving, make the determination of the Ki-67 score easier and minimize the uncertainty of the score by introducing a more objective and standardized score. Tissue Micro Array cores stained for Ki-67 and their neighbor slide stained for Pan Cytokeratin were aligned and Ki-67 positive and negative nuclei were identified inside tumor regions. A heatmap was calculated based on these and illustrates the distribution of the heterogenous response of Ki-67 positive nuclei in the tumor tissue. An automatic hot spot detection was developed and the Ki-67 score was calculated. All scores were compared with scores provided by a pathologist using linear regression models. No significant difference was found between the Ki-67 scores guided by the developed heatmap and the scores provided by a pathologist. For comparison, scores were also calculated at a random place outside the hot spot and these scores were found to be significantly different from the pathologist scores. A heatmap visualizing the heterogeneity in tumor tissue expressed by Ki-67 was developed and used for an automatic identification of hot spots in which a Ki-67 score was calculated. The Ki-67 scores did not differ significantly from scores provided by a pathologist. © 2017 International Society for Advancement of Cytometry.

Advancing Clinicopathologic Diagnosis of High-risk Neuroblastoma Using Computerized Image Analysis and Proteomic Profiling

A subset of patients with neuroblastoma are at extremely high risk for treatment failure, though they are not identifiable at diagnosis and therefore have the highest mortality with conventional treatment approaches. Despite tremendous understanding of clinical and biological features that correlate with prognosis, neuroblastoma at ultra-high risk for treatment failure remains a diagnostic challenge. As a first step towards improving prognostic risk stratification within the high-risk group of patients, we determined the feasibility of using computerized image analysis and proteomic profiling on single slides from diagnostic tissue specimens. After expert pathologist review of tumor sections to ensure quality and representative material input, we evaluated multiple regions of single slides as well as multiple sections from different patients' tumors using computational histologic analysis and semiquantitative proteomic profiling. We found that both approaches determined that intertumor heterogeneity was greater than intratumor heterogeneity. Unbiased clustering of samples was greatest within a tumor, suggesting a single section can be representative of the tumor as a whole. There is expected heterogeneity between tumor samples from different individuals with a high degree of similarity among specimens derived from the same patient. Both techniques are novel to supplement pathologist review of neuroblastoma for refined risk stratification, particularly since we demonstrate these results using only a single slide derived from what is usually a scarce tissue resource. Due to limitations of traditional approaches for upfront stratification, integration of new modalities with data derived from one section of tumor hold promise as tools to improve outcomes.