High-throughput detection of prostate cancer in histological sections using probabilistic pairwise Markov models

An update on computational pathology tools for genitourinary pathology practice: A review paper from the Genitourinary Pathology Society (GUPS)

Machine learning has been leveraged for image analysis applications throughout a multitude of subspecialties. This position paper provides a perspective on the evolutionary trajectory of practical deep learning tools for genitourinary pathology through evaluating the most recent iterations of such algorithmic devices. Deep learning tools for genitourinary pathology demonstrate potential to enhance prognostic and predictive capacity for tumor assessment including grading, staging, and subtype identification, yet limitations in data availability, regulation, and standardization have stymied their implementation.

Applications of Digital and Computational Pathology and Artificial Intelligence in Genitourinary Pathology Diagnostics

As machine learning (ML) solutions for genitourinary pathology image analysis are fostered by a progressively digitized laboratory landscape, these integrable modalities usher in a revolution in histopathological diagnosis. As technology advances, limitations stymying clinical artificial intelligence (AI) will not be extinguished without thorough validation and interrogation of ML tools by pathologists and regulatory bodies alike. ML solutions deployed in clinical settings for applications in prostate pathology yield promising results. Recent breakthroughs in clinical artificial intelligence for genitourinary pathology demonstrate unprecedented generalizability, heralding prospects for a future in which AI-driven assistive solutions may be seen as laboratory faculty, rather than novelty.

A deep learning network for Gleason grading of prostate biopsies using EfficientNet

Objectives

The most crucial part in the diagnosis of cancer is severity grading. Gleason’s score is a widely used grading system for prostate cancer. Manual examination of the microscopic images and grading them is tiresome and consumes a lot of time. Hence to automate the Gleason grading process, a novel deep learning network is proposed in this work.

Methods

In this work, a deep learning network for Gleason grading of prostate cancer is proposed based on EfficientNet architecture. It applies a compound scaling method to balance the dimensions of the underlying network. Also, an additional attention branch is added to EfficientNet-B7 for precise feature weighting.

Result

To the best of our knowledge, this is the first work that integrates an additional attention branch with EfficientNet architecture for Gleason grading. The proposed models were trained using H&E-stained samples from prostate cancer Tissue Microarrays (TMAs) in the Harvard Dataverse dataset.

Conclusions

The proposed network was able to outperform the existing methods and it achieved an Kappa score of 0.5775.

Pathomics in urology

PURPOSE OF REVIEW

Pathomics, the fusion of digitalized pathology and artificial intelligence, is currently changing the landscape of medical pathology and biologic disease classification. In this review, we give an overview of Pathomics and summarize its most relevant applications in urology.

RECENT FINDINGS

There is a steady rise in the number of studies employing Pathomics, and especially deep learning, in urology. In prostate cancer, several algorithms have been developed for the automatic differentiation between benign and malignant lesions and to differentiate Gleason scores. Furthermore, several applications have been developed for the automatic cancer cell detection in urine and for tumor assessment in renal cancer. Despite the explosion in research, Pathomics is not fully ready yet for widespread clinical application.

SUMMARY

In prostate cancer and other urologic pathologies, Pathomics is avidly being researched with commercial applications on the close horizon. Pathomics is set to improve the accuracy, speed, reliability, cost-effectiveness and generalizability of pathology, especially in uro-oncology.

Machine Learning Methods for Histopathological Image Analysis: A Review

Histopathological images (HIs) are the gold standard for evaluating some types of tumors for cancer diagnosis. The analysis of such images is time and resource-consuming and very challenging even for experienced pathologists, resulting in inter-observer and intra-observer disagreements. One of the ways of accelerating such an analysis is to use computer-aided diagnosis (CAD) systems. This paper presents a review on machine learning methods for histopathological image analysis, including shallow and deep learning methods. We also cover the most common tasks in HI analysis, such as segmentation and feature extraction. Besides, we present a list of publicly available and private datasets that have been used in HI research.

Multistage Segmentation of Prostate Cancer Tissues Using Sample Entropy Texture Analysis

In this study, a multistage segmentation technique is proposed that identifies cancerous cells in prostate tissue samples. The benign areas of the tissue are distinguished from the cancerous regions using the texture of glands. The texture is modeled based on wavelet packet features along with sample entropy values. In a multistage segmentation process, the mean-shift algorithm is applied on the pre-processed images to perform a coarse segmentation of the tissue. Wavelet packets are employed in the second stage to obtain fine details of the structured shape of glands. Finally, the texture of the gland is modeled by the sample entropy values, which identifies epithelial regions from stroma patches. Although there are three stages of the proposed algorithm, the computation is fast as wavelet packet features and sample entropy values perform robust modeling for the required regions of interest. A comparative analysis with other state-of-the-art texture segmentation techniques is presented and dice ratios are computed for the comparison. It has been observed that our algorithm not only outperforms other techniques, but, by introducing sample entropy features, identification of cancerous regions of tissues is achieved with 90% classification accuracy, which shows the robustness of the proposed algorithm.

Deep learning in digital pathology image analysis: a survey

Deep learning (DL) has achieved state-of-the-art performance in many digital pathology analysis tasks. Traditional methods usually require hand-crafted domain-specific features, and DL methods can learn representations without manually designed features. In terms of feature extraction, DL approaches are less labor intensive compared with conventional machine learning methods. In this paper, we comprehensively summarize recent DL-based image analysis studies in histopathology, including different tasks (e.g., classification, semantic segmentation, detection, and instance segmentation) and various applications (e.g., stain normalization, cell/gland/region structure analysis). DL methods can provide consistent and accurate outcomes. DL is a promising tool to assist pathologists in clinical diagnosis.

Automatic cancer detection on digital histopathology images of mid-gland radical prostatectomy specimens

Purpose: Automatic cancer detection on radical prostatectomy (RP) sections facilitates graphical and quantitative surgical pathology reporting, which can potentially benefit postsurgery follow-up care and treatment planning. It can also support imaging validation studies using a histologic reference standard and pathology research studies. This problem is challenging due to the large sizes of digital histopathology whole-mount whole-slide images (WSIs) of RP sections and staining variability across different WSIs. Approach: We proposed a calibration-free adaptive thresholding algorithm, which compensates for staining variability and yields consistent tissue component maps (TCMs) of the nuclei, lumina, and other tissues. We used and compared three machine learning methods for classifying each cancer versus noncancer region of interest (ROI) throughout each WSI: (1) conventional machine learning methods and 14 texture features extracted from TCMs, (2) transfer learning with pretrained AlexNet fine-tuned by TCM ROIs, and (3) transfer learning with pretrained AlexNet fine-tuned with raw image ROIs. Results: The three methods yielded areas under the receiver operating characteristic curve of 0.96, 0.98, and 0.98, respectively, in leave-one-patient-out cross validation using 1.3 million ROIs from 286 mid-gland whole-mount WSIs from 68 patients. Conclusion: Transfer learning with the use of TCMs demonstrated state-of-the-art overall performance and is more stable with respect to sample size across different tissue types. For the tissue types involving Gleason 5 (most aggressive) cancer, it achieved the best performance compared to the other tested methods. This tool can be translated to clinical workflow to assist graphical and quantitative pathology reporting for surgical specimens upon further multicenter validation.

Histologic tissue components provide major cues for machine learning-based prostate cancer detection and grading on prostatectomy specimens

Automatically detecting and grading cancerous regions on radical prostatectomy (RP) sections facilitates graphical and quantitative pathology reporting, potentially benefitting post-surgery prognosis, recurrence prediction, and treatment planning after RP. Promising results for detecting and grading prostate cancer on digital histopathology images have been reported using machine learning techniques. However, the importance and applicability of those methods have not been fully investigated. We computed three-class tissue component maps (TCMs) from the images, where each pixel was labeled as nuclei, lumina, or other. We applied seven different machine learning approaches: three non-deep learning classifiers with features extracted from TCMs, and four deep learning, using transfer learning with the 1) TCMs, 2) nuclei maps, 3) lumina maps, and 4) raw images for cancer detection and grading on whole-mount RP tissue sections. We performed leave-one-patient-out cross-validation against expert annotations using 286 whole-slide images from 68 patients. For both cancer detection and grading, transfer learning using TCMs performed best. Transfer learning using nuclei maps yielded slightly inferior overall performance, but the best performance for classifying higher-grade cancer. This suggests that 3-class TCMs provide the major cues for cancer detection and grading primarily using nucleus features, which are the most important information for identifying higher-grade cancer.

Glandular orientation and shape determined by computational pathology could identify aggressive tumor for early colon carcinoma: a triple-center study

BACKGROUND

Identifying the early-stage colon adenocarcinoma (ECA) patients who have lower risk cancer vs. the higher risk cancer could improve disease prognosis. Our study aimed to explore whether the glandular morphological features determined by computational pathology could identify high risk cancer in ECA via H&E images digitally.

METHODS

532 ECA patients retrospectively from 2 independent data centers, as well as 113 from The Cancer Genome Atlas (TCGA), were enrolled in this study. Four tissue microarrays (TMAs) were constructed across ECA hematoxylin and eosin (H&E) stained slides. 797 quantitative glandular morphometric features were extracted and 5 most prognostic features were identified using minimum redundancy maximum relevance to construct an image classifier. The image classifier was evaluated on D2/D3 = 223, D4 = 46, D5 = 113. The expression of Ki67 and serum CEA levels were scored on D3, aiming to explore the correlations between image classifier and immunohistochemistry data and serum CEA levels. The roles of clinicopathological data and ECAHBC were evaluated by univariate and multivariate analyses for prognostic value.

RESULTS

The image classifier could predict ECA recurrence (accuracy of 88.1%). ECA histomorphometric-based image classifier (ECAHBC) was an independent prognostic factor for poorer disease-specific survival [DSS, (HR = 9.65, 95% CI 2.15-43.12, P = 0.003)]. Significant correlations were observed between ECAHBC-positive patients and positivity of Ki67 labeling index (Ki67Li) and serum CEA.

CONCLUSION

Glandular orientation and shape could predict the high risk cancer in ECA and contribute to precision oncology. Computational pathology is emerging as a viable and objective means of identifying predictive biomarkers for cancer patients.

A Hybrid End-to-End Approach Integrating Conditional Random Fields into CNNs for Prostate Cancer Detection on MRI

Prostate Cancer (PCa) is the most common oncological disease in Western men. Even though a growing effort has been carried out by the scientific community in recent years, accurate and reliable automated PCa detection methods on multiparametric Magnetic Resonance Imaging (mpMRI) are still a compelling issue. In this work, a Deep Neural Network architecture is developed for the task of classifying clinically significant PCa on non-contrast-enhanced MR images. In particular, we propose the use of Conditional Random Fields as a Recurrent Neural Network (CRF-RNN) to enhance the classification performance of XmasNet, a Convolutional Neural Network (CNN) architecture specifically tailored to the PROSTATEx17 Challenge. The devised approach builds a hybrid end-to-end trainable network, CRF-XmasNet, composed of an initial CNN component performing feature extraction and a CRF-based probabilistic graphical model component for structured prediction, without the need for two separate training procedures. Experimental results show the suitability of this method in terms of classification accuracy and training time, even though the high-variability of the observed results must be reduced before transferring the resulting architecture to a clinical environment. Interestingly, the use of CRFs as a separate postprocessing method achieves significantly lower performance with respect to the proposed hybrid end-to-end approach. The proposed hybrid end-to-end CRF-RNN approach yields excellent peak performance for all the CNN architectures taken into account, but it shows a high-variability, thus requiring future investigation on the integration of CRFs into a CNN.

First-Stage Prostate Cancer Identification on Histopathological Images: Hand-Driven versus Automatic Learning

Analysis of histopathological image supposes the most reliable procedure to identify prostate cancer. Most studies try to develop computer aid-systems to face the Gleason grading problem. On the contrary, we delve into the discrimination between healthy and cancerous tissues in its earliest stage, only focusing on the information contained in the automatically segmented gland candidates. We propose a hand-driven learning approach, in which we perform an exhaustive hand-crafted feature extraction stage combining in a novel way descriptors of morphology, texture, fractals and contextual information of the candidates under study. Then, we carry out an in-depth statistical analysis to select the most relevant features that constitute the inputs to the optimised machine-learning classifiers. Additionally, we apply for the first time on prostate segmented glands, deep-learning algorithms modifying the popular VGG19 neural network. We fine-tuned the last convolutional block of the architecture to provide the model specific knowledge about the gland images. The hand-driven learning approach, using a nonlinear Support Vector Machine, reports a slight outperforming over the rest of experiments with a final multi-class accuracy of 0.876 ± 0.026 in the discrimination between false glands (artefacts), benign glands and Gleason grade 3 glands.

Comparison of Artificial Intelligence Techniques to Evaluate Performance of a Classifier for Automatic Grading of Prostate Cancer From Digitized Histopathologic Images

IMPORTANCE

Proper evaluation of the performance of artificial intelligence techniques in the analysis of digitized medical images is paramount for the adoption of such techniques by the medical community and regulatory agencies.

OBJECTIVES

To compare several cross-validation (CV) approaches to evaluate the performance of a classifier for automatic grading of prostate cancer in digitized histopathologic images and compare the performance of the classifier when trained using data from 1 expert and multiple experts.

DESIGN, SETTING, AND PARTICIPANTS

This quality improvement study used tissue microarray data (333 cores) from 231 patients who underwent radical prostatectomy at the Vancouver General Hospital between June 27, 1997, and June 7, 2011. Digitized images of tissue cores were annotated by 6 pathologists for 4 classes (benign and Gleason grades 3, 4, and 5) between December 12, 2016, and October 5, 2017. Patches of 192 µm2 were extracted from these images. There was no overlap between patches. A deep learning classifier based on convolutional neural networks was trained to predict a class label from among the 4 classes (benign and Gleason grades 3, 4, and 5) for each image patch. The classification performance was evaluated in leave-patches-out CV, leave-cores-out CV, and leave-patients-out 20-fold CV. The analysis was performed between November 15, 2018, and January 1, 2019.

MAIN OUTCOMES AND MEASURES

The classifier performance was evaluated by its accuracy, sensitivity, and specificity in detection of cancer (benign vs cancer) and in low-grade vs high-grade differentiation (Gleason grade 3 vs grades 4-5). The statistical significance analysis was performed using the McNemar test. The agreement level between pathologists and the classifier was quantified using a quadratic-weighted κ statistic.

RESULTS

On 333 tissue microarray cores from 231 participants with prostate cancer (mean [SD] age, 63.2 [6.3] years), 20-fold leave-patches-out CV resulted in mean (SD) accuracy of 97.8% (1.2%), sensitivity of 98.5% (1.0%), and specificity of 97.5% (1.2%) for classifying benign patches vs cancerous patches. By contrast, 20-fold leave-patients-out CV resulted in mean (SD) accuracy of 85.8% (4.3%), sensitivity of 86.3% (4.1%), and specificity of 85.5% (7.2%). Similarly, 20-fold leave-cores-out CV resulted in mean (SD) accuracy of 86.7% (3.7%), sensitivity of 87.2% (4.0%), and specificity of 87.7% (5.5%). Results of McNemar tests showed that the leave-patches-out CV accuracy, sensitivity, and specificity were significantly higher than those for both leave-patients-out CV and leave-cores-out CV. Similar results were observed for classifying low-grade cancer vs high-grade cancer. When trained on a single expert, the overall agreement in grading between pathologists and the classifier ranged from 0.38 to 0.58; when trained using the majority vote among all experts, it was 0.60.

CONCLUSIONS AND RELEVANCE

Results of this study suggest that in prostate cancer classification from histopathologic images, patch-wise CV and single-expert training and evaluation may lead to a biased estimation of classifier's performance. To allow reproducibility and facilitate comparison between automatic classification methods, studies in the field should evaluate their performance using patient-based CV and multiexpert data. Some of these conclusions may be generalizable to other histopathologic applications and to other applications of machine learning in medicine.

Comparing radiomic classifiers and classifier ensembles for detection of peripheral zone prostate tumors on T2-weighted MRI: a multi-site study

Background

For most computer-aided diagnosis (CAD) problems involving prostate cancer detection via medical imaging data, the choice of classifier has been largely ad hoc, or been motivated by classifier comparison studies that have involved large synthetic datasets. More significantly, it is currently unknown how classifier choices and trends generalize across multiple institutions, due to heterogeneous acquisition and intensity characteristics (especially when considering MR imaging data). In this work, we empirically evaluate and compare a number of different classifiers and classifier ensembles in a multi-site setting, for voxel-wise detection of prostate cancer (PCa) using radiomic texture features derived from high-resolution in vivo T2-weighted (T2w) MRI.

Methods

Twelve different supervised classifier schemes: Quadratic Discriminant Analysis (QDA), Support Vector Machines (SVMs), naïve Bayes, Decision Trees (DTs), and their ensemble variants (bagging, boosting), were compared in terms of classification accuracy as well as execution time. Our study utilized 85 prostate cancer T2w MRI datasets acquired from across 3 different institutions (1 for discovery, 2 for independent validation), from patients who later underwent radical prostatectomy. Surrogate ground truth for disease extent on MRI was established by expert annotation of pre-operative MRI through spatial correlation with corresponding ex vivo whole-mount histology sections. Classifier accuracy in detecting PCa extent on MRI on a per-voxel basis was evaluated via area under the ROC curve.

Results

The boosted DT classifier yielded the highest cross-validated AUC (= 0.744) for detecting PCa in the discovery cohort. However, in independent validation, the boosted QDA classifier was identified as the most accurate and robust for voxel-wise detection of PCa extent (AUCs of 0.735, 0.683, 0.768 across the 3 sites). The next most accurate and robust classifier was the single QDA classifier, which also enjoyed the advantage of significantly lower computation times compared to any of the other methods.

Conclusions

Our results therefore suggest that simpler classifiers (such as QDA and its ensemble variants) may be more robust, accurate, and efficient for prostate cancer CAD problems, especially in the context of multi-site validation.

Computer-Aided Laser Dissection: A Microdissection Workflow Leveraging Image Analysis Tools

Introduction

The development and application of new molecular diagnostic assays based on next-generation sequencing and proteomics require improved methodologies for procurement of target cells from histological sections. Laser microdissection can successfully isolate distinct cells from tissue specimens based on visual selection for many research and clinical applications. However, this can be a daunting task when a large number of cells are required for molecular analysis or when a sizeable number of specimens need to be evaluated.

Materials and Methods

To improve the efficiency of the cellular identification process, we describe a microdissection workflow that leverages recently developed and open source image analysis algorithms referred to as computer-aided laser dissection (CALD). CALD permits a computer algorithm to identify the cells of interest and drive the dissection process.

Results

We describe several "use cases" that demonstrate the integration of image analytic tools probabilistic pairwise Markov model, ImageJ, spatially invariant vector quantization (SIVQ), and eSeg onto the ThermoFisher Scientific ArcturusXT and Leica LMD7000 microdissection platforms.

Conclusions

The CALD methodology demonstrates the integration of image analysis tools with the microdissection workflow and shows the potential impact to clinical and life science applications.

Identifying the morphologic basis for radiomic features in distinguishing different Gleason grades of prostate cancer on MRI: Preliminary findings

Translation of radiomics into the clinic may require a more comprehensive understanding of the underlying morphologic tissue characteristics they reflect. In the context of prostate cancer (PCa), some studies have correlated gross histological measurements of gland lumen, epithelium, and nuclei with disease appearance on MRI. Quantitative histomorphometry (QH), like radiomics for radiologic images, is the computer based extraction of features for describing tumor morphology on digitized tissue images. In this work, we attempt to establish the histomorphometric basis for radiomic features for prostate cancer by (1) identifying the radiomic features from T2w MRI most discriminating of low vs. intermediate/high Gleason score, (2) identifying QH features correlated with the most discriminating radiomic features previously identified, and (3) evaluating the discriminative ability of QH features found to be correlated with spatially co-localized radiomic features. On a cohort of 36 patients (23 for training, 13 for validation), Gabor texture features were identified as being most predictive of Gleason grade on MRI (AUC of 0.69) and gland lumen shape features were identified as the most predictive QH features (AUC = 0.75). Our results suggest that the PCa grade discriminability of Gabor features is a consequence of variations in gland shape and morphology at the tissue level.

Computer aided analysis of prostate histopathology images Gleason grading especially for Gleason score 7

Clinically, prostate adenocarcinoma is diagnosed by recognizing certain morphology on histology. While the Gleason grading system has been shown to be the strongest prognostic factor for men with prostrate adenocarcinoma, there is a significant intra and interobserver variability between pathologists in assigning this grading system. In this study, we present a new method for prostate gland segmentation from which we then utilize to develop a computer aided Gleason grading. The novelty of our method is a region-based nuclei segmentation to get individual gland without using lumen as prior information. Because each gland region is surrounded by nuclei, individual gland can be segmented by using the structure features and Delaunay Triangulation. The precision, recal and F1 of this approach are 0.94±0.11, 0.60±0.23 and 0.70±0.19 respectively. Our method achieves a high accuracy for prostate gland segmentation with less computation time.

Image Processing Based Colorectal Cancer Detection in Histopathological Images

The article proposes an image processing-based automatic methodology for early diagnosis of colorectal cancer. In pathology, staining and sectioning of tissues are routinely used as a primary technique to detect cancer. In this methodology, the colorectal gland tissues are segmented by using adaptive threshold method. Also, it includes an analysis of geometrical features of colorectal tissues as well as it does classification of cancerous cells which classify the cancerous and non-cancerous cell efficiently. The classification is based on discriminatory geometrical features which gives good result. Unlike existing methods, it quantifies lumen and epithelial cells only in the ROI, which makes this method computationally efficient. Automatic supervised classification is accomplished on the extracted discriminatory features using support vector machine classifier. The proposed methodology segments and classifies the cancerous / non-cancerous region with an accuracy of 93.74%. The proposed method is also computationally fast which makes it suitable for real time applications.

Segmentation and classification of colon glands with deep convolutional neural networks and total variation regularization

Segmentation of histopathology sections is a necessary preprocessing step for digital pathology. Due to the large variability of biological tissue, machine learning techniques have shown superior performance over conventional image processing methods. Here we present our deep neural network-based approach for segmentation and classification of glands in tissue of benign and malignant colorectal cancer, which was developed to participate in the GlaS@MICCAI2015 colon gland segmentation challenge. We use two distinct deep convolutional neural networks (CNN) for pixel-wise classification of Hematoxylin-Eosin stained images. While the first classifier separates glands from background, the second classifier identifies gland-separating structures. In a subsequent step, a figure-ground segmentation based on weighted total variation produces the final segmentation result by regularizing the CNN predictions. We present both quantitative and qualitative segmentation results on the recently released and publicly available Warwick-QU colon adenocarcinoma dataset associated with the GlaS@MICCAI2015 challenge and compare our approach to the simultaneously developed other approaches that participated in the same challenge. On two test sets, we demonstrate our segmentation performance and show that we achieve a tissue classification accuracy of 98% and 95%, making use of the inherent capability of our system to distinguish between benign and malignant tissue. Our results show that deep learning approaches can yield highly accurate and reproducible results for biomedical image analysis, with the potential to significantly improve the quality and speed of medical diagnoses.

Discriminative Scale Learning (DiScrn): Applications to Prostate Cancer Detection from MRI and Needle Biopsies

There has been recent substantial interest in extracting sub-visual features from medical images for improved disease characterization compared to what might be achievable via visual inspection alone. Features such as Haralick and Gabor can provide a multi-scale representation of the original image by extracting measurements across differently sized neighborhoods. While these multi-scale features are effective, on large-scale digital pathological images, the process of extracting these features is computationally expensive. Moreover for different problems, different scales and neighborhood sizes may be more or less important and thus a large number of features extracted might end up being redundant. In this paper, we present a Discriminative Scale learning (DiScrn) approach that attempts to automatically identify the distinctive scales at which features are able to best separate cancerous from non-cancerous regions on both radiologic and digital pathology tissue images. To evaluate the efficacy of our approach, our approach was employed to detect presence and extent of prostate cancer on a total of 60 MRI and digitized histopathology images. Compared to a multi-scale feature analysis approach invoking features across all scales, DiScrn achieved 66% computational efficiency while also achieving comparable or even better classifier performance.

Automated analysis of co-localized protein expression in histologic sections of prostate cancer

An automated approach based on routinely-processed, whole-slide immunohistochemistry (IHC) was implemented to study co-localized protein expression in tissue samples. Expression of two markers was chosen to represent stromal (CD31) and epithelial (Ki-67) compartments in prostate cancer. IHC was performed on whole-slide sections representing low-, intermediate-, and high-grade disease from 15 patients. The automated workflow was developed using a training set of regions-of-interest in sequential tissue sections. Protein expression was studied on digital representations of IHC images across entire slides representing formalin-fixed paraffin embedded blocks. Using the training-set, the known association between Ki-67 and Gleason grade was confirmed. CD31 expression was more heterogeneous across samples and remained invariant with grade in this cohort. Interestingly, the Ki-67/CD31 ratio was significantly increased in high (Gleason ≥ 8) versus low/intermediate (Gleason ≤7) samples when assessed in the training-set and the whole-tissue block images. Further, the feasibility of the automated approach to process Tissue Microarray (TMA) samples in high throughput was evaluated. This work establishes an initial framework for automated analysis of co-localized protein expression and distribution in high-resolution digital microscopy images based on standard IHC techniques. Applied to a larger sample population, the approach may help to elucidate the biologic basis for the Gleason grade, which is the strongest, single factor distinguishing clinically aggressive from indolent prostate cancer.

Multiview boosting digital pathology analysis of prostate cancer

BACKGROUND AND OBJECTIVE

Various digital pathology tools have been developed to aid in analyzing tissues and improving cancer pathology. The multi-resolution nature of cancer pathology, however, has not been fully analyzed and utilized. Here, we develop an automated, cooperative, and multi-resolution method for improving prostate cancer diagnosis.

METHODS

Digitized tissue specimen images are obtained from 5 tissue microarrays (TMAs). The TMAs include 70 benign and 135 cancer samples (TMA1), 74 benign and 89 cancer samples (TMA2), 70 benign and 115 cancer samples (TMA3), 79 benign and 82 cancer samples (TMA4), and 72 benign and 86 cancer samples (TMA5). The tissue specimen images are segmented using intensity- and texture-based features. Using the segmentation results, a number of morphological features from lumens and epithelial nuclei are computed to characterize tissues at different resolutions. Applying a multiview boosting algorithm, tissue characteristics, obtained from differing resolutions, are cooperatively combined to achieve accurate cancer detection.

RESULTS

In segmenting prostate tissues, the multiview boosting method achieved≥ 0.97 AUC using TMA1. For detecting cancers, the multiview boosting method achieved an AUC of 0.98 (95% CI: 0.97-0.99) as trained on TMA2 and tested on TMA3, TMA4, and TMA5. The proposed method was superior to single-view approaches, utilizing features from a single resolution or merging features from all the resolutions. Moreover, the performance of the proposed method was insensitive to the choice of the training dataset. Trained on TMA3, TMA4, and TMA5, the proposed method obtained an AUC of 0.97 (95% CI: 0.96-0.98), 0.98 (95% CI: 0.96-0.99), and 0.97 (95% CI: 0.96-0.98), respectively.

CONCLUSIONS

The multiview boosting method is capable of integrating information from multiple resolutions in an effective and efficient fashion and identifying cancers with high accuracy. The multiview boosting method holds a great potential for improving digital pathology tools and research.

Gland segmentation in prostate histopathological images

Glandular structural features are important for the tumor pathologist in the assessment of cancer malignancy of prostate tissue slides. The varying shapes and sizes of glands combined with the tedious manual observation task can result in inaccurate assessment. There are also discrepancies and low-level agreement among pathologists, especially in cases of Gleason pattern 3 and pattern 4 prostate adenocarcinoma. An automated gland segmentation system can highlight various glandular shapes and structures for further analysis by the pathologist. These objective highlighted patterns can help reduce the assessment variability. We propose an automated gland segmentation system. Forty-three hematoxylin and eosin-stained images were acquired from prostate cancer tissue slides and were manually annotated for gland, lumen, periacinar retraction clefting, and stroma regions. Our automated gland segmentation system was trained using these manual annotations. It identifies these regions using a combination of pixel and object-level classifiers by incorporating local and spatial information for consolidating pixel-level classification results into object-level segmentation. Experimental results show that our method outperforms various texture and gland structure-based gland segmentation algorithms in the literature. Our method has good performance and can be a promising tool to help decrease interobserver variability among pathologists.

Evaluating stability of histomorphometric features across scanner and staining variations: prostate cancer diagnosis from whole slide images

Quantitative histomorphometry (QH) is the process of computerized feature extraction from digitized tissue slide images to predict disease presence, behavior, and outcome. Feature stability between sites may be compromised by laboratory-specific variables including dye batch, slice thickness, and the whole slide scanner used. We present two new measures, preparation-induced instability score and latent instability score, to quantify feature instability across and within datasets. In a use case involving prostate cancer, we examined QH features which may detect cancer on whole slide images. Using our method, we found that five feature families (graph, shape, co-occurring gland tensor, sub-graph, and texture) were different between datasets in 19.7% to 48.6% of comparisons while the values expected without site variation were 4.2% to 4.6%. Color normalizing all images to a template did not reduce instability. Scanning the same 34 slides on three scanners demonstrated that Haralick features were most substantively affected by scanner variation, being unstable in 62% of comparisons. We found that unstable feature families performed significantly worse in inter- than intrasite classification. Our results appear to suggest QH features should be evaluated across sites to assess robustness, and class discriminability alone should not represent the benchmark for digital pathology feature selection.

Clinically-inspired automatic classification of ovarian carcinoma subtypes

CONTEXT

It has been shown that ovarian carcinoma subtypes are distinct pathologic entities with differing prognostic and therapeutic implications. Histotyping by pathologists has good reproducibility, but occasional cases are challenging and require immunohistochemistry and subspecialty consultation. Motivated by the need for more accurate and reproducible diagnoses and to facilitate pathologists' workflow, we propose an automatic framework for ovarian carcinoma classification.

MATERIALS AND METHODS

Our method is inspired by pathologists' workflow. We analyse imaged tissues at two magnification levels and extract clinically-inspired color, texture, and segmentation-based shape descriptors using image-processing methods. We propose a carefully designed machine learning technique composed of four modules: A dissimilarity matrix, dimensionality reduction, feature selection and a support vector machine classifier to separate the five ovarian carcinoma subtypes using the extracted features.

RESULTS

This paper presents the details of our implementation and its validation on a clinically derived dataset of eighty high-resolution histopathology images. The proposed system achieved a multiclass classification accuracy of 95.0% when classifying unseen tissues. Assessment of the classifier's confusion (confusion matrix) between the five different ovarian carcinoma subtypes agrees with clinician's confusion and reflects the difficulty in diagnosing endometrioid and serous carcinomas.

CONCLUSIONS

Our results from this first study highlight the difficulty of ovarian carcinoma diagnosis which originate from the intrinsic class-imbalance observed among subtypes and suggest that the automatic analysis of ovarian carcinoma subtypes could be valuable to clinician's diagnostic procedure by providing a second opinion.

A robust nonlinear tissue-component discrimination method for computational pathology

Advances in digital pathology, specifically imaging instrumentation and data management, have allowed for the development of computational pathology tools with the potential for better, faster, and cheaper diagnosis, prognosis, and prediction of disease. Images of tissue sections frequently vary in color appearance across research laboratories and medical facilities because of differences in tissue fixation, staining protocols, and imaging instrumentation, leading to difficulty in the development of robust computational tools. To address this challenge, we propose a novel nonlinear tissue-component discrimination (NLTD) method to register automatically the color space of histopathology images and visualize individual tissue components, independent of color differences between images. Our results show that the NLTD method could effectively discriminate different tissue components from different types of tissues prepared at different institutions. Further, we demonstrate that NLTD can improve the accuracy of nuclear detection and segmentation algorithms, compared with using conventional color deconvolution methods, and can quantitatively analyze immunohistochemistry images. Together, the NLTD method is objective, robust, and effective, and can be easily implemented in the emerging field of computational pathology.

Automated classification of brain tumor type in whole-slide digital pathology images using local representative tiles

Computerized analysis of digital pathology images offers the potential of improving clinical care (e.g. automated diagnosis) and catalyzing research (e.g. discovering disease subtypes). There are two key challenges thwarting computerized analysis of digital pathology images: first, whole slide pathology images are massive, making computerized analysis inefficient, and second, diverse tissue regions in whole slide images that are not directly relevant to the disease may mislead computerized diagnosis algorithms. We propose a method to overcome both of these challenges that utilizes a coarse-to-fine analysis of the localized characteristics in pathology images. An initial surveying stage analyzes the diversity of coarse regions in the whole slide image. This includes extraction of spatially localized features of shape, color and texture from tiled regions covering the slide. Dimensionality reduction of the features assesses the image diversity in the tiled regions and clustering creates representative groups. A second stage provides a detailed analysis of a single representative tile from each group. An Elastic Net classifier produces a diagnostic decision value for each representative tile. A weighted voting scheme aggregates the decision values from these tiles to obtain a diagnosis at the whole slide level. We evaluated our method by automatically classifying 302 brain cancer cases into two possible diagnoses (glioblastoma multiforme (N = 182) versus lower grade glioma (N = 120)) with an accuracy of 93.1% (p << 0.001). We also evaluated our method in the dataset provided for the 2014 MICCAI Pathology Classification Challenge, in which our method, trained and tested using 5-fold cross validation, produced a classification accuracy of 100% (p << 0.001). Our method showed high stability and robustness to parameter variation, with accuracy varying between 95.5% and 100% when evaluated for a wide range of parameters. Our approach may be useful to automatically differentiate between the two cancer subtypes.

Influence of Texture and Colour in Breast TMA Classification

Breast cancer diagnosis is still done by observation of biopsies under the microscope. The development of automated methods for breast TMA classification would reduce diagnostic time. This paper is a step towards the solution for this problem and shows a complete study of breast TMA classification based on colour models and texture descriptors. The TMA images were divided into four classes: i) benign stromal tissue with cellularity, ii) adipose tissue, iii) benign and benign anomalous structures, and iv) ductal and lobular carcinomas. A relevant set of features was obtained on eight different colour models from first and second order Haralick statistical descriptors obtained from the intensity image, Fourier, Wavelets, Multiresolution Gabor, M-LBP and textons descriptors. Furthermore, four types of classification experiments were performed using six different classifiers: (1) classification per colour model individually, (2) classification by combination of colour models, (3) classification by combination of colour models and descriptors, and (4) classification by combination of colour models and descriptors with a previous feature set reduction. The best result shows an average of 99.05% accuracy and 98.34% positive predictive value. These results have been obtained by means of a bagging tree classifier with combination of six colour models and the use of 1719 non-correlated (correlation threshold of 97%) textural features based on Statistical, M-LBP, Gabor and Spatial textons descriptors.

Diagnostic assessment of osteosarcoma chemoresistance based on Virtual Clinical Trials

Osteosarcoma is the most common primary bone tumor in pediatric and young adult patients. Successful treatment of osteosarcomas requires a combination of surgical resection and systemic chemotherapy, both neoadjuvant (prior to surgery) and adjuvant (after surgery). The degree of necrosis following neoadjuvant chemotherapy correlates with the subsequent probability of disease-free survival. Tumors with less than 10% of viable cells after treatment represent patients with a more favorable prognosis. However, being able to predict early, such as at the time of the pre-treatment tumor biopsy, how the patient will respond to the standard chemotherapy would provide an opportunity for more personalized patient care. Patients with unfavorable predictions could be studied in a protocol, rather than a standard setting, towards improving therapeutic success. The onset of necrotic cells in osteosarcomas treated with chemotherapeutic agents is a measure of tumor sensitivity to the drugs. We hypothesize that the remaining viable cells, i.e., cells that have not responded to the treatment, are chemoresistant, and that the pathological characteristics of these chemoresistant tumor cells within the osteosarcoma pre-treatment biopsy can predict tumor response to the standard-of-care chemotherapeutic treatment. This hypothesis can be tested by comparing patient histopathology samples before, as well as after treatment to identify both morphological and immunochemical cellular features that are characteristic of chemoresistant cells, i.e., cells that survived treatment. Consequently, using computational simulations of dynamic changes in tumor pathology under the simulated standard of care chemotherapeutic treatment, one can couple the pre- and post-treatment morphological and spatial patterns of chemoresistant cells, and correlate them with patient clinical diagnoses. This procedure, that we named 'Virtual Clinical Trials', can serve as a potential predictive biomarker providing a novel value-added decision support tool for oncologists.

Machine learning approaches to analyze histological images of tissues from radical prostatectomies

Computerized evaluation of histological preparations of prostate tissues involves identification of tissue components such as stroma (ST), benign/normal epithelium (BN) and prostate cancer (PCa). Image classification approaches have been developed to identify and classify glandular regions in digital images of prostate tissues; however their success has been limited by difficulties in cellular segmentation and tissue heterogeneity. We hypothesized that utilizing image pixels to generate intensity histograms of hematoxylin (H) and eosin (E) stains deconvoluted from H&E images numerically captures the architectural difference between glands and stroma. In addition, we postulated that joint histograms of local binary patterns and local variance (LBPxVAR) can be used as sensitive textural features to differentiate benign/normal tissue from cancer. Here we utilized a machine learning approach comprising of a support vector machine (SVM) followed by a random forest (RF) classifier to digitally stratify prostate tissue into ST, BN and PCa areas. Two pathologists manually annotated 210 images of low- and high-grade tumors from slides that were selected from 20 radical prostatectomies and digitized at high-resolution. The 210 images were split into the training (n=19) and test (n=191) sets. Local intensity histograms of H and E were used to train a SVM classifier to separate ST from epithelium (BN+PCa). The performance of SVM prediction was evaluated by measuring the accuracy of delineating epithelial areas. The Jaccard J=59.5 ± 14.6 and Rand Ri=62.0 ± 7.5 indices reported a significantly better prediction when compared to a reference method (Chen et al., Clinical Proteomics 2013, 10:18) based on the averaged values from the test set. To distinguish BN from PCa we trained a RF classifier with LBPxVAR and local intensity histograms and obtained separate performance values for BN and PCa: JBN=35.2 ± 24.9, OBN=49.6 ± 32, JPCa=49.5 ± 18.5, OPCa=72.7 ± 14.8 and Ri=60.6 ± 7.6 in the test set. Our pixel-based classification does not rely on the detection of lumens, which is prone to errors and has limitations in high-grade cancers and has the potential to aid in clinical studies in which the quantification of tumor content is necessary to prognosticate the course of the disease. The image data set with ground truth annotation is available for public use to stimulate further research in this area.

Content-based image retrieval of digitized histopathology in boosted spectrally embedded spaces

Context:

Content-based image retrieval (CBIR) systems allow for retrieval of images from within a database that are similar in visual content to a query image. This is useful for digital pathology, where text-based descriptors alone might be inadequate to accurately describe image content. By representing images via a set of quantitative image descriptors, the similarity between a query image with respect to archived, annotated images in a database can be computed and the most similar images retrieved. Recently, non-linear dimensionality reduction methods have become popular for embedding high-dimensional data into a reduced-dimensional space while preserving local object adjacencies, thereby allowing for object similarity to be determined more accurately in the reduced-dimensional space. However, most dimensionality reduction methods implicitly assume, in computing the reduced-dimensional representation, that all features are equally important.

Aims:

In this paper we present boosted spectral embedding(BoSE), which utilizes a boosted distance metric to selectively weight individual features (based on training data) to subsequently map the data into a reduced-dimensional space.

Settings and Design:

BoSE is evaluated against spectral embedding (SE) (which employs equal feature weighting) in the context of CBIR of digitized prostate and breast cancer histopathology images.

Materials and Methods:

The following datasets, which were comprised of a total of 154 hematoxylin and eosin stained histopathology images, were used: (1) Prostate cancer histopathology (benign vs. malignant), (2) estrogen receptor (ER) + breast cancer histopathology (low vs. high grade), and (3) HER2+ breast cancer histopathology (low vs. high levels of lymphocytic infiltration).

Statistical Analysis Used:

We plotted and calculated the area under precision-recall curves (AUPRC) and calculated classification accuracy using the Random Forest classifier.

Results:

BoSE outperformed SE both in terms of CBIR-based (area under the precision-recall curve) and classifier-based (classification accuracy) on average across all of the dimensions tested for all three datasets: (1) Prostate cancer histopathology (AUPRC: BoSE = 0.79, SE = 0.63; Accuracy: BoSE = 0.93, SE = 0.80), (2) ER + breast cancer histopathology (AUPRC: BoSE = 0.79, SE = 0.68; Accuracy: BoSE = 0.96, SE = 0.96), and (3) HER2+ breast cancer histopathology (AUPRC: BoSE = 0.54, SE = 0.44; Accuracy: BoSE = 0.93, SE = 0.91).

Conclusion:

Our results suggest that BoSE could serve as an important tool for CBIR and classification of high-dimensional biomedical data.

Supervised multi-view canonical correlation analysis (sMVCCA): integrating histologic and proteomic features for predicting recurrent prostate cancer

In this work, we present a new methodology to facilitate prediction of recurrent prostate cancer (CaP) following radical prostatectomy (RP) via the integration of quantitative image features and protein expression in the excised prostate. Creating a fused predictor from high-dimensional data streams is challenging because the classifier must 1) account for the "curse of dimensionality" problem, which hinders classifier performance when the number of features exceeds the number of patient studies and 2) balance potential mismatches in the number of features across different channels to avoid classifier bias towards channels with more features. Our new data integration methodology, supervised Multi-view Canonical Correlation Analysis (sMVCCA), aims to integrate infinite views of highdimensional data to provide more amenable data representations for disease classification. Additionally, we demonstrate sMVCCA using Spearman's rank correlation which, unlike Pearson's correlation, can account for nonlinear correlations and outliers. Forty CaP patients with pathological Gleason scores 6-8 were considered for this study. 21 of these men revealed biochemical recurrence (BCR) following RP, while 19 did not. For each patient, 189 quantitative histomorphometric attributes and 650 protein expression levels were extracted from the primary tumor nodule. The fused histomorphometric/proteomic representation via sMVCCA combined with a random forest classifier predicted BCR with a mean AUC of 0.74 and a maximum AUC of 0.9286. We found sMVCCA to perform statistically significantly (p < 0.05) better than comparative state-of-the-art data fusion strategies for predicting BCR. Furthermore, Kaplan-Meier analysis demonstrated improved BCR-free survival prediction for the sMVCCA-fused classifier as compared to histology or proteomic features alone.

Prostate cancer grading: use of graph cut and spatial arrangement of nuclei

Tissue image grading is one of the most important steps in prostate cancer diagnosis, where the pathologist relies on the gland structure to assign a Gleason grade to the tissue image. In this grading scheme, the discrimination between grade 3 and grade 4 is the most difficult, and receives the most attention from researchers. In this study, we propose a novel method (called nuclei-based method) that 1) utilizes graph theory techniques to segment glands and 2) computes a gland-score (based on the spatial arrangement of nuclei) to estimate how similar a segmented region is to a gland. Next, we create a fusion method by combining this nuclei-based method with the lumen-based method presented in our previous work to improve the performance of grade 3 versus grade 4 classification problem (the accuracy is now improved to 87.3% compared to 81.1% of the lumen-based method alone). To segment glands, we build a graph of nuclei and lumina in the image, and use the normalized cut method to partition the graph into different components, each corresponding to a gland. Unlike most state-of-the-art lumen-based gland segmentation method, the nuclei-based method is able to segment glands without lumen or glands with multiple lumina. Moreover, another important contribution in this research is the development of a set of measures to exploit the difference in nuclei spatial arrangement between grade 3 images (where nuclei form closed chain structure on the gland boundary) and grade 4 image (where nuclei distribute more randomly in the gland). These measures are combined to generate a single gland-score value, which estimates how similar a segmented region (which is a set of nuclei and lumina) is to a gland.

Computer-Aided Prostate Cancer Diagnosis From Digitized Histopathology: A Review on Texture-Based Systems

Prostate cancer (PCa) is currently diagnosed by microscopic evaluation of biopsy samples. Since tissue assessment heavily relies on the pathologists level of expertise and interpretation criteria, it is still a subjective process with high intra- and interobserver variabilities. Computer-aided diagnosis (CAD) may have a major impact on detection and grading of PCa by reducing the pathologists reading time, and increasing the accuracy and reproducibility of diagnosis outcomes. However, the complexity of the prostatic tissue and the large volumes of data generated by biopsy procedures make the development of CAD systems for PCa a challenging task. The problem of automated diagnosis of prostatic carcinoma from histopathology has received a lot of attention. As a result, a number of CAD systems, have been proposed for quantitative image analysis and classification. This review aims at providing a detailed description of selected literature in the field of CAD of PCa, emphasizing the role of texture analysis methods in tissue description. It includes a review of image analysis tools for image preprocessing, feature extraction, classification, and validation techniques used in PCa detection and grading, as well as future directions in pursuit of better texture-based CAD systems.

Co-occurring gland angularity in localized subgraphs: predicting biochemical recurrence in intermediate-risk prostate cancer patients

Quantitative histomorphometry (QH) refers to the application of advanced computational image analysis to reproducibly describe disease appearance on digitized histopathology images. QH thus could serve as an important complementary tool for pathologists in interrogating and interpreting cancer morphology and malignancy. In the US, annually, over 60,000 prostate cancer patients undergo radical prostatectomy treatment. Around 10,000 of these men experience biochemical recurrence within 5 years of surgery, a marker for local or distant disease recurrence. The ability to predict the risk of biochemical recurrence soon after surgery could allow for adjuvant therapies to be prescribed as necessary to improve long term treatment outcomes. The underlying hypothesis with our approach, co-occurring gland angularity (CGA), is that in benign or less aggressive prostate cancer, gland orientations within local neighborhoods are similar to each other but are more chaotically arranged in aggressive disease. By modeling the extent of the disorder, we can differentiate surgically removed prostate tissue sections from (a) benign and malignant regions and (b) more and less aggressive prostate cancer. For a cohort of 40 intermediate-risk (mostly Gleason sum 7) surgically cured prostate cancer patients where half suffered biochemical recurrence, the CGA features were able to predict biochemical recurrence with 73% accuracy. Additionally, for 80 regions of interest chosen from the 40 studies, corresponding to both normal and cancerous cases, the CGA features yielded a 99% accuracy. CGAs were shown to be statistically signicantly () better at predicting BCR compared to state-of-the-art QH methods and postoperative prostate cancer nomograms.

Separation of benign and malignant glands in prostatic adenocarcinoma

This paper presents an analysis of the high resolution histopathology images of the prostate with a focus on the evolution of morphological gland features in prostatic adenocarcinoma. Here we propose a novel technique of labeling individual glands as malignant or benign. In the first step, the gland and nuclei objects of the images are automatically segmented. Individual gland units are segmented out by consolidating their lumina with the surrounding layers of epithelium and nuclei. The nuclei objects are segmented by using a marker controlled watershed algorithm. Two new features, Number of Nuclei Layer (N(NL)) and Ratio of Epithelial layer area to Lumen area (R(EL)) have been extracted from the segmented units. The main advantage of this approach is that it can detect individual malignant gland units, irrespective of neighboring histology and/or the spatial extent of the cancer. The proposed algorithm has been tested on 40 histopathology scenes taken from 10 high resolution whole mount images and achieved a sensitivity of 0.83 and specificity of 0.81 in a leave-75%-out cross-validation.

Weakly supervised histopathology cancer image segmentation and classification

Labeling a histopathology image as having cancerous regions or not is a critical task in cancer diagnosis; it is also clinically important to segment the cancer tissues and cluster them into various classes. Existing supervised approaches for image classification and segmentation require detailed manual annotations for the cancer pixels, which are time-consuming to obtain. In this paper, we propose a new learning method, multiple clustered instance learning (MCIL) (along the line of weakly supervised learning) for histopathology image segmentation. The proposed MCIL method simultaneously performs image-level classification (cancer vs. non-cancer image), medical image segmentation (cancer vs. non-cancer tissue), and patch-level clustering (different classes). We embed the clustering concept into the multiple instance learning (MIL) setting and derive a principled solution to performing the above three tasks in an integrated framework. In addition, we introduce contextual constraints as a prior for MCIL, which further reduces the ambiguity in MIL. Experimental results on histopathology colon cancer images and cytology images demonstrate the great advantage of MCIL over the competing methods.

A quantitative histomorphometric classifier (QuHbIC) identifies aggressive versus indolent p16-positive oropharyngeal squamous cell carcinoma

Human papillomavirus-related (p16-positive) oropharyngeal squamous cell carcinoma patients develop recurrent disease, mostly distant metastasis, in approximately 10% of cases, and the remaining patients, despite cure, can have major morbidity from treatment. Identifying patients with aggressive versus indolent tumors is critical. Hematoxylin and eosin-stained slides of a microarray cohort of p16-positive oropharyngeal squamous cell carcinoma cases were digitally scanned. A novel cluster cell graph was constructed using the nuclei as vertices to characterize and measure spatial distribution and cell clustering. A series of topological features defined on each node of the subgraph were analyzed, and a random forest decision tree classifier was developed. The classifier (QuHbIC) was validated over 25 runs of 3-fold cross-validation using case subsets for independent training and testing. Nineteen (11.9%) of the 160 patients on the array developed recurrence. QuHbIC correctly predicted outcomes in 140 patients (87.5% accuracy). There were 23 positive patients, of whom 11 developed recurrence (47.8% positive predictive value), and 137 negative patients, of whom only 8 developed recurrence (94.2% negative predictive value). The best other predictive features were stage T4 (18 patients; 83.1% accuracy) and N3 nodal disease (10 patients; 88.6% accuracy). QuHbIC-positive patients had poorer overall, disease-free, and disease-specific survival (P<0.001 for each). In multivariate analysis, QuHbIC-positive patients still showed significantly poorer disease-free and disease-specific survival, independent of all other variables. In summary, using just tiny hematoxylin and eosin punches, a computer-aided histomorphometric classifier (QuHbIC) can strongly predict recurrence risk. With prospective validation, this testing may be useful to stratify patients into different treatment groups.

Explicit shape descriptors: novel morphologic features for histopathology classification

Object morphology, defined as shape and size characteristics, observed on medical imagery is often an important marker for disease presence and/or aggressiveness. In the context of prostate cancer histopathology, gland morphology is an integral component of the Gleason grading system which enables discrimination between low and high grade disease. However, clinicians are often unable to distinguish between subtle differences in object morphology, as evidenced by high inter-observer variability in Gleason grading. Boundary-based morphologic descriptors, such as the variance in the distance from points on the boundary of an object to its center, may not have the requisite discriminability to separate objects with subtle shape differences. In this paper, we present a set of novel explicit shape descriptors (ESDs) which are capable of distinguishing subtle shape differences between prostate glands of intermediate Gleason grades (grades 3 and 4) on prostate cancer histopathology. Calculation of ESDs involves: (1) representing object morphology using an explicit shape model (e.g. medial axis); (2) aligning the shape models via a non-rigid registration scheme with a diffeomorphic constraint and quantifying shape model dissimilarity; and (3) applying a non-linear dimensionality reduction scheme (e.g. Graph Embedding) to learn a low dimensional projection encoding the shape differences between objects. ESDs are hence the principal eigenvectors in the reduced embedding space. In this work we demonstrate that ESDs in conjunction with a Support Vector Machine classifier are able to correctly distinguish between 888 prostate glands corresponding to different Gleason grades (benign, grade 3, or grade 4) of prostate cancer from 58 needle biopsy specimens with a maximum accuracy of 0.89 and corresponding area under the receiver operating characteristic curve of 0.78.

Level set segmentation of medical images based on local region statistics and maximum a posteriori probability

This paper presents a variational level set method for simultaneous segmentation and bias field estimation of medical images with intensity inhomogeneity. In our model, the statistics of image intensities belonging to each different tissue in local regions are characterized by Gaussian distributions with different means and variances. According to maximum a posteriori probability (MAP) and Bayes' rule, we first derive a local objective function for image intensities in a neighborhood around each pixel. Then this local objective function is integrated with respect to the neighborhood center over the entire image domain to give a global criterion. In level set framework, this global criterion defines an energy in terms of the level set functions that represent a partition of the image domain and a bias field that accounts for the intensity inhomogeneity of the image. Therefore, image segmentation and bias field estimation are simultaneously achieved via a level set evolution process. Experimental results for synthetic and real images show desirable performances of our method.

Pathology imaging informatics for quantitative analysis of whole-slide images

Objectives

With the objective of bringing clinical decision support systems to reality, this article reviews histopathological whole-slide imaging informatics methods, associated challenges, and future research opportunities.

Target audience

This review targets pathologists and informaticians who have a limited understanding of the key aspects of whole-slide image (WSI) analysis and/or a limited knowledge of state-of-the-art technologies and analysis methods.

Scope

First, we discuss the importance of imaging informatics in pathology and highlight the challenges posed by histopathological WSI. Next, we provide a thorough review of current methods for: quality control of histopathological images; feature extraction that captures image properties at the pixel, object, and semantic levels; predictive modeling that utilizes image features for diagnostic or prognostic applications; and data and information visualization that explores WSI for de novo discovery. In addition, we highlight future research directions and discuss the impact of large public repositories of histopathological data, such as the Cancer Genome Atlas, on the field of pathology informatics. Following the review, we present a case study to illustrate a clinical decision support system that begins with quality control and ends with predictive modeling for several cancer endpoints. Currently, state-of-the-art software tools only provide limited image processing capabilities instead of complete data analysis for clinical decision-making. We aim to inspire researchers to conduct more research in pathology imaging informatics so that clinical decision support can become a reality.

Multi-kernel graph embedding for detection, Gleason grading of prostate cancer via MRI/MRS

Even though 1 in 6 men in the US, in their lifetime are expected to be diagnosed with prostate cancer (CaP), only 1 in 37 is expected to die on account of it. Consequently, among many men diagnosed with CaP, there has been a recent trend to resort to active surveillance (wait and watch) if diagnosed with a lower Gleason score on biopsy, as opposed to seeking immediate treatment. Some researchers have recently identified imaging markers for low and high grade CaP on multi-parametric (MP) magnetic resonance (MR) imaging (such as T2 weighted MR imaging (T2w MRI) and MR spectroscopy (MRS)). In this paper, we present a novel computerized decision support system (DSS), called Semi Supervised Multi Kernel Graph Embedding (SeSMiK-GE), that quantitatively combines structural, and metabolic imaging data for distinguishing (a) benign versus cancerous, and (b) high- versus low-Gleason grade CaP regions from in vivo MP-MRI. A total of 29 1.5Tesla endorectal pre-operative in vivo MP MRI (T2w MRI, MRS) studies from patients undergoing radical prostatectomy were considered in this study. Ground truth for evaluation of the SeSMiK-GE classifier was obtained via annotation of disease extent on the pre-operative imaging by visually correlating the MRI to the ex vivo whole mount histologic specimens. The SeSMiK-GE framework comprises of three main modules: (1) multi-kernel learning, (2) semi-supervised learning, and (3) dimensionality reduction, which are leveraged for the construction of an integrated low dimensional representation of the different imaging and non-imaging MRI protocols. Hierarchical classifiers for diagnosis and Gleason grading of CaP are then constructed within this unified low dimensional representation. Step 1 of the hierarchical classifier employs a random forest classifier in conjunction with the SeSMiK-GE based data representation and a probabilistic pairwise Markov Random Field algorithm (which allows for imposition of local spatial constraints) to yield a voxel based classification of CaP presence. The CaP region of interest identified in Step 1 is then subsequently classified as either high or low Gleason grade CaP in Step 2. Comparing SeSMiK-GE with unimodal T2w MRI, MRS classifiers and a commonly used feature concatenation (COD) strategy, yielded areas (AUC) under the receiver operative curve (ROC) of (a) 0.89±0.09 (SeSMiK), 0.54±0.18 (T2w MRI), 0.61±0.20 (MRS), and 0.64±0.23 (COD) for distinguishing benign from CaP regions, and (b) 0.84±0.07 (SeSMiK),0.54±0.13 (MRI), 0.59±0.19 (MRS), and 0.62±0.18 (COD) for distinguishing high and low grade CaP using a leave one out cross-validation strategy, all evaluations being performed on a per voxel basis. Our results suggest that following further rigorous validation, SeSMiK-GE could be developed into a powerful diagnostic and prognostic tool for detection and grading of CaP in vivo and in helping to determine the appropriate treatment option. Identifying low grade disease in vivo might allow CaP patients to opt for active surveillance rather than immediately opt for aggressive therapy such as radical prostatectomy.

Class-specific weighting for Markov random field estimation: application to medical image segmentation

Many estimation tasks require Bayesian classifiers capable of adjusting their performance (e.g. sensitivity/specificity). In situations where the optimal classification decision can be identified by an exhaustive search over all possible classes, means for adjusting classifier performance, such as probability thresholding or weighting the a posteriori probabilities, are well established. Unfortunately, analogous methods compatible with Markov random fields (i.e. large collections of dependent random variables) are noticeably absent from the literature. Consequently, most Markov random field (MRF) based classification systems typically restrict their performance to a single, static operating point (i.e. a paired sensitivity/specificity). To address this deficiency, we previously introduced an extension of maximum posterior marginals (MPM) estimation that allows certain classes to be weighted more heavily than others, thus providing a means for varying classifier performance. However, this extension is not appropriate for the more popular maximum a posteriori (MAP) estimation. Thus, a strategy for varying the performance of MAP estimators is still needed. Such a strategy is essential for several reasons: (1) the MAP cost function may be more appropriate in certain classification tasks than the MPM cost function, (2) the literature provides a surfeit of MAP estimation implementations, several of which are considerably faster than the typical Markov Chain Monte Carlo methods used for MPM, and (3) MAP estimation is used far more often than MPM. Consequently, in this paper we introduce multiplicative weighted MAP (MWMAP) estimation-achieved via the incorporation of multiplicative weights into the MAP cost function-which allows certain classes to be preferred over others. This creates a natural bias for specific classes, and consequently a means for adjusting classifier performance. Similarly, we show how this multiplicative weighting strategy can be applied to the MPM cost function (in place of the strategy we presented previously), yielding multiplicative weighted MPM (MWMPM) estimation. Furthermore, we describe how MWMAP and MWMPM can be implemented using adaptations of current estimation strategies such as iterated conditional modes and MPM Monte Carlo. To illustrate these implementations, we first integrate them into two separate MRF-based classification systems for detecting carcinoma of the prostate (CaP) on (1) digitized histological sections from radical prostatectomies and (2) T2-weighted 4 Tesla ex vivo prostate MRI. To highlight the extensibility of MWMAP and MWMPM to estimation tasks involving more than two classes, we also incorporate these estimation criteria into a MRF-based classifier used to segment synthetic brain MR images. In the context of these tasks, we show how our novel estimation criteria can be used to arbitrarily adjust the sensitivities of these systems, yielding receiver operator characteristic curves (and surfaces).

Supervised regularized canonical correlation analysis: integrating histologic and proteomic data for predicting biochemical failures

Multimodal data, especially imaging and non-imaging data, is being routinely acquired in the context of disease diagnostics; however computational challenges have limited the ability to quantitatively integrate imaging and non-imaging data channels with different dimensionalities for making diagnostic and prognostic predictions. The objective of this work is to create a common subspace to simultaneously accommodate both the imaging and non-imaging data, called a metaspace. This metaspace can be used to build a meta-classifier that produces better classification results than a classifier that is based on a single modality alone. In this paper, we present a novel Supervised Regularized Canonical Correlation Analysis (SRCCA) algorithm that (1) enables the quantitative integration of data from multiple modalities using a feature selection scheme, (2) is regularized, and (3) is computationally cheap. We leverage this SRCCA framework towards the fusion of proteomic and histologic image signatures for identifying prostate cancer patients at risk for biochemical recurrence following radical prostatectomy. For a cohort of 19 prostate cancer patients, SRCCA was able to yield a lower fused dimensional metaspace comprising both the histological and proteomic attributes. In conjunction with SRCCA, a random forest classifier was able to identify patients at risk for biochemical failure with a maximum accuracy of 93%. The classifier performance in the SRCCA space was statistically significantly higher compared to the fused data representations obtained either with Canonical Correlation Analysis (CCA) or Regularized CCA.

Detection of prostate cancer on histopathology using color fractals and Probabilistic Pairwise Markov models

In this paper we present a system for detecting regions of carcinoma of the prostate (CaP) in H&E stained radical prostatectomy specimens using the color fractal dimension. Color textural information is known to be a valuable characteristic to distinguish CaP from benign tissue. In addition to color information, we know that cancer tends to form contiguous regions. Our system leverages the color staining information of histology as well as spatial dependencies. The color and textural information is first captured using color fractal dimension. To incorporate spatial dependencies, we combine the probability map constructed via color fractal dimension with a novel Markov prior called the Probabilistic Pairwise Markov Model (PPMM). To demonstrate the capability of this CaP detection system, we applied the algorithm to 27 radical prostatectomy specimens from 10 patients. A per pixel evaluation was conducted with ground truth provided by an expert pathologist using only the color fractal feature first, yielding an area under the receiver operator characteristic curve (AUC) curve of 0.790. In conjunction with a Markov prior, the resultant color fractal dimension + Markov random field (MRF) classifier yielded an AUC of 0.831.

Digital Pathology: Data-Intensive Frontier in Medical Imaging: Health-information sharing, specifically of digital pathology, is the subject of this paper which discusses how sharing the rich images in pathology can stretch the capabilities of all otherwise well-practiced disciplines

Pathology is a medical subspecialty that practices the diagnosis of disease. Microscopic examination of tissue reveals information enabling the pathologist to render accurate diagnoses and to guide therapy. The basic process by which anatomic pathologists render diagnoses has remained relatively unchanged over the last century, yet advances in information technology now offer significant opportunities in image-based diagnostic and research applications. Pathology has lagged behind other healthcare practices such as radiology where digital adoption is widespread. As devices that generate whole slide images become more practical and affordable, practices will increasingly adopt this technology and eventually produce an explosion of data that will quickly eclipse the already vast quantities of radiology imaging data. These advances are accompanied by significant challenges for data management and storage, but they also introduce new opportunities to improve patient care by streamlining and standardizing diagnostic approaches and uncovering disease mechanisms. Computer-based image analysis is already available in commercial diagnostic systems, but further advances in image analysis algorithms are warranted in order to fully realize the benefits of digital pathology in medical discovery and patient care. In coming decades, pathology image analysis will extend beyond the streamlining of diagnostic workflows and minimizing interobserver variability and will begin to provide diagnostic assistance, identify therapeutic targets, and predict patient outcomes and therapeutic responses.