Automated prostate tissue referencing for cancer detection and diagnosis

Digital analysis of the prostate tumor microenvironment with high-order chromogenic multiplexing

As our understanding of the tumor microenvironment grows, the pathology field is increasingly utilizing multianalyte diagnostic assays to understand important characteristics of tumor growth. In clinical settings, brightfield chromogenic assays represent the gold-standard and have developed significant trust as the first-line diagnostic method. However, conventional brightfield tests have been limited to low-order assays that are visually interrogated. We have developed a hybrid method of brightfield chromogenic multiplexing that overcomes these limitations and enables high-order multiplex assays. However, how compatible high-order brightfield multiplexed images are with advanced analytical algorithms has not been extensively evaluated. In the present study, we address this gap by developing a novel 6-marker prostate cancer assay that targets diverse aspects of the tumor microenvironment such as prostate-specific biomarkers (PSMA and p504s), immune biomarkers (CD8 and PD-L1), a prognostic biomarker (Ki-67), as well as an adjunctive diagnostic biomarker (basal cell cocktail) and apply the assay to 143 differentially graded adenocarcinoma prostate tissues. The tissues were then imaged on our spectroscopic multiplexing imaging platform and mined for proteomic and spatial features that were correlated with cancer presence and disease grade. Extracted features were used to train a UMAP model that differentiated healthy from cancerous tissue with an accuracy of 89% and identified clusters of cells based on cancer grade. For spatial analysis, cell-to-cell distances were calculated for all biomarkers and differences between healthy and adenocarcinoma tissues were studied. We report that p504s positive cells were at least 2× closer to cells expressing PD-L1, CD8, Ki-67, and basal cell in adenocarcinoma tissues relative to the healthy control tissues. These findings offer a powerful insight to understand the fingerprint of the prostate tumor microenvironment and indicate that high-order chromogenic multiplexing is compatible with digital analysis. Thus, the presented chromogenic multiplexing system combines the clinical applicability of brightfield assays with the emerging diagnostic power of high-order multiplexing in a digital pathology friendly format that is well-suited for translational studies to better understand mechanisms of tumor development and growth.

Prostate cancer histopathology using label-free multispectral deep-UV microscopy quantifies phenotypes of tumor aggressiveness and enables multiple diagnostic virtual stains

Identifying prostate cancer patients that are harboring aggressive forms of prostate cancer remains a significant clinical challenge. Here we develop an approach based on multispectral deep-ultraviolet (UV) microscopy that provides novel quantitative insight into the aggressiveness and grade of this disease, thus providing a new tool to help address this important challenge. We find that UV spectral signatures from endogenous molecules give rise to a phenotypical continuum that provides unique structural insight (i.e., molecular maps or “optical stains") of thin tissue sections with subcellular (nanoscale) resolution. We show that this phenotypical continuum can also be applied as a surrogate biomarker of prostate cancer malignancy, where patients with the most aggressive tumors show a ubiquitous glandular phenotypical shift. In addition to providing several novel “optical stains” with contrast for disease, we also adapt a two-part Cycle-consistent Generative Adversarial Network to translate the label-free deep-UV images into virtual hematoxylin and eosin (H&E) stained images, thus providing multiple stains (including the gold-standard H&E) from the same unlabeled specimen. Agreement between the virtual H&E images and the H&E-stained tissue sections is evaluated by a panel of pathologists who find that the two modalities are in excellent agreement. This work has significant implications towards improving our ability to objectively quantify prostate cancer grade and aggressiveness, thus improving the management and clinical outcomes of prostate cancer patients. This same approach can also be applied broadly in other tumor types to achieve low-cost, stain-free, quantitative histopathological analysis.

Fast and scalable search of whole-slide images via self-supervised deep learning

The adoption of digital pathology has enabled the curation of large repositories of gigapixel whole-slide images (WSIs). Computationally identifying WSIs with similar morphologic features within large repositories without requiring supervised training can have significant applications. However, the retrieval speeds of algorithms for searching similar WSIs often scale with the repository size, which limits their clinical and research potential. Here we show that self-supervised deep learning can be leveraged to search for and retrieve WSIs at speeds that are independent of repository size. The algorithm, which we named SISH (for self-supervised image search for histology) and provide as an open-source package, requires only slide-level annotations for training, encodes WSIs into meaningful discrete latent representations and leverages a tree data structure for fast searching followed by an uncertainty-based ranking algorithm for WSI retrieval. We evaluated SISH on multiple tasks (including retrieval tasks based on tissue-patch queries) and on datasets spanning over 22,000 patient cases and 56 disease subtypes. SISH can also be used to aid the diagnosis of rare cancer types for which the number of available WSIs is often insufficient to train supervised deep-learning models.

A Digital Pathology Solution to Resolve the Tissue Floater Conundrum

CONTEXT.—

Pathologists may encounter extraneous pieces of tissue (tissue floaters) on glass slides because of specimen cross-contamination. Troubleshooting this problem, including performing molecular tests for tissue identification if available, is time consuming and often does not satisfactorily resolve the problem.

OBJECTIVE.—

To demonstrate the feasibility of using an image search tool to resolve the tissue floater conundrum.

DESIGN.—

A glass slide was produced containing 2 separate hematoxylin and eosin (H&E)-stained tissue floaters. This fabricated slide was digitized along with the 2 slides containing the original tumors used to create these floaters. These slides were then embedded into a dataset of 2325 whole slide images comprising a wide variety of H&E stained diagnostic entities. Digital slides were broken up into patches and the patch features converted into barcodes for indexing and easy retrieval. A deep learning-based image search tool was employed to extract features from patches via barcodes, hence enabling image matching to each tissue floater.

RESULTS.—

There was a very high likelihood of finding a correct tumor match for the queried tissue floater when searching the digital database. Search results repeatedly yielded a correct match within the top 3 retrieved images. The retrieval accuracy improved when greater proportions of the floater were selected. The time to run a search was completed within several milliseconds.

CONCLUSIONS.—

Using an image search tool offers pathologists an additional method to rapidly resolve the tissue floater conundrum, especially for those laboratories that have transitioned to going fully digital for primary diagnosis.

Deep learning-based image-analysis algorithm for classification and quantification of multiple histopathological lesions in rat liver

Artificial intelligence (AI)-based image analysis is increasingly being used for preclinical safety-assessment studies in the pharmaceutical industry. In this paper, we present an AI-based solution for preclinical toxicology studies. We trained a set of algorithms to learn and quantify multiple typical histopathological findings in whole slide images (WSIs) of the livers of young Sprague Dawley rats by using a U-Net-based deep learning network. The trained algorithms were validated using 255 liver WSIs to detect, classify, and quantify seven types of histopathological findings (including vacuolation, bile duct hyperplasia, and single-cell necrosis) in the liver. The algorithms showed consistently good performance in detecting abnormal areas. Approximately 75% of all specimens could be classified as true positive or true negative. In general, findings with clear boundaries with the surrounding normal structures, such as vacuolation and single-cell necrosis, were accurately detected with high statistical scores. The results of quantitative analyses and classification of the diagnosis based on the threshold values between “no findings” and “abnormal findings” correlated well with diagnoses made by professional pathologists. However, the scores for findings ambiguous boundaries, such as hepatocellular hypertrophy, were poor. These results suggest that deep learning-based algorithms can detect, classify, and quantify multiple findings simultaneously on rat liver WSIs. Thus, it can be a useful supportive tool for a histopathological evaluation, especially for primary screening in rat toxicity studies.

Quantification of histopathological findings using a novel image analysis platform

Digital pathology, including image analysis and automatic diagnosis of pathological tissue, has been developed remarkably. HALO is an image analysis platform specialized for the study of pathological tissues, which enables tissue segmentation by using artificial intelligence. In this study, we used HALO to quantify various histopathological changes and findings that were difficult to analyze using conventional image processing software. Using the tissue classifier module, the morphological features of degeneration/necrosis of the hepatocytes and muscle fibers, bile duct in the liver, basophilic tubules and hyaline casts in the kidney, cortex in the thymus, and red pulp, white pulp, and marginal zone in the spleen were learned and separated, and areas of interest were quantified. Furthermore, using the cytonuclear module and vacuole module in combination with the tissue classifier module, the number of erythroblasts in the red pulp of the spleen and each area of acinar cells in the parotid gland were quantified. The results of quantitative analysis were correlated with the histopathological grades evaluated by pathologists. By using artificial intelligence and other functions of HALO, we recognized morphological features, analyzed histopathological changes, and quantified the histopathological grades of various findings. The analysis of histopathological changes using HALO is expected to support pathology evaluations.

Extended Multiplicative Signal Correction for Infrared Microspectroscopy of Heterogeneous Samples with Cylindrical Domains

Optical scattering corrections are invoked to computationally distinguish between scattering and absorption contributions to recorded data in infrared (IR) microscopy, with a goal to obtain an absorption spectrum that is relatively free of the effects of sample morphology. Here, we present a modification of the extended multiplicative signal correction (EMSC) approach that allows for spectral recovery from fibers and cylindrical domains in heterogeneous samples. The developed theoretical approach is based on exact Mie theory for infinite cylinders. Although rigorous Mie theory implies utilization of comprehensive and time-consuming calculations, we propose to change the workflow of the original EMSC algorithm to minimize extensive calculations for each recorded spectrum at each iteration step. This makes the modified EMSC approach practical for routine use. First, we tested our approach using synthetic data derived from a rigorous model of scattering from cylinders in an IR microscope. Second, we applied the approach to Fourier transform IR (FT-IR) microspectroscopy data recorded from filamentous fungal and cellulose samples with pronounced fiber-like shapes. While the corrected spectra show greatly reduced baseline offsets and consistency, strongly absorbing regions of the spectrum require further refinement. The modified EMSC algorithm broadly mitigates the effects of scattering, offering a practical approach to more consistent and accurate spectra from cylindrical objects or heterogeneous samples with cylindrical domains.

Similar image search for histopathology: SMILY

The increasing availability of large institutional and public histopathology image datasets is enabling the searching of these datasets for diagnosis, research, and education. Although these datasets typically have associated metadata such as diagnosis or clinical notes, even carefully curated datasets rarely contain annotations of the location of regions of interest on each image. As pathology images are extremely large (up to 100,000 pixels in each dimension), further laborious visual search of each image may be needed to find the feature of interest. In this paper, we introduce a deep-learning-based reverse image search tool for histopathology images: Similar Medical Images Like Yours (SMILY). We assessed SMILY's ability to retrieve search results in two ways: using pathologist-provided annotations, and via prospective studies where pathologists evaluated the quality of SMILY search results. As a negative control in the second evaluation, pathologists were blinded to whether search results were retrieved by SMILY or randomly. In both types of assessments, SMILY was able to retrieve search results with similar histologic features, organ site, and prostate cancer Gleason grade compared with the original query. SMILY may be a useful general-purpose tool in the pathologist's arsenal, to improve the efficiency of searching large archives of histopathology images, without the need to develop and implement specific tools for each application.

An EM-based semi-supervised deep learning approach for semantic segmentation of histopathological images from radical prostatectomies

Automated Gleason grading is an important preliminary step for quantitative histopathological feature extraction. Different from the traditional task of classifying small pre-selected homogeneous regions, semantic segmentation provides pixel-wise Gleason predictions across an entire slide. Deep learning-based segmentation models can automatically learn visual semantics from data, which alleviates the need for feature engineering. However, performance of deep learning models is limited by the scarcity of large-scale fully annotated datasets, which can be both expensive and time-consuming to create. One way to address this problem is to leverage external weakly labeled datasets to augment models trained on the limited data. In this paper, we developed an expectation maximization-based approach constrained by an approximated prior distribution in order to extract useful representations from a large number of weakly labeled images generated from low-magnification annotations. This method was utilized to improve the performance of a model trained on a limited fully annotated dataset. Our semi-supervised approach trained with 135 fully annotated and 1800 weakly annotated tiles achieved a mean Jaccard Index of 49.5% on an independent test set, which was 14% higher than the initial model trained only on the fully annotated dataset.

Dielectric Sphere Clusters as a Model to Understand Infrared Spectroscopic Imaging Data Recorded from Complex Samples

Understanding the infrared (IR) spectral response of materials as a function of their morphology is not only of fundamental importance but also of contemporary practical need in the analysis of biological and synthetic materials. While significant work has recently been reported in understanding the spectra of particles with well-defined geometries, we report here on samples that consist of collections of particles. First, we theoretically model the importance of multiple scattering effects and computationally predict the impact of local particles' environment on the recorded IR spectra. Both monodisperse and polydisperse particles are considered in clusters with various degrees of packing. We show that recorded spectra are highly dependent on the cluster morphology and size of particles but the origin of this dependence is largely due to the scattering that depends on morphology and not absorbance that largely depends on the volume of material. The effect of polydispersity is to reduce the fine scattering features in the spectrum, resulting in a closer resemblance to bulk spectra. Fourier transform-IR (FT-IR) spectra of clusters of electromagnetically coupled poly(methyl methacrylate) (PMMA) spheres with wavelength-scale diameters were recorded and compared to simulated results. Measured spectra agreed well with those predicted. Of note, when PMMA spheres occupy a volume greater than 18% of the focal volume, the recorded IR spectrum becomes almost independent of the cluster's morphological changes. This threshold, where absorbance starts to dominate the signal, exactly matches the percolation threshold for hard spheres and quantifies the transition between the single particle and bulk behavior. Our finding enables an understanding of the spectral response of structured samples and points to appropriate models for recovering accurate chemical information from in IR microspectroscopy data.

Quantitative analysis of histopathological findings using image processing software

In evaluating pathological changes in drug efficacy and toxicity studies, morphometric analysis can be quite robust. In this experiment, we examined whether morphometric changes of major pathological findings in various tissue specimens stained with hematoxylin and eosin could be recognized and quantified using image processing software. Using Tissue Studio, hypertrophy of hepatocytes and adrenocortical cells could be quantified based on the method of a previous report, but the regions of red pulp, white pulp, and marginal zones in the spleen could not be recognized when using one setting condition. Using Image-Pro Plus, lipid-derived vacuoles in the liver and mucin-derived vacuoles in the intestinal mucosa could be quantified using two criteria (area and/or roundness). Vacuoles derived from phospholipid could not be quantified when small lipid deposition coexisted in the liver and adrenal cortex. Mononuclear inflammatory cell infiltration in the liver could be quantified to some extent, except for specimens with many clustered infiltrating cells. Adipocyte size and the mean linear intercept could be quantified easily and efficiently using morphological processing and the macro tool equipped in Image-Pro Plus. These methodologies are expected to form a base system that can recognize morphometric features and analyze quantitatively pathological findings through the use of information technology.

Multiview boosting digital pathology analysis of prostate cancer

BACKGROUND AND OBJECTIVE

Various digital pathology tools have been developed to aid in analyzing tissues and improving cancer pathology. The multi-resolution nature of cancer pathology, however, has not been fully analyzed and utilized. Here, we develop an automated, cooperative, and multi-resolution method for improving prostate cancer diagnosis.

METHODS

Digitized tissue specimen images are obtained from 5 tissue microarrays (TMAs). The TMAs include 70 benign and 135 cancer samples (TMA1), 74 benign and 89 cancer samples (TMA2), 70 benign and 115 cancer samples (TMA3), 79 benign and 82 cancer samples (TMA4), and 72 benign and 86 cancer samples (TMA5). The tissue specimen images are segmented using intensity- and texture-based features. Using the segmentation results, a number of morphological features from lumens and epithelial nuclei are computed to characterize tissues at different resolutions. Applying a multiview boosting algorithm, tissue characteristics, obtained from differing resolutions, are cooperatively combined to achieve accurate cancer detection.

RESULTS

In segmenting prostate tissues, the multiview boosting method achieved≥ 0.97 AUC using TMA1. For detecting cancers, the multiview boosting method achieved an AUC of 0.98 (95% CI: 0.97-0.99) as trained on TMA2 and tested on TMA3, TMA4, and TMA5. The proposed method was superior to single-view approaches, utilizing features from a single resolution or merging features from all the resolutions. Moreover, the performance of the proposed method was insensitive to the choice of the training dataset. Trained on TMA3, TMA4, and TMA5, the proposed method obtained an AUC of 0.97 (95% CI: 0.96-0.98), 0.98 (95% CI: 0.96-0.99), and 0.97 (95% CI: 0.96-0.98), respectively.

CONCLUSIONS

The multiview boosting method is capable of integrating information from multiple resolutions in an effective and efficient fashion and identifying cancers with high accuracy. The multiview boosting method holds a great potential for improving digital pathology tools and research.