Epithelium segmentation using deep learning in H&E-stained prostate specimens with immunohistochemistry as reference standard

Deep learning in histopathology: the path to the clinic

Machine learning techniques have great potential to improve medical diagnostics, offering ways to improve accuracy, reproducibility and speed, and to ease workloads for clinicians. In the field of histopathology, deep learning algorithms have been developed that perform similarly to trained pathologists for tasks such as tumor detection and grading. However, despite these promising results, very few algorithms have reached clinical implementation, challenging the balance between hope and hype for these new techniques. This Review provides an overview of the current state of the field, as well as describing the challenges that still need to be addressed before artificial intelligence in histopathology can achieve clinical value.

Pathomics in urology

PURPOSE OF REVIEW

Pathomics, the fusion of digitalized pathology and artificial intelligence, is currently changing the landscape of medical pathology and biologic disease classification. In this review, we give an overview of Pathomics and summarize its most relevant applications in urology.

RECENT FINDINGS

There is a steady rise in the number of studies employing Pathomics, and especially deep learning, in urology. In prostate cancer, several algorithms have been developed for the automatic differentiation between benign and malignant lesions and to differentiate Gleason scores. Furthermore, several applications have been developed for the automatic cancer cell detection in urine and for tumor assessment in renal cancer. Despite the explosion in research, Pathomics is not fully ready yet for widespread clinical application.

SUMMARY

In prostate cancer and other urologic pathologies, Pathomics is avidly being researched with commercial applications on the close horizon. Pathomics is set to improve the accuracy, speed, reliability, cost-effectiveness and generalizability of pathology, especially in uro-oncology.

Optimized tumour infiltrating lymphocyte assessment for triple negative breast cancer prognostics

The tumour microenvironment has been shown to be a valuable source of prognostic information for different cancer types. This holds in particular for triple negative breast cancer (TNBC), a breast cancer subtype for which currently no prognostic biomarkers are established. Although different methods to assess tumour infiltrating lymphocytes (TILs) have been published, it remains unclear which method (marker, region) yields the most optimal prognostic information. In addition, to date, no objective TILs assessment methods are available. For this proof of concept study, a subset of our previously described TNBC cohort (n = 94) was stained for CD3, CD8 and FOXP3 using multiplex immunohistochemistry and subsequently imaged by a multispectral imaging system. Advanced whole-slide image analysis algorithms, including convolutional neural networks (CNN) were used to register unmixed multispectral images and corresponding H&E sections, to segment the different tissue compartments (tumour, stroma) and to detect all individual positive lymphocytes. Densities of positive lymphocytes were analysed in different regions within the tumour and its neighbouring environment and correlated to relapse free survival (RFS) and overall survival (OS). We found that for all TILs markers the presence of a high density of positive cells correlated with an improved survival. None of the TILs markers was superior to the others. The results of TILs assessment in the various regions did not show marked differences between each other. The negative correlation between TILs and survival in our cohort are in line with previous studies. Our results provide directions for optimizing TILs assessment methodology.

Artificial Intelligence Techniques for Prostate Cancer Detection through Dual-Channel Tissue Feature Engineering

Simple Summary

Artificial intelligence techniques were used for the detection of prostate cancer through tissue feature engineering. A radiomic method was used to extract the important features or information from histopathology tissue images to perform binary classification (i.e., benign vs. malignant). This method can identify a histological pattern that is invisible to the human eye, which helps researchers to predict and detect prostate cancer. We used different performance metrics to evaluate the results of the classification. In the future, it is expected that a method like radiomic will provide a consistent contribution to analyze histopathology tissue images and differentiate between cancerous and noncancerous tumors.

Abstract

The optimal diagnostic and treatment strategies for prostate cancer (PCa) are constantly changing. Given the importance of accurate diagnosis, texture analysis of stained prostate tissues is important for automatic PCa detection. We used artificial intelligence (AI) techniques to classify dual-channel tissue features extracted from Hematoxylin and Eosin (H&E) tissue images, respectively. Tissue feature engineering was performed to extract first-order statistic (FOS)-based textural features from each stained channel, and cancer classification between benign and malignant was carried out based on important features. Recursive feature elimination (RFE) and one-way analysis of variance (ANOVA) methods were used to identify significant features, which provided the best five features out of the extracted six features. The AI techniques used in this study for binary classification (benign vs. malignant and low-grade vs. high-grade) were support vector machine (SVM), logistic regression (LR), bagging tree, boosting tree, and dual-channel bidirectional long short-term memory (DC-BiLSTM) network. Further, a comparative analysis was carried out between the AI algorithms. Two different datasets were used for PCa classification. Out of these, the first dataset (private) was used for training and testing the AI models and the second dataset (public) was used only for testing to evaluate model performance. The automatic AI classification system performed well and showed satisfactory results according to the hypothesis of this study.

Deep Learning Methods for Lung Cancer Segmentation in Whole-Slide Histopathology Images-The ACDC@LungHP Challenge 2019

Accurate segmentation of lung cancer in pathology slides is a critical step in improving patient care. We proposed the ACDC@LungHP (Automatic Cancer Detection and Classification in Whole-slide Lung Histopathology) challenge for evaluating different computer-aided diagnosis (CADs) methods on the automatic diagnosis of lung cancer. The ACDC@LungHP 2019 focused on segmentation (pixel-wise detection) of cancer tissue in whole slide imaging (WSI), using an annotated dataset of 150 training images and 50 test images from 200 patients. This paper reviews this challenge and summarizes the top 10 submitted methods for lung cancer segmentation. All methods were evaluated using metrics using the precision, accuracy, sensitivity, specificity, and DICE coefficient (DC). The DC ranged from 0.7354 ±0.1149 to 0.8372 ±0.0858. The DC of the best method was close to the inter-observer agreement (0.8398 ±0.0890). All methods were based on deep learning and categorized into two groups: multi-model method and single model method. In general, multi-model methods were significantly better (p 0.01) than single model methods, with mean DC of 0.7966 and 0.7544, respectively. Deep learning based methods could potentially help pathologists find suspicious regions for further analysis of lung cancer in WSI.

Deep neural network models for computational histopathology: A survey

Histopathological images contain rich phenotypic information that can be used to monitor underlying mechanisms contributing to disease progression and patient survival outcomes. Recently, deep learning has become the mainstream methodological choice for analyzing and interpreting histology images. In this paper, we present a comprehensive review of state-of-the-art deep learning approaches that have been used in the context of histopathological image analysis. From the survey of over 130 papers, we review the field's progress based on the methodological aspect of different machine learning strategies such as supervised, weakly supervised, unsupervised, transfer learning and various other sub-variants of these methods. We also provide an overview of deep learning based survival models that are applicable for disease-specific prognosis tasks. Finally, we summarize several existing open datasets and highlight critical challenges and limitations with current deep learning approaches, along with possible avenues for future research.

Developing a Qualification and Verification Strategy for Digital Tissue Image Analysis in Toxicological Pathology

Digital tissue image analysis is a computational method for analyzing whole-slide images and extracting large, complex, and quantitative data sets. However, as with any analysis method, the quality of generated results is dependent on a well-designed quality control system for the entire digital pathology workflow. Such system requires clear procedural controls, appropriate user training, and involvement of specialists to oversee key steps of the workflow. The toxicologic pathologist is responsible for reporting data obtained by digital image analysis and therefore needs to ensure that it is correct. To accomplish that, they must understand the main parameters of the quality control system and should play an integral part in its conception and implementation. This manuscript describes the most common digital tissue image analysis end points and potential sources of analysis errors. In addition, it outlines recommended approaches for ensuring quality and correctness of results for both classical and machine-learning based image analysis solutions, as adapted from a recently proposed Food and Drug Administration regulatory framework for modifications to artificial intelligence/machine learning-based software as a medical device. These approaches are beneficial for any type of toxicopathologic study which uses the described end points and can be adjusted based on the intended use of the image analysis solution.

An Artificial Intelligence-based Support Tool for Automation and Standardisation of Gleason Grading in Prostate Biopsies

BACKGROUND

Gleason grading is the standard diagnostic method for prostate cancer and is essential for determining prognosis and treatment. The dearth of expert pathologists, the inter- and intraobserver variability, as well as the labour intensity of Gleason grading all necessitate the development of a user-friendly tool for robust standardisation.

OBJECTIVE

To develop an artificial intelligence (AI) algorithm, based on machine learning and convolutional neural networks, as a tool for improved standardisation in Gleason grading in prostate cancer biopsies.

DESIGN, SETTING, AND PARTICIPANTS

A total of 698 prostate biopsy sections from 174 patients were used for training. The training sections were annotated by two senior consultant pathologists. The final algorithm was tested on 37 biopsy sections from 21 patients, with digitised slide images from two different scanners.

OUTCOME MEASUREMENTS AND STATISTICAL ANALYSIS

Correlation, sensitivity, and specificity parameters were calculated.

RESULTS AND LIMITATIONS

The algorithm shows high accuracy in detecting cancer areas (sensitivity: 100%, specificity: 68%). Compared with the pathologists, the algorithm also performed well in detecting cancer areas (intraclass correlation coefficient [ICC]: 0.99) and assigning the Gleason patterns correctly: Gleason patterns 3 and 4 (ICC: 0.96 and 0.94, respectively), and to a lesser extent, Gleason pattern 5 (ICC: 0.82). Similar results were obtained using two different scanners.

CONCLUSIONS

Our AI-based algorithm can reliably detect prostate cancer and quantify the Gleason patterns in core needle biopsies, with similar accuracy as pathologists. The results are reproducible on images from different scanners with a proven low level of intraobserver variability. We believe that this AI tool could be regarded as an efficient and interactive tool for pathologists.

PATIENT SUMMARY

We developed a sensitive artificial intelligence tool for prostate biopsies, which detects and grades cancer with similar accuracy to pathologists. This tool holds promise to improve the diagnosis of prostate cancer.

Spatial transcriptomics inferred from pathology whole-slide images links tumor heterogeneity to survival in breast and lung cancer

Digital analysis of pathology whole-slide images is fast becoming a game changer in cancer diagnosis and treatment. Specifically, deep learning methods have shown great potential to support pathology analysis, with recent studies identifying molecular traits that were not previously recognized in pathology H&E whole-slide images. Simultaneous to these developments, it is becoming increasingly evident that tumor heterogeneity is an important determinant of cancer prognosis and susceptibility to treatment, and should therefore play a role in the evolving practices of matching treatment protocols to patients. State of the art diagnostic procedures, however, do not provide automated methods for characterizing and/or quantifying tumor heterogeneity, certainly not in a spatial context. Further, existing methods for analyzing pathology whole-slide images from bulk measurements require many training samples and complex pipelines. Our work addresses these two challenges. First, we train deep learning models to spatially resolve bulk mRNA and miRNA expression levels on pathology whole-slide images (WSIs). Our models reach up to 0.95 AUC on held-out test sets from two cancer cohorts using a simple training pipeline and a small number of training samples. Using the inferred gene expression levels, we further develop a method to spatially characterize tumor heterogeneity. Specifically, we produce tumor molecular cartographies and heterogeneity maps of WSIs and formulate a heterogeneity index (HTI) that quantifies the level of heterogeneity within these maps. Applying our methods to breast and lung cancer slides, we show a significant statistical link between heterogeneity and survival. Our methods potentially open a new and accessible approach to investigating tumor heterogeneity and other spatial molecular properties and their link to clinical characteristics, including treatment susceptibility and survival.

Combining multiple spatial statistics enhances the description of immune cell localisation within tumours

Digital pathology enables computational analysis algorithms to be applied at scale to histological images. An example is the identification of immune cells within solid tumours. Image analysis algorithms can extract precise cell locations from immunohistochemistry slides, but the resulting spatial coordinates, or point patterns, can be difficult to interpret. Since localisation of immune cells within tumours may reflect their functional status and correlates with patient prognosis, novel descriptors of their spatial distributions are of biological and clinical interest. A range of spatial statistics have been used to analyse such point patterns but, individually, these approaches only partially describe complex immune cell distributions. In this study, we apply three spatial statistics to locations of CD68+ macrophages within human head and neck tumours, and show that images grouped semi-quantitatively by a pathologist share similar statistics. We generate a synthetic dataset which emulates human samples and use it to demonstrate that combining multiple spatial statistics with a maximum likelihood approach better predicts human classifications than any single statistic. We can also estimate the error associated with our classifications. Importantly, this methodology is adaptable and can be extended to other histological investigations or applied to point patterns outside of histology.

Pathadin - The essential set of tools to start with whole slide analysis

Pathadin (https://gitlab.com/Digipathology/Pathadin) was designed as a WSI oriented open-source set of tools for beginners to experience the advantages of computer-assisted image analysis and cover essential features, frequently strenuous to access with the alternative programs. It is mainly oriented to work with histology slides but also includes a significant part of modern image formats. Introducing Pathadin, the manuscript aims to improve understanding of contemporary image analysis components, resolve technophobia and misbeliefs in the computational field, simplifying pathology research work, and shifting it into a quantitative paradigm. Despite being easy to use, Pathadin includes both basic and advanced analytical algorithms, as the application of machine learning. The functionality of Pathadin is demonstrated by AI-enhanced quantification of epithelial and stromal changes in prostate carcinoma, and their dependence on ISUP grade. The material included 5 radical prostatectomy samples for training and 83 (including 11 autopsies) samples for analysis. The analytical material was stained with Masson's trichrome and Ki67, as widely available and potentially prognostic markers. An integrated solution by HistomicsTK for Ki67 evaluation was used. A U-net model for separating glands and stroma was trained, simplifying the independent analysis of these components. During the process, the model successfully highlighted glands and stroma. Masson's trichrome stain demonstrated a gradual increase in collagen expression, being statistically significant between controls vs. G1, and G3 vs. G4. Although there was considerable overlap between adjacent groups, there was only a minor overlap in collagen amount between high- and low-grade carcinomas, affirming that with further research, stroma could be an additional diagnostic marker in prostate adenocarcinoma. Ki67 showed a statistically significant gradual increase in all groups except G1 vs. G2 and G4 vs. G5. Pathadin demonstrates that there is no need for significant resources to experience the advantages of modern computer-assisted analysis.

Clinically applicable histopathological diagnosis system for gastric cancer detection using deep learning

The early detection and accurate histopathological diagnosis of gastric cancer increase the chances of successful treatment. The worldwide shortage of pathologists offers a unique opportunity for the use of artificial intelligence assistance systems to alleviate the workload and increase diagnostic accuracy. Here, we report a clinically applicable system developed at the Chinese PLA General Hospital, China, using a deep convolutional neural network trained with 2,123 pixel-level annotated H&E-stained whole slide images. The model achieves a sensitivity near 100% and an average specificity of 80.6% on a real-world test dataset with 3,212 whole slide images digitalized by three scanners. We show that the system could aid pathologists in improving diagnostic accuracy and preventing misdiagnoses. Moreover, we demonstrate that our system performs robustly with 1,582 whole slide images from two other medical centres. Our study suggests the feasibility and benefits of using histopathological artificial intelligence assistance systems in routine practice scenarios.

TissueWand, a Rapid Histopathology Annotation Tool

Background:

Recent advancements in machine learning (ML) bring great possibilities for the development of tools to assist with diagnostic tasks within histopathology. However, these approaches typically require a large amount of ground truth training data in the form of image annotations made by human experts. As such annotation work is a very time-consuming task, there is a great need for tools that can assist in this process, saving time while not sacrificing annotation quality.

Methods:

In an iterative design process, we developed TissueWand – an interactive tool designed for efficient annotation of gigapixel-sized histopathological images, not being constrained to a predefined annotation task.

Results:

Several findings regarding appropriate interaction concepts were made, where a key design component was semi-automation based on rapid interaction feedback in a local region. In a user study, the resulting tool was shown to cause substantial speed-up compared to manual work while maintaining quality.

Conclusions:

The TissueWand tool shows promise to replace manual methods for early stages of dataset curation where no task-specific ML model yet exists to aid the effort.

Combination of Peri-Tumoral and Intra-Tumoral Radiomic Features on Bi-Parametric MRI Accurately Stratifies Prostate Cancer Risk: A Multi-Site Study

Background: Prostate cancer (PCa) influences its surrounding habitat, which tends to manifest as different phenotypic appearances on magnetic resonance imaging (MRI). This region surrounding the PCa lesion, or the peri-tumoral region, may encode useful information that can complement intra-tumoral information to enable better risk stratification. Purpose: To evaluate the role of peri-tumoral radiomic features on bi-parametric MRI (T2-weighted and Diffusion-weighted) to distinguish PCa risk categories as defined by D’Amico Risk Classification System. Materials and Methods: We studied a retrospective, HIPAA-compliant, 4-institution cohort of 231 PCa patients (n = 301 lesions) who underwent 3T multi-parametric MRI prior to biopsy. PCa regions of interest (ROIs) were delineated on MRI by experienced radiologists following which peri-tumoral ROIs were defined. Radiomic features were extracted within the intra- and peri-tumoral ROIs. Radiomic features differentiating low-risk from: (1) high-risk (L-vs.-H), and (2) (intermediate- and high-risk (L-vs.-I + H)) lesions were identified. Using a multi-institutional training cohort of 151 lesions (D1, N = 116 patients), machine learning classifiers were trained using peri- and intra-tumoral features individually and in combination. The remaining 150 lesions (D2, N = 115 patients) were used for independent hold-out validation and were evaluated using Receiver Operating Characteristic (ROC) analysis and compared with PI-RADS v2 scores. Results: Validation on D2 using peri-tumoral radiomics alone resulted in areas under the ROC curve (AUCs) of 0.84 and 0.73 for the L-vs.-H and L-vs.-I + H classifications, respectively. The best combination of intra- and peri-tumoral features resulted in AUCs of 0.87 and 0.75 for the L-vs.-H and L-vs.-I + H classifications, respectively. This combination improved the risk stratification results by 3–6% compared to intra-tumoral features alone. Our radiomics-based model resulted in a 53% accuracy in differentiating L-vs.-H compared to PI-RADS v2 (48%), on the validation set. Conclusion: Our findings suggest that peri-tumoral radiomic features derived from prostate bi-parametric MRI add independent predictive value to intra-tumoral radiomic features for PCa risk assessment.

A deep learning model to predict RNA-Seq expression of tumours from whole slide images

Deep learning methods for digital pathology analysis are an effective way to address multiple clinical questions, from diagnosis to prediction of treatment outcomes. These methods have also been used to predict gene mutations from pathology images, but no comprehensive evaluation of their potential for extracting molecular features from histology slides has yet been performed. We show that HE2RNA, a model based on the integration of multiple data modes, can be trained to systematically predict RNA-Seq profiles from whole-slide images alone, without expert annotation. Through its interpretable design, HE2RNA provides virtual spatialization of gene expression, as validated by CD3- and CD20-staining on an independent dataset. The transcriptomic representation learned by HE2RNA can also be transferred on other datasets, even of small size, to increase prediction performance for specific molecular phenotypes. We illustrate the use of this approach in clinical diagnosis purposes such as the identification of tumors with microsatellite instability.

RNA-sequencing of tumour tissue can provide important diagnostic and prognostic information but this is costly and not routinely performed in all clinical settings. Here, the authors show that whole slide histology slides—part of routine care—can be used to predict RNA-sequencing data and thus reduce the need for additional analyses.

The 2019 International Society of Urological Pathology (ISUP) Consensus Conference on Grading of Prostatic Carcinoma

Five years after the last prostatic carcinoma grading consensus conference of the International Society of Urological Pathology (ISUP), accrual of new data and modification of clinical practice require an update of current pathologic grading guidelines. This manuscript summarizes the proceedings of the ISUP consensus meeting for grading of prostatic carcinoma held in September 2019, in Nice, France. Topics brought to consensus included the following: (1) approaches to reporting of Gleason patterns 4 and 5 quantities, and minor/tertiary patterns, (2) an agreement to report the presence of invasive cribriform carcinoma, (3) an agreement to incorporate intraductal carcinoma into grading, and (4) individual versus aggregate grading of systematic and multiparametric magnetic resonance imaging-targeted biopsies. Finally, developments in the field of artificial intelligence in the grading of prostatic carcinoma and future research perspectives were discussed.

Weakly-Supervised Classification of HER2 Expression in Breast Cancer Haematoxylin and Eosin Stained Slides

Human epidermal growth factor receptor 2 (HER2) evaluation commonly requires immunohistochemistry (IHC) tests on breast cancer tissue, in addition to the standard haematoxylin and eosin (H&E) staining tests. Additional costs and time spent on further testing might be avoided if HER2 overexpression could be effectively inferred from H&E stained slides, as a preliminary indication of the IHC result. In this paper, we propose the first method that aims to achieve this goal. The proposed method is based on multiple instance learning (MIL), using a convolutional neural network (CNN) that separately processes H&E stained slide tiles and outputs an IHC label. This CNN is pretrained on IHC stained slide tiles but does not use these data during inference/testing. H&E tiles are extracted from invasive tumour areas segmented with the HASHI algorithm. The individual tile labels are then combined to obtain a single label for the whole slide. The network was trained on slides from the HER2 Scoring Contest dataset (HER2SC) and tested on two disjoint subsets of slides from the HER2SC database and the TCGA-TCIA-BRCA (BRCA) collection. The proposed method attained 83.3 % classification accuracy on the HER2SC test set and 53.8 % on the BRCA test set. Although further efforts should be devoted to achieving improved performance, the obtained results are promising, suggesting that it is possible to perform HER2 overexpression classification on H&E stained tissue slides.

Deep learning approach for quantification of organelles and misfolded polypeptide delivery within degradative compartments

Endolysosomal compartments maintain cellular fitness by clearing dysfunctional organelles and proteins from cells. Modulation of their activity offers therapeutic opportunities. Quantification of cargo delivery to and/or accumulation within endolysosomes is instrumental for characterizing lysosome-driven pathways at the molecular level and monitoring consequences of genetic or environmental modifications. Here we introduce LysoQuant, a deep learning approach for segmentation and classification of fluorescence images capturing cargo delivery within endolysosomes for clearance. LysoQuant is trained for unbiased and rapid recognition with human-level accuracy, and the pipeline informs on a series of quantitative parameters such as endolysosome number, size, shape, position within cells, and occupancy, which report on activity of lysosome-driven pathways. In our selected examples, LysoQuant successfully determines the magnitude of mechanistically distinct catabolic pathways that ensure lysosomal clearance of a model organelle, the endoplasmic reticulum, and of a model protein, polymerogenic ATZ. It does so with accuracy and velocity compatible with those of high-throughput analyses.

De novo discovery of metabolic heterogeneity with immunophenotype-guided imaging mass spectrometry

Imaging mass spectrometry enables in situ label-free detection of thousands of metabolites from intact tissue samples. However, automated steps for multi-omics analyses and interpretation of histological images have not yet been implemented in mass spectrometry data analysis workflows. The characterization of molecular properties within cellular and histological features is done via time-consuming, non-objective, and irreproducible definitions of regions of interest, which are often accompanied by a loss of spatial resolution due to mass spectra averaging.

Methods: We developed a new imaging pipeline called Spatial Correlation Image Analysis (SPACiAL), which is a computational multimodal workflow designed to combine molecular imaging data with multiplex immunohistochemistry (IHC). SPACiAL allows comprehensive and spatially resolved in situ correlation analyses on a cellular resolution. To demonstrate the method, matrix-assisted laser desorption-ionization (MALDI) Fourier-transform ion cyclotron resonance (FTICR) imaging mass spectrometry of metabolites and multiplex IHC staining were performed on the very same tissue section of mouse pancreatic islets and on human gastric cancer tissue specimens. The SPACiAL pipeline was used to perform an automatic, semantic-based, functional tissue annotation of histological and cellular features to identify metabolic profiles. Spatial correlation networks were generated to analyze metabolic heterogeneity associated with cellular features.

Results: To demonstrate the new method, the SPACiAL pipeline was used to identify metabolic signatures of alpha and beta cells within islets of Langerhans, which are cell types that are not distinguishable via morphology alone. The semantic-based, functional tissue annotation allows an unprecedented analysis of metabolic heterogeneity via the generation of spatial correlation networks. Additionally, we demonstrated intra- and intertumoral metabolic heterogeneity within HER2/neu-positive and -negative gastric tumor cells.

Conclusions: We developed the SPACiAL workflow to provide IHC-guided in situ metabolomics on intact tissue sections. Diminishing the workload by automated recognition of histological and functional features, the pipeline allows comprehensive analyses of metabolic heterogeneity. The multimodality of immunohistochemical staining and extensive molecular information from imaging mass spectrometry has the advantage of increasing both the efficiency and precision for spatially resolved analyses of specific cell types. The SPACiAL method is a stepping stone for the objective analysis of high-throughput, multi-omics data from clinical research and practice that is required for diagnostics, biomarker discovery, or therapy response prediction.

•

Novel method enables phenotype-guided in situ metabolomics on intact tissue sections.

•

Metabolic heterogeneity can be objectified under (patho-)physiological conditions.

•

Innovative approach for tissue-based, preclinical, and clinical research.

Automated deep-learning system for Gleason grading of prostate cancer using biopsies: a diagnostic study

BACKGROUND

The Gleason score is the strongest correlating predictor of recurrence for prostate cancer, but has substantial inter-observer variability, limiting its usefulness for individual patients. Specialised urological pathologists have greater concordance; however, such expertise is not widely available. Prostate cancer diagnostics could thus benefit from robust, reproducible Gleason grading. We aimed to investigate the potential of deep learning to perform automated Gleason grading of prostate biopsies.

METHODS

In this retrospective study, we developed a deep-learning system to grade prostate biopsies following the Gleason grading standard. The system was developed using randomly selected biopsies, sampled by the biopsy Gleason score, from patients at the Radboud University Medical Center (pathology report dated between Jan 1, 2012, and Dec 31, 2017). A semi-automatic labelling technique was used to circumvent the need for manual annotations by pathologists, using pathologists' reports as the reference standard during training. The system was developed to delineate individual glands, assign Gleason growth patterns, and determine the biopsy-level grade. For validation of the method, a consensus reference standard was set by three expert urological pathologists on an independent test set of 550 biopsies. Of these 550, 100 were used in an observer experiment, in which the system, 13 pathologists, and two pathologists in training were compared with respect to the reference standard. The system was also compared to an external test dataset of 886 cores, which contained 245 cores from a different centre that were independently graded by two pathologists.

FINDINGS

We collected 5759 biopsies from 1243 patients. The developed system achieved a high agreement with the reference standard (quadratic Cohen's kappa 0·918, 95% CI 0·891-0·941) and scored highly at clinical decision thresholds: benign versus malignant (area under the curve 0·990, 95% CI 0·982-0·996), grade group of 2 or more (0·978, 0·966-0·988), and grade group of 3 or more (0·974, 0·962-0·984). In an observer experiment, the deep-learning system scored higher (kappa 0·854) than the panel (median kappa 0·819), outperforming 10 of 15 pathologist observers. On the external test dataset, the system obtained a high agreement with the reference standard set independently by two pathologists (quadratic Cohen's kappa 0·723 and 0·707) and within inter-observer variability (kappa 0·71).

INTERPRETATION

Our automated deep-learning system achieved a performance similar to pathologists for Gleason grading and could potentially contribute to prostate cancer diagnosis. The system could potentially assist pathologists by screening biopsies, providing second opinions on grade group, and presenting quantitative measurements of volume percentages.

FUNDING

Dutch Cancer Society.

Strategies to Reduce the Expert Supervision Required for Deep Learning-Based Segmentation of Histopathological Images

The emergence of computational pathology comes with a demand to extract more and more information from each tissue sample. Such information extraction often requires the segmentation of numerous histological objects (e.g., cell nuclei, glands, etc.) in histological slide images, a task for which deep learning algorithms have demonstrated their effectiveness. However, these algorithms require many training examples to be efficient and robust. For this purpose, pathologists must manually segment hundreds or even thousands of objects in histological images, i.e., a long, tedious and potentially biased task. The present paper aims to review strategies that could help provide the very large number of annotated images needed to automate the segmentation of histological images using deep learning. This review identifies and describes four different approaches: the use of immunohistochemical markers as labels, realistic data augmentation, Generative Adversarial Networks (GAN), and transfer learning. In addition, we describe alternative learning strategies that can use imperfect annotations. Adding real data with high-quality annotations to the training set is a safe way to improve the performance of a well configured deep neural network. However, the present review provides new perspectives through the use of artificially generated data and/or imperfect annotations, in addition to transfer learning opportunities.

Head and Neck Cancer Detection in Digitized Whole-Slide Histology Using Convolutional Neural Networks

Primary management for head and neck cancers, including squamous cell carcinoma (SCC), involves surgical resection with negative cancer margins. Pathologists guide surgeons during these operations by detecting cancer in histology slides made from the excised tissue. In this study, 381 digitized, histological whole-slide images (WSI) from 156 patients with head and neck cancer were used to train, validate, and test an inception-v4 convolutional neural network. The proposed method is able to detect and localize primary head and neck SCC on WSI with an AUC of 0.916 for patients in the SCC testing group and 0.954 for patients in the thyroid carcinoma testing group. Moreover, the proposed method is able to diagnose WSI with cancer versus normal slides with an AUC of 0.944 and 0.995 for the SCC and thyroid carcinoma testing groups, respectively. For comparison, we tested the proposed, diagnostic method on an open-source dataset of WSI from sentinel lymph nodes with breast cancer metastases, CAMELYON 2016, to obtain patch-based cancer localization and slide-level cancer diagnoses. The experimental design yields a robust method with potential to help create a tool to increase efficiency and accuracy of pathologists detecting head and neck cancers in histological images.

Convolutional neural networks can accurately distinguish four histologic growth patterns of lung adenocarcinoma in digital slides

During the diagnostic workup of lung adenocarcinomas (LAC), pathologists evaluate distinct histological tumor growth patterns. The percentage of each pattern on multiple slides bears prognostic significance. To assist with the quantification of growth patterns, we constructed a pipeline equipped with a convolutional neural network (CNN) and soft-voting as the decision function to recognize solid, micropapillary, acinar, and cribriform growth patterns, and non-tumor areas. Slides of primary LAC were obtained from Cedars-Sinai Medical Center (CSMC), the Military Institute of Medicine in Warsaw and the TCGA portal. Several CNN models trained with 19,924 image tiles extracted from 78 slides (MIMW and CSMC) were evaluated on 128 test slides from the three sites by F1-score and accuracy using manual tumor annotations by pathologist. The best CNN yielded F1-scores of 0.91 (solid), 0.76 (micropapillary), 0.74 (acinar), 0.6 (cribriform), and 0.96 (non-tumor) respectively. The overall accuracy of distinguishing the five tissue classes was 89.24%. Slide-based accuracy in the CSMC set (88.5%) was significantly better (p < 2.3E-4) than the accuracy in the MIMW (84.2%) and TCGA (84%) sets due to superior slide quality. Our model can work side-by-side with a pathologist to accurately quantify the percentages of growth patterns in tumors with mixed LAC patterns.