Assessment of a computerized quantitative quality control tool for whole slide images of kidney biopsies

Improving quality control of whole slide images by explicit artifact augmentation

The problem of artifacts in whole slide image acquisition, prevalent in both clinical workflows and research-oriented settings, necessitates human intervention and re-scanning. Overcoming this challenge requires developing quality control algorithms, that are hindered by the limited availability of relevant annotated data in histopathology. The manual annotation of ground-truth for artifact detection methods is expensive and time-consuming. This work addresses the issue by proposing a method dedicated to augmenting whole slide images with artifacts. The tool seamlessly generates and blends artifacts from an external library to a given histopathology dataset. The augmented datasets are then utilized to train artifact classification methods. The evaluation shows their usefulness in classification of the artifacts, where they show an improvement from 0.10 to 0.01 AUROC depending on the artifact type. The framework, model, weights, and ground-truth annotations are freely released to facilitate open science and reproducible research.

Clinical Relevance of Computationally Derived Tubular Features: Spatial Relationships and the Development of Tubulointerstitial Scarring in MCD/FSGS

Background

Visual scoring of tubular damage has limitations in capturing the full spectrum of structural changes and prognostic potential. We investigate if computationally quantified tubular features can enhance prognostication and reveal spatial relationships with interstitial fibrosis.

Methods

Deep-learning and image-processing-based segmentations were employed in N=254/266 PAS-WSIs from the NEPTUNE/CureGN datasets (135/153 focal segmental glomerulosclerosis and 119/113 minimal change disease) for: cortex, tubular lumen (TL), epithelium (TE), nuclei (TN), and basement membrane (TBM). N=104 pathomic features were extracted from these segmented tubular substructures and summarized at the patient level using summary statistics. The tubular features were quantified across the biopsy and in manually segmented regions of mature interstitial fibrosis and tubular atrophy (IFTA), pre-IFTA and non-IFTA in the NEPTUNE dataset. Minimum Redundancy Maximum Relevance was used in the NEPTUNE dataset to select features most associated with disease progression and proteinuria remission. Ridge-penalized Cox models evaluated their predictive discrimination compared to clinical/demographic data and visual-assessment. Models were evaluated in the CureGN dataset.

Results

N=9 features were predictive of disease progression and/or proteinuria remission. Models with tubular features had high prognostic accuracy in both NEPTUNE and CureGN datasets and increased prognostic accuracy for both outcomes (5.6%-7.7% and 1.6%-4.6% increase for disease progression and proteinuria remission, respectively) compared to conventional parameters alone in the NEPTUNE dataset. TBM thickness/area and TE simplification progressively increased from non- to pre- and mature IFTA.

Conclusions

Previously under-recognized, quantifiable, and clinically relevant tubular features in the kidney parenchyma can enhance understanding of mechanisms of disease progression and risk stratification.

CohortFinder: an open-source tool for data-driven partitioning of digital pathology and imaging cohorts to yield robust machine-learning models

Batch effects (BEs) refer to systematic technical differences in data collection unrelated to biological variations whose noise is shown to negatively impact machine learning (ML) model generalizability. Here we release CohortFinder (http://cohortfinder.com), an open-source tool aimed at mitigating BEs via data-driven cohort partitioning. We demonstrate CohortFinder improves ML model performance in downstream digital pathology and medical image processing tasks. CohortFinder is freely available for download at cohortfinder.com.

Revolutionizing Pathology with Artificial Intelligence: Innovations in Immunohistochemistry

Artificial intelligence (AI) is a reality of our times, and it has been successfully implemented in all fields, including medicine. As a relatively new domain, all efforts are directed towards creating algorithms applicable in most medical specialties. Pathology, as one of the most important areas of interest for precision medicine, has received significant attention in the development and implementation of AI algorithms. This focus is especially important for achieving accurate diagnoses. Moreover, immunohistochemistry (IHC) serves as a complementary diagnostic tool in pathology. It can be further augmented through the application of deep learning (DL) and machine learning (ML) algorithms for assessing and analyzing immunohistochemical markers. Such advancements can aid in delineating targeted therapeutic approaches and prognostic stratification. This article explores the applications and integration of various AI software programs and platforms used in immunohistochemical analysis. It concludes by highlighting the application of these technologies to pathologies such as breast, prostate, lung, melanocytic proliferations, and hematologic conditions. Additionally, it underscores the necessity for further innovative diagnostic algorithms to assist physicians in the diagnostic process.

Artificial Intelligence-Based Quality Assessment of Histopathology Whole-Slide Images within a Clinical Workflow: Assessment of 'PathProfiler' in a Diagnostic Pathology Setting

Digital pathology continues to gain momentum, with the promise of artificial intelligence to aid diagnosis and for assessment of features which may impact prognosis and clinical management. Successful adoption of these technologies depends upon the quality of digitised whole-slide images (WSI); however, current quality control largely depends upon manual assessment, which is inefficient and subjective. We previously developed PathProfiler, an automated image quality assessment tool, and in this feasibility study we investigate its potential for incorporation into a diagnostic clinical pathology setting in real-time. A total of 1254 genitourinary WSI were analysed by PathProfiler. PathProfiler was developed and trained on prostate tissue and, of the prostate biopsy WSI, representing 46% of the WSI analysed, 4.5% were flagged as potentially being of suboptimal quality for diagnosis. All had concordant subjective issues, mainly focus-related, 54% severe enough to warrant remedial action which resulted in improved image quality. PathProfiler was less reliable in assessment of non-prostate surgical resection-type cases, on which it had not been trained. PathProfiler shows potential for incorporation into a digitised clinical pathology workflow, with opportunity for image quality improvement. Whilst its reliability in the current form appears greatest for assessment of prostate specimens, other specimen types, particularly biopsies, also showed benefit.

Pathogenomics for accurate diagnosis, treatment, prognosis of oncology: a cutting edge overview

The capability to gather heterogeneous data, alongside the increasing power of artificial intelligence to examine it, leading a revolution in harnessing multimodal data in the life sciences. However, most approaches are limited to unimodal data, leaving integrated approaches across modalities relatively underdeveloped in computational pathology. Pathogenomics, as an invasive method to integrate advanced molecular diagnostics from genomic data, morphological information from histopathological imaging, and codified clinical data enable the discovery of new multimodal cancer biomarkers to propel the field of precision oncology in the coming decade. In this perspective, we offer our opinions on synthesizing complementary modalities of data with emerging multimodal artificial intelligence methods in pathogenomics. It includes correlation between the pathological and genomic profile of cancer, fusion of histology, and genomics profile of cancer. We also present challenges, opportunities, and avenues for future work.

Improvements in digital pathology equipment for renal biopsies: updating the standard model

INTRODUCTION

Digital pathology can improve the technical and interpretative workflows in nephropathology by creating hub-spoke networks and virtuous collaboration projects among centers in different geographical regions. New high-resolution fast-scanning instruments combined with currently existing equipment were tested in a nephropathology hub to evaluate possible upgrading in the routine processing phases.

METHODS

The scanning performance of two different instruments (Aperio vs hybrid MIDI II) was evaluated and a comparative quality control check was performed on obtained whole slide images.

RESULTS

Both with default and custom settings for light microscopy, MIDI II proved to be faster, with only slightly more time required to prepare the scan and larger final file size as compared to Aperio (p < 0.001). No differences were noted in the number of out-of-focus slides per case (p = 0.75). Regarding immunofluorescence, the new scanner required longer preparation time (p = 0.001) with comparable scanning times and final file size (p = 0.169 and p = 0.177, respectively). Quality control showed differences in 3 quality features related to white background and blurriness (p < 0.001). No major discordances in the final diagnosis were recorded after comparing the report obtained with slides scanned using the two instruments, with only one case (4%) showing minor disagreement.

CONCLUSION

The present report describes the experience of a hub nephropathology center adopting next generation digital pathology tools for the routine assessment of renal biopsies, highlighting the need for a complementary approach towards a philosophy of interoperability.

Banff Digital Pathology Working Group: Image Bank, Artificial Intelligence Algorithm, and Challenge Trial Developments

The Banff Digital Pathology Working Group (DPWG) was established with the goal to establish a digital pathology repository; develop, validate, and share models for image analysis; and foster collaborations using regular videoconferencing. During the calls, a variety of artificial intelligence (AI)-based support systems for transplantation pathology were presented. Potential collaborations in a competition/trial on AI applied to kidney transplant specimens, including the DIAGGRAFT challenge (staining of biopsies at multiple institutions, pathologists' visual assessment, and development and validation of new and pre-existing Banff scoring algorithms), were also discussed. To determine the next steps, a survey was conducted, primarily focusing on the feasibility of establishing a digital pathology repository and identifying potential hosts. Sixteen of the 35 respondents (46%) had access to a server hosting a digital pathology repository, with 2 respondents that could serve as a potential host at no cost to the DPWG. The 16 digital pathology repositories collected specimens from various organs, with the largest constituent being kidney (n = 12,870 specimens). A DPWG pilot digital pathology repository was established, and there are plans for a competition/trial with the DIAGGRAFT project. Utilizing existing resources and previously established models, the Banff DPWG is establishing new resources for the Banff community.

Reproducibility of Rejection Grading in Uterus Transplantation: A Multicenter Study

Background

Diagnosis of rejection after uterus transplantation is based on histopathological examination of ectocervical biopsies. Inflammation at the stromal-epithelial interface is the backbone of the histopathological classification proposed by our group in 2017. However, the reproducibility of this grading scheme has not been tested, and it is unclear whether it covers the full morphological spectrum of rejection.

Methods

We present a multicenter study in which 5 pathologists from 4 uterus transplantation centers performed 2 rounds of grading on 145 and 48 cervical biopsies, respectively. Three of the centers provided biopsies. Additionally, the presence of perivascular stromal inflammation was recorded. During discussions after the first round, further histological lesions (venous endothelial inflammation and apoptosis) were identified for closer evaluation and added to the panel of lesions to score in the second round. All participants completed a questionnaire to explore current practices in handling and reporting uterus transplant biopsies.

Results

Cervical biopsies were commonly performed in all centers to monitor rejection. Intraobserver reproducibility of rejection grading (performed by 1 rater) was excellent, whereas interobserver reproducibility was moderate and did not improve in the second round. Reproducibility of perivascular stromal inflammation was moderate but unsatisfactory for venous endothelial inflammation and apoptosis. All lesions were more frequent in, but not restricted to, biopsies with rejection patterns.

Conclusions

Grading of rejection in cervical biopsies is reproducible and applicable to biopsies from different centers. Diagnosis of rejection may be improved by adding further histological lesions to the grading system; however, lesions require rigorous consensus definition.

Applications of Artificial Intelligence in Breast Pathology

CONTEXT.—

Increasing implementation of whole slide imaging together with digital workflow and advances in computing capacity enable the use of artificial intelligence (AI) in pathology, including breast pathology. Breast pathologists often face a significant workload, with diagnosis complexity, tedious repetitive tasks, and semiquantitative evaluation of biomarkers. Recent advances in developing AI algorithms have provided promising approaches to meet the demand in breast pathology.

OBJECTIVE.—

To provide an updated review of AI in breast pathology. We examined the success and challenges of current and potential AI applications in diagnosing and grading breast carcinomas and other pathologic changes, detecting lymph node metastasis, quantifying breast cancer biomarkers, predicting prognosis and therapy response, and predicting potential molecular changes.

DATA SOURCES.—

We obtained data and information by searching and reviewing literature on AI in breast pathology from PubMed and based our own experience.

CONCLUSIONS.—

With the increasing application in breast pathology, AI not only assists in pathology diagnosis to improve accuracy and reduce pathologists' workload, but also provides new information in predicting prognosis and therapy response.

Artifact Augmentation for Learning-based Quality Control of Whole Slide Images

The acquisition of whole slide images is prone to artifacts that can require human control and re-scanning, both in clinical workflows and in research-oriented settings. Quality control algorithms are a first step to overcome this challenge, as they limit the use of low quality images. Developing quality control systems in histopathology is not straightforward, also due to the limited availability of data related to this topic. We address the problem by proposing a tool to augment data with artifacts. The proposed method seamlessly generates and blends artifacts from an external library to a given histopathology dataset. The datasets augmented by the blended artifacts are then used to train an artifact detection network in a supervised way. We use the YOLOv5 model for the artifact detection with a slightly modified training pipeline. The proposed tool can be extended into a complete framework for the quality assessment of whole slide images.Clinical relevance- The proposed method may be useful for the initial quality screening of whole slide images. Each year, millions of whole slide images are acquired and digitized worldwide. Numerous of them contain artifacts affecting the following AI-oriented analysis. Therefore, a tool operating at the acquisition phase and improving the initial quality assessment is crucial to increase the performance of digital pathology algorithms, e.g., early cancer diagnosis.

Discovery of Novel Digital Biomarkers for Type 2 Diabetic Nephropathy Classification via Integration of Urinary Proteomics and Pathology

Background

The heterogeneous phenotype of diabetic nephropathy (DN) from type 2 diabetes complicates appropriate treatment approaches and outcome prediction. Kidney histology helps diagnose DN and predict its outcomes, and an artificial intelligence (AI)-based approach will maximize clinical utility of histopathological evaluation. Herein, we addressed whether AI-based integration of urine proteomics and image features improves DN classification and its outcome prediction, altogether augmenting and advancing pathology practice.

Methods

We studied whole slide images (WSIs) of periodic acid-Schiff-stained kidney biopsies from 56 DN patients with associated urinary proteomics data. We identified urinary proteins differentially expressed in patients who developed end-stage kidney disease (ESKD) within two years of biopsy. Extending our previously published human-AI-loop pipeline, six renal sub-compartments were computationally segmented from each WSI. Hand-engineered image features for glomeruli and tubules, and urinary protein measurements, were used as inputs to deep-learning frameworks to predict ESKD outcome. Differential expression was correlated with digital image features using the Spearman rank sum coefficient.

Results

A total of 45 urinary proteins were differentially detected in progressors, which was most predictive of ESKD (AUC=0.95), while tubular and glomerular features were less predictive (AUC=0.71 and AUC=0.63, respectively). Accordingly, a correlation map between canonical cell-type proteins, such as epidermal growth factor and secreted phosphoprotein 1, and AI-based image features was obtained, which supports previous pathobiological results.

Conclusions

Computational method-based integration of urinary and image biomarkers may improve the pathophysiological understanding of DN progression as well as carry clinical implications in histopathological evaluation.

Clinical Relevance of Computationally Derived Attributes of Peritubular Capillaries from Kidney Biopsies

Key Points

Computational image analysis allows for the extraction of new information from whole-slide images with potential clinical relevance. Peritubular capillary (PTC) density is decreased in areas of interstitial fibrosis and tubular atrophy when measured in interstitial fractional space. PTC shape (aspect ratio) is associated with clinical outcome in glomerular diseases.

Background

The association between peritubular capillary (PTC) density and disease progression has been studied in a variety of kidney diseases using immunohistochemistry. However, other PTC attributes, such as PTC shape, have not been explored yet. The recent development of computer vision techniques provides the opportunity for the quantification of PTC attributes using conventional stains and whole-slide images.

Methods

To explore the relationship between PTC characteristics and clinical outcome, n =280 periodic acid–Schiff-stained kidney biopsies (88 minimal change disease, 109 focal segmental glomerulosclerosis, 46 membranous nephropathy, and 37 IgA nephropathy) from the Nephrotic Syndrome Study Network digital pathology repository were computationally analyzed. A previously validated deep learning model was applied to segment cortical PTCs. Average PTC aspect ratio (PTC major to minor axis ratio), size (PTC pixels per PTC segmentation), and density (PTC pixels per unit cortical area) were computed for each biopsy. Cox proportional hazards models were used to assess associations between these PTC parameters and outcome (40% eGFR decline or kidney failure). Cortical PTC characteristics and interstitial fractional space PTC density were compared between areas of interstitial fibrosis and tubular atrophy (IFTA) and areas without IFTA.

Results

When normalized PTC aspect ratio was below 0.6, a 0.1, increase in normalized PTC aspect ratio was significantly associated with disease progression, with a hazard ratio (95% confidence interval) of 1.28 (1.04 to 1.59) (P = 0.019), while PTC density and size were not significantly associated with outcome. Interstitial fractional space PTC density was lower in areas of IFTA compared with non-IFTA areas.

Conclusions

Computational image analysis enables quantification of the status of the kidney microvasculature and the discovery of a previously unrecognized PTC biomarker (aspect ratio) of clinical outcome.

Whole-Slide Cytometric Feature Mapping for Distinguishing Tumor Genomic Subtypes in Head and Neck Squamous Cell Carcinoma Whole-Slide Images

Head and neck squamous cell carcinoma (HNSCC) is a heterogeneous disease where, in advanced stages, clinical and pathologic stages do not correlate with outcome. Molecular and genomic biomarkers for HNSCC classification have shown promise for prognostic and therapeutic applications. This study utilized automated image analysis techniques in whole-slide images of HNSCC tumors to identify relationships between cytometric features and genomic phenotypes. Hematoxylin and eosin-stained slides of HNSCC tumors (N = 49) were obtained from The Cancer Imaging Archive, along with accompanying clinical, pathologic, genomic, and proteomic reports. Automated nuclear detection was performed across the entirety of slides, and cytometric feature maps were generated. Forty-one cytometric features were evaluated for associations with tumor grade, tumor stage, tumor subsite, and integrated genomic subtype. Thirty-two features demonstrated significant association with integrated genomic subtype when corrected for multiple comparisons. In particular, the basal subtype was visually distinguishable from the chromosomal instability and immune subtypes based on cytometric feature measurements. No features were significantly associated with tumor grade, stage, or subsite. This study provides preliminary evidence that features derived from tissue pathology slides could provide insights into genomic phenotypes of HNSCC.

Efficient quality control of whole slide pathology images with human-in-the-loop training

Histopathology whole slide images (WSIs) are being widely used to develop deep learning-based diagnostic solutions, especially for precision oncology. Most of these diagnostic softwares are vulnerable to biases and impurities in the training and test data which can lead to inaccurate diagnoses. For instance, WSIs contain multiple types of tissue regions, at least some of which might not be relevant to the diagnosis. We introduce HistoROI, a robust yet lightweight deep learning-based classifier to segregate WSI into 6 broad tissue regions-epithelium, stroma, lymphocytes, adipose, artifacts, and miscellaneous. HistoROI is trained using a novel human in-the-loop and active learning paradigm that ensures variations in training data for labeling efficient generalization. HistoROI consistently performs well across multiple organs, despite being trained on only a single dataset, demonstrating strong generalization. Further, we have examined the utility of HistoROI in improving the performance of downstream deep learning-based tasks using the CAMELYON breast cancer lymph node and TCGA lung cancer datasets. For the former dataset, the area under the receiver operating characteristic curve (AUC) for metastasis versus normal tissue of a neural network trained using weakly supervised learning increased from 0.88 to 0.92 by filtering the data using HistoROI. Similarly, the AUC increased from 0.88 to 0.93 for the classification between adenocarcinoma and squamous cell carcinoma on the lung cancer dataset. We also found that the performance of the HistoROI improves upon HistoQC for artifact detection on a test dataset of 93 annotated WSIs. The limitations of the proposed model are analyzed, and potential extensions are also discussed.

Computational pathology, new horizons and challenges for anatomical pathology

The emergence of digital pathology environments and the application of computer vision to the analysis of histological sections has given rise to a new area of Anatomical Pathology, termed Computational Pathology. Advances in Computational Pathology may substantially change the routine of Anatomical Pathology laboratories and the work profile of the pathologist.

Recommendations on compiling test datasets for evaluating artificial intelligence solutions in pathology

Artificial intelligence (AI) solutions that automatically extract information from digital histology images have shown great promise for improving pathological diagnosis. Prior to routine use, it is important to evaluate their predictive performance and obtain regulatory approval. This assessment requires appropriate test datasets. However, compiling such datasets is challenging and specific recommendations are missing. A committee of various stakeholders, including commercial AI developers, pathologists, and researchers, discussed key aspects and conducted extensive literature reviews on test datasets in pathology. Here, we summarize the results and derive general recommendations on compiling test datasets. We address several questions: Which and how many images are needed? How to deal with low-prevalence subsets? How can potential bias be detected? How should datasets be reported? What are the regulatory requirements in different countries? The recommendations are intended to help AI developers demonstrate the utility of their products and to help pathologists and regulatory agencies verify reported performance measures. Further research is needed to formulate criteria for sufficiently representative test datasets so that AI solutions can operate with less user intervention and better support diagnostic workflows in the future.

Role of tumor cell sialylation in pancreatic cancer progression

Pancreatic ductal adenocarcinoma (PDAC) is one of the deadliest malignancies and is currently the third leading cause of cancer death. The aggressiveness of PDAC stems from late diagnosis, early metastasis, and poor efficacy of current chemotherapies. Thus, there is an urgent need for effective biomarkers for early detection of PDAC and development of new therapeutic strategies. It has long been known that cellular glycosylation is dysregulated in pancreatic cancer cells, however, tumor-associated glycans and their cognate glycosylating enzymes have received insufficient attention as potential clinical targets. Aberrant glycosylation affects a broad range of pathways that underpin tumor initiation, metastatic progression, and resistance to cancer treatment. One of the prevalent alterations in the cancer glycome is an enrichment in a select group of sialylated glycans including sialylated, branched N-glycans, sialyl Lewis antigens, and sialylated forms of truncated O-glycans such as the sialyl Tn antigen. These modifications affect the activity of numerous cell surface receptors, which collectively impart malignant characteristics typified by enhanced cell proliferation, migration, invasion and apoptosis-resistance. Additionally, sialic acids on tumor cells engage inhibitory Siglec receptors on immune cells to dampen anti-tumor immunity, further promoting cancer progression. The goal of this review is to summarize the predominant changes in sialylation occurring in pancreatic cancer, the biological functions of sialylated glycoproteins in cancer pathogenesis, and the emerging strategies for targeting sialoglycans and Siglec receptors in cancer therapeutics.

MMO-Net (Multi-Magnification Organ Network): A use case for Organ Identification using Multiple Magnifications in Preclinical Pathology Studies

Identifying organs within histology images is a fundamental and non-trivial step in toxicological digital pathology workflows as multiple organs often appear on the same whole slide image (WSI). Previous works in automated tissue classification have investigated the use of single magnifications, and demonstrated limitations when attempting to identify small and contiguous organs at low magnifications. In order to overcome these shortcomings, we present a multi-magnification convolutional neural network (CNN), called MMO-Net, which employs context and cellular detail from different magnifications to facilitate the recognition of complex organs. Across N=320 WSI from 3 contract research organization (CRO) laboratories, we demonstrate state-of-the-art organ detection and segmentation performance of 7 rat organs with and without lesions: liver, kidney, thyroid gland, parathyroid gland, urinary bladder, salivary gland, and mandibular lymph node (AUROC=0.99-1.0 for all organs, Dice≥0.9 except parathyroid (0.73)). Evaluation takes place at both inter- and intra CRO levels, suggesting strong generalizability performance. Results are qualitatively reviewed using visualization masks to ensure separation of organs in close proximity (e.g., thyroid vs parathyroid glands). MMO-Net thus offers organ localization that serves as a potential quality control tool to validate WSI metadata and as a preprocessing step for subsequent organ-specific artificial intelligence (AI) use cases. To facilitate research in this area, all associated WSI and metadata used for this study are being made freely available, forming a first of its kind dataset for public use.

The fibrogenic niche in kidney fibrosis: components and mechanisms

Kidney fibrosis, characterized by excessive deposition of extracellular matrix (ECM) that leads to tissue scarring, is the final common outcome of a wide variety of chronic kidney diseases. Rather than being distributed uniformly across the kidney parenchyma, renal fibrotic lesions initiate at certain focal sites in which the fibrogenic niche is formed in a spatially confined fashion. This niche provides a unique tissue microenvironment that is orchestrated by a specialized ECM network consisting of de novo-induced matricellular proteins. Other structural elements of the fibrogenic niche include kidney resident and infiltrated inflammatory cells, extracellular vesicles, soluble factors and metabolites. ECM proteins in the fibrogenic niche recruit soluble factors including WNTs and transforming growth factor-β from the extracellular milieu, creating a distinctive profibrotic microenvironment. Studies using decellularized ECM scaffolds from fibrotic kidneys show that the fibrogenic niche autonomously promotes fibroblast proliferation, tubular injury, macrophage activation and endothelial cell depletion, pathological features that recapitulate key events in the pathogenesis of chronic kidney disease. The concept of the fibrogenic niche represents a paradigm shift in understanding of the mechanism of kidney fibrosis that could lead to the development of non-invasive biomarkers and novel therapies not only for chronic kidney disease, but also for fibrotic diseases of other organs.

Automated quality assessment of large digitised histology cohorts by artificial intelligence

Research using whole slide images (WSIs) of histopathology slides has increased exponentially over recent years. Glass slides from retrospective cohorts, some with patient follow-up data are digitised for the development and validation of artificial intelligence (AI) tools. Such resources, therefore, become very important, with the need to ensure that their quality is of the standard necessary for downstream AI development. However, manual quality control of large cohorts of WSIs by visual assessment is unfeasible, and whilst quality control AI algorithms exist, these focus on bespoke aspects of image quality, e.g. focus, or use traditional machine-learning methods, which are unable to classify the range of potential image artefacts that should be considered. In this study, we have trained and validated a multi-task deep neural network to automate the process of quality control of a large retrospective cohort of prostate cases from which glass slides have been scanned several years after production, to determine both the usability of the images at the diagnostic level (considered in this study to be the minimal standard for research) and the common image artefacts present. Using a two-layer approach, quality overlays of WSIs were generated from a quality assessment (QA) undertaken at patch-level at \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$5\times$$\end{document}5× magnification. From these quality overlays the slide-level quality scores were predicted and then compared to those generated by three specialist urological pathologists, with a Pearson correlation of 0.89 for overall ‘usability’ (at a diagnostic level), and 0.87 and 0.82 for focus and H&E staining quality scores respectively. To demonstrate its wider potential utility, we subsequently applied our QA pipeline to the TCGA prostate cancer cohort and to a colorectal cancer cohort, for comparison. Our model, designated as PathProfiler, indicates comparable predicted usability of images from the cohorts assessed (86–90% of WSIs predicted to be usable), and perhaps more significantly is able to predict WSIs that could benefit from an intervention such as re-scanning or re-staining for quality improvement. We have shown in this study that AI can be used to automate the process of quality control of large retrospective WSI cohorts to maximise their utility for research.

Integrating digital pathology into clinical practice

The field of anatomic pathology has been evolving in the last few decades and the advancements have been largely fostered by innovative technology. Immunohistochemistry enabled a paradigm shift in discovery and diagnostic evaluation, followed by booming genomic advancements which allowed for submicroscopic pathologic characterization, and now the field of digital pathology coupled with machine learning and big data acquisition is paving the way to revolutionize the pathology medical domain. Whole slide imaging (WSI) is a disruptive technology where glass slides are digitized to produce on-screen whole slide images. Specifically, in the past decade, there have been significant advances in digital pathology systems that have allowed this technology to promote integration into clinical practice. Whole slide images (WSI), or digital slides, can be viewed and navigated comparable to glass slides on a microscope, as digital files. Whole slide imaging has increased in adoption among pathologists, pathology departments, and scientists for clinical, educational, and research initiatives. Integration of digital pathology systems requires a coordinated effort with numerous stakeholders, not only within the pathology department, but across the entire enterprise. Each pathology department has distinct needs, use cases and blueprints, however the framework components and variables for successful clinical integration can be generalized across any organization seeking to undergo a digital transformation at any scale. This article will review those components and considerations for integrating digital pathology systems into clinical practice.

Assessment of glomerular morphological patterns by deep learning algorithms

Background

Compilation of different morphological lesion signatures is characteristic of renal pathology. Previous studies have documented the potential value of artificial intelligence (AI) in recognizing relatively clear-cut glomerular structures and patterns, such as segmental or global sclerosis or mesangial hypercellularity. This study aimed to test the capacity of deep learning algorithms to recognize complex glomerular structural changes that reflect common diagnostic dilemmas in nephropathology.

Methods

For this purpose, we defined nine classes of glomerular morphological patterns and trained twelve convolutional neuronal network (CNN) models on these. The two-step training process was done on a first dataset defined by an expert nephropathologist (12,253 images) and a second consensus dataset (11,142 images) defined by three experts in the field.

Results

The efficacy of CNN training was evaluated using another set with 180 consensus images, showing convincingly good classification results (kappa-values 0.838–0.938).

Furthermore, we elucidated the image areas decisive for CNN-based decision making by class activation maps. Finally, we demonstrated that the algorithm could decipher glomerular disease patterns coinciding in a single glomerulus (e.g. necrosis along with mesangial and endocapillary hypercellularity).

Conclusions

In summary, our model, focusing on glomerular lesions detectable by conventional microscopy, is the first sui generis to deploy deep learning as a reliable and promising tool in recognition of even discrete and/or overlapping morphological changes. Our results provide a stimulus for ongoing projects that integrate further input levels next to morphology (such as immunohistochemistry, electron microscopy, and clinical information) to develop a novel tool applicable for routine diagnostic nephropathology.

Supplementary Information

The online version contains supplementary material available at 10.1007/s40620-021-01221-9.

Computer extracted gland features from H&E predicts prostate cancer recurrence comparably to a genomic companion diagnostic test: a large multi-site study

Existing tools for post-radical prostatectomy (RP) prostate cancer biochemical recurrence (BCR) prognosis rely on human pathologist-derived parameters such as tumor grade, with the resulting inter-reviewer variability. Genomic companion diagnostic tests such as Decipher tend to be tissue destructive, expensive, and not routinely available in most centers. We present a tissue non-destructive method for automated BCR prognosis, termed "Histotyping", that employs computational image analysis of morphologic patterns of prostate tissue from a single, routinely acquired hematoxylin and eosin slide. Patients from two institutions (n = 214) were used to train Histotyping for identifying high-risk patients based on six features of glandular morphology extracted from RP specimens. Histotyping was validated for post-RP BCR prognosis on a separate set of n = 675 patients from five institutions and compared against Decipher on n = 167 patients. Histotyping was prognostic of BCR in the validation set (p < 0.001, univariable hazard ratio [HR] = 2.83, 95% confidence interval [CI]: 2.03-3.93, concordance index [c-index] = 0.68, median years-to-BCR: 1.7). Histotyping was also prognostic in clinically stratified subsets, such as patients with Gleason grade group 3 (HR = 4.09) and negative surgical margins (HR = 3.26). Histotyping was prognostic independent of grade group, margin status, pathological stage, and preoperative prostate-specific antigen (PSA) (multivariable p < 0.001, HR = 2.09, 95% CI: 1.40-3.10, n = 648). The combination of Histotyping, grade group, and preoperative PSA outperformed Decipher (c-index = 0.75 vs. 0.70, n = 167). These results suggest that a prognostic classifier for prostate cancer based on digital images could serve as an alternative or complement to molecular-based companion diagnostic tests.