Digital Pathology: A Regulatory Overview

Artificial Intelligence in Pathology: A Simple and Practical Guide

Artificial intelligence (AI) is having an increasing impact on the field of pathology, as computation techniques allow computers to perform tasks previously performed by people. Here, we offer a simple and practical guide to AI methods used in pathology, such as digital image analysis, next-generation sequencing, and natural language processing. We not only provide a comprehensive review, but also discuss relevant history and future directions of AI in pathology. We additionally provide a short tabular dictionary of AI terminology which will help practicing pathologists and researchers to understand this field.

A Regulatory Science Initiative to Harmonize and Standardize Digital Pathology and Machine Learning Processes to Speed up Clinical Innovation to Patients

Unlocking the full potential of pathology data by gaining computational access to histological pixel data and metadata (digital pathology) is one of the key promises of computational pathology. Despite scientific progress and several regulatory approvals for primary diagnosis using whole-slide imaging, true clinical adoption at scale is slower than anticipated. In the U.S., advances in digital pathology are often siloed pursuits by individual stakeholders, and to our knowledge, there has not been a systematic approach to advance the field through a regulatory science initiative. The Alliance for Digital Pathology (the Alliance) is a recently established, volunteer, collaborative, regulatory science initiative to standardize digital pathology processes to speed up innovation to patients. The purpose is: (1) to account for the patient perspective by including patient advocacy; (2) to investigate and develop methods and tools for the evaluation of effectiveness, safety, and quality to specify risks and benefits in the precompetitive phase; (3) to help strategize the sequence of clinically meaningful deliverables; (4) to encourage and streamline the development of ground-truth data sets for machine learning model development and validation; and (5) to clarify regulatory pathways by investigating relevant regulatory science questions. The Alliance accepts participation from all stakeholders, and we solicit clinically relevant proposals that will benefit the field at large. The initiative will dissolve once a clinical, interoperable, modularized, integrated solution (from tissue acquisition to diagnostic algorithm) has been implemented. In times of rapidly evolving discoveries, scientific input from subject-matter experts is one essential element to inform regulatory guidance and decision-making. The Alliance aims to establish and promote synergistic regulatory science efforts that will leverage diverse inputs to move digital pathology forward and ultimately improve patient care.

Development and applications of computer image analysis algorithms for scoring of PD-L1 immunohistochemistry

Immune checkpoint inhibitors targeting programmed cell death 1 (PD-1) and programmed cell death ligand 1 (PD-L1) have rapidly become integral to standard-of-care therapy for non-small cell lung cancer and other cancers. Immunohistochemical (IHC) staining of PD-L1 is currently the accepted and approved diagnostic assay for selecting patients for PD-L1/PD-1 axis therapies in certain indications. However, the inherent biological complexity of PD-L1 and the availability of several PD-L1 assays – each with different detection systems, platforms, scoring algorithms and cut-offs – have created challenges to ensure reliable and reproducible results based on subjective visual assessment by pathologists. The increasing adoption of computer technologies into the daily workflow of pathology provides an opportunity to leverage these tools towards improving the clinical value of PD-L1 IHC assays. This review describes several image analysis software programs of computer-aided PD-L1 scoring in the hope of driving further discussion and technological advancement in digital pathology and artificial intelligence approaches, particularly as precision medicine evolves to encompass accurate simultaneous assessment of multiple features of cancer cells and their interactions with the tumor microenvironment.

Quantitative Image Analysis of Human Epidermal Growth Factor Receptor 2 Immunohistochemistry for Breast Cancer: Guideline From the College of American Pathologists

CONTEXT.—

Advancements in genomic, computing, and imaging technology have spurred new opportunities to use quantitative image analysis (QIA) for diagnostic testing.

OBJECTIVE.—

To develop evidence-based recommendations to improve accuracy, precision, and reproducibility in the interpretation of human epidermal growth factor receptor 2 (HER2) immunohistochemistry (IHC) for breast cancer where QIA is used.

DESIGN.—

The College of American Pathologists (CAP) convened a panel of pathologists, histotechnologists, and computer scientists with expertise in image analysis, immunohistochemistry, quality management, and breast pathology to develop recommendations for QIA of HER2 IHC in breast cancer. A systematic review of the literature was conducted to address 5 key questions. Final recommendations were derived from strength of evidence, open comment feedback, expert panel consensus, and advisory panel review.

RESULTS.—

Eleven recommendations were drafted: 7 based on CAP laboratory accreditation requirements and 4 based on expert consensus opinions. A 3-week open comment period received 180 comments from more than 150 participants.

CONCLUSIONS.—

To improve accurate, precise, and reproducible interpretation of HER2 IHC results for breast cancer, QIA and procedures must be validated before implementation, followed by regular maintenance and ongoing evaluation of quality control and quality assurance. HER2 QIA performance, interpretation, and reporting should be supervised by pathologists with expertise in QIA.

Computer-assisted image analysis of the tumor microenvironment on an oral tongue squamous cell carcinoma tissue microarray

•

Tissue segmentation can be achieved using a spatially registered cytokeratin mask.

•

Automated and manual cell counts and stain intensities were highly correlated.

•

Smokers had significantly stronger PD-L1 stain intensity and higher numbers TILs.

•

After radiotherapy, greater CD8+ TILs was associated with inferior survival.

Oral tongue squamous cell carcinoma (OTSCC) displays variable levels of immune cells within the tumor microenvironment. The quantity and localization of tumor infiltrating lymphocytes (TILs), specific functional TIL subsets (e.g., CD8+), and biomarker-expressing cells (e.g., PD-L1+) may have prognostic and predictive value. The purpose of this study was to evaluate the robustness and utility of computer-assisted image analysis tools to quantify and localize immunohistochemistry-based biomarkers within the tumor microenvironment on a tissue microarray (TMA). We stained a 91-patient OTSCC TMA with antibodies targeting CD3, CD4, CD8, FOXP3, IDO, and PD-L1. Cell populations were segmented into epithelial (tumor) or stromal compartments according to a mask derived from a pan-cytokeratin stain. Definiens Tissue Studio was used to enumerate marker-positive cells or to quantify the staining intensity. Automated methods were validated against manual tissue segmentation, cell count, and stain intensity quantification. Univariate associations of cell count and stain intensity with smoking status, stage, overall survival (OS), and disease-free survival (DFS) were determined. Our results revealed that the accuracy of automated tissue segmentation was dependent on the distance of the tissue section from the cytokeratin mask and the proportion of the tissue containing tumor vs. stroma. Automated and manual cell counts and stain intensities were highly correlated (Pearson coefficient range: 0.46–0.90; p < 0.001). Within this OTSCC cohort, smokers had significantly stronger PD-L1 stain intensity and higher numbers of CD3+, CD4+ and FOXP3+ TILs. In the subset of patients who had received adjuvant radiotherapy, a higher number of CD8+ TILs was associated with inferior OS and DFS. Taken together, this proof-of-principle study demonstrates the robustness and utility of computer-assisted image analysis for high-throughput assessment of multiple IHC markers on TMAs, with potential implications for studies on prognostic and predictive biomarkers.

Impact of Altering Various Image Parameters on Human Epidermal Growth Factor Receptor 2 Image Analysis Data Quality

Introduction:

The quality of data obtained from image analysis can be directly affected by several preanalytical (e.g., staining, image acquisition), analytical (e.g., algorithm, region of interest [ROI]), and postanalytical (e.g., computer processing) variables. Whole-slide scanners generate digital images that may vary depending on the type of scanner and device settings. Our goal was to evaluate the impact of altering brightness, contrast, compression, and blurring on image analysis data quality.

Methods:

Slides from 55 patients with invasive breast carcinoma were digitized to include a spectrum of human epidermal growth factor receptor 2 (HER2) scores analyzed with Visiopharm (30 cases with score 0, 10 with 1+, 5 with 2+, and 10 with 3+). For all images, an ROI was selected and four parameters (brightness, contrast, JPEG2000 compression, out-of-focus blurring) then serially adjusted. HER2 scores were obtained for each altered image.

Results:

HER2 scores decreased with increased illumination, higher compression ratios, and increased blurring. HER2 scores increased with greater contrast. Cases with HER2 score 0 were least affected by image adjustments.

Conclusion:

This experiment shows that variations in image brightness, contrast, compression, and blurring can have major influences on image analysis results. Such changes can result in under- or over-scoring with image algorithms. Standardization of image analysis is recommended to minimize the undesirable impact such variations may have on data output.

Influence of study design on digital pathology image quality evaluation: the need to define a clinical task

Despite the current rapid advance in technologies for whole slide imaging, there is still no scientific consensus on the recommended methodology for image quality assessment of digital pathology slides. For medical images in general, it has been recommended to assess image quality in terms of doctors' success rates in performing a specific clinical task while using the images (clinical image quality, cIQ). However, digital pathology is a new modality, and already identifying the appropriate task is difficult. In an alternative common approach, humans are asked to do a simpler task such as rating overall image quality (perceived image quality, pIQ), but that involves the risk of nonclinically relevant findings due to an unknown relationship between the pIQ and cIQ. In this study, we explored three different experimental protocols: (1) conducting a clinical task (detecting inclusion bodies), (2) rating image similarity and preference, and (3) rating the overall image quality. Additionally, within protocol 1, overall quality ratings were also collected (task-aware pIQ). The experiments were done by diagnostic veterinary pathologists in the context of evaluating the quality of hematoxylin and eosin-stained digital pathology slides of animal tissue samples under several common image alterations: additive noise, blurring, change in gamma, change in color saturation, and JPG compression. While the size of our experiments was small and prevents drawing strong conclusions, the results suggest the need to define a clinical task. Importantly, the pIQ data collected under protocols 2 and 3 did not always rank the image alterations the same as their cIQ from protocol 1, warning against using conventional pIQ to predict cIQ. At the same time, there was a correlation between the cIQ and task-aware pIQ ratings from protocol 1, suggesting that the clinical experiment context (set by specifying the clinical task) may affect human visual attention and bring focus to their criteria of image quality. Further research is needed to assess whether and for which purposes (e.g., preclinical testing) task-aware pIQ ratings could substitute cIQ for a given clinical task.

Current State of the Regulatory Trajectory for Whole Slide Imaging Devices in the USA

The regulatory field for digital pathology (DP) has advanced significantly. A major milestone was accomplished when the FDA allowed the first vendor to market their device for primary diagnostic use in the USA and published in the classification order that this device, and substantially equivalent devices of this generic type, should be classified into class II instead of class III as previously proposed. The Digital Pathology Association (DPA) regulatory task force had a major role in the accomplishment of getting the application request for Whole Slide Imaging (WSI) Systems recommended for a de novo. This article reviews the past and emerging regulatory environment of WSI for clinical use in the USA. A WSI system with integrated subsystems is defined in the context of medical device regulations. The FDA technical performance assessment guideline is also discussed as well as parameters involved in analytical testing and clinical studies to demonstrate that WSI devices are safe and effective for clinical use.

iPathology cockpit diagnostic station: validation according to College of American Pathologists Pathology and Laboratory Quality Center recommendation at the Hospital Trust and University of Verona

Background

Validation of digital whole slide images is crucial to ensure that diagnostic performance is at least equivalent to that of glass slides and light microscopy. The College of American Pathologists Pathology and Laboratory Quality Center recently developed recommendations for internal digital pathology system validation. Following these guidelines we sought to validate the performance of a digital approach for routine diagnosis by using an iPad and digital control widescreen-assisted workstation through a pilot study.

Methods

From January 2014, 61 histopathological slides were scanned by ScanScope Digital Slides Scanner (Aperio, Vista, CA). Two independent pathologists performed diagnosis on virtual slides in front of a widescreen by using two computer devices (ImageScope viewing software) located to different Health Institutions (AOUI Verona) connected by local network and a remote image server using an iPad tablet (Aperio, Vista, CA), after uploading the Citrix receiver for iPad. Quality indicators related to image characters and work-flow of the e-health cockpit enterprise system were scored based on subjective (high vs poor) perception. The images were re-evaluated two weeks apart.

Results

The whole glass slides encountered 10 liver: hepatocarcinoma, 10 renal carcinoma, 10 gastric carcinoma and 10 prostate biopsies: adenocarcinoma, 5 excisional skin biopsies: melanoma, 5 lymph-nodes: lymphoma. 6 immuno- and 5 special stains were available for intra- and internet remote viewing. Scan times averaged two minutes and 54 seconds per slide (standard deviation 2 minutes 34 seconds). Megabytes ranged from 256 to 680 (mean 390) per slide storage. Reliance on glass slide, image quality (resolution and color fidelity), slide navigation time, simultaneous viewers in geographically remote locations were considered of high performance score. Side by side comparisons between diagnosis performed on tissue glass slides versus widescreen were excellent showing an almost perfect concordance (0.81, kappa index).

Conclusions

We validated our institutional digital pathology system for routine diagnostic facing with whole slide images in a cockpit enterprise digital system or iPad tablet. Computer widescreens are better for diagnosing scanned glass slide that iPad. For urgent requests, iPad may be used. Legal aspects have to be soon faced with to permit the clinical use of this technology in a manner that does not compromise patient care.

Evaluation of mitotic activity index in breast cancer using whole slide digital images

Introduction

Mitotic Activity Index (MAI) is an important independent prognostic factor and an integral part of the breast cancer grading system. Thus, correct estimation of this prognostically relevant feature is essential for guiding treatment decision and assessing patient prognosis.

The aim of this study was to validate the use of high resolution Whole Slide Images (WSI) in estimating MAI in breast cancer specimens.

Methods

MAI was evaluated in 100 consecutive breast cancer specimens by three observers on two occasions, microscopically and on WSI with a wash out period of 4 months. MAI was also translated to mitotic scores as in grading. Inter- and intra-observer agreement between microscopic and digital MAI counts and scores was measured.

Results

Almost perfect inter-observer agreements were obtained from counting MAI using a conventional microscope (intra-class correlation coefficient (ICCC) 0.879) as well as on WSI (ICCC 0.924). K coefficients reflected good inter-observer agreements among observers' microscopic mitotic scores (average kappa 0.642). Comparable results were also observed among digital mitotic scores (average kappa 0.635). There was strong to perfect intra-observer agreements between MAI counts and mitotic scores for the two diagnostic modalities (ICCC 0.716–0.863, kappa 0.506–0.617). There were no significant differences in mitotic scores using both diagnostic modalities.

Conclusion

Scoring mitoses using WSI in breast cancer seems to be just as reliable and reproducible as when using a microscope. Further development of software and image quality will definitely encourage the use of WSI in routine pathology practice.

Whole-slide imaging and automated image analysis: considerations and opportunities in the practice of pathology

Digital pathology, the practice of pathology using digitized images of pathologic specimens, has been transformed in recent years by the development of whole-slide imaging systems, which allow for the evaluation and interpretation of digital images of entire histologic sections. Applications of whole-slide imaging include rapid transmission of pathologic data for consultations and collaborations, standardization and distribution of pathologic materials for education, tissue specimen archiving, and image analysis of histologic specimens. Histologic image analysis allows for the acquisition of objective measurements of histomorphologic, histochemical, and immunohistochemical properties of tissue sections, increasing both the quantity and quality of data obtained from histologic assessments. Currently, numerous histologic image analysis software solutions are commercially available. Choosing the appropriate solution is dependent on considerations of the investigative question, computer programming and image analysis expertise, and cost. However, all studies using histologic image analysis require careful consideration of preanalytical variables, such as tissue collection, fixation, and processing, and experimental design, including sample selection, controls, reference standards, and the variables being measured. The fields of digital pathology and histologic image analysis are continuing to evolve, and their potential impact on pathology is still growing. These methodologies will increasingly transform the practice of pathology, allowing it to mature toward a quantitative science. However, this maturation requires pathologists to be at the forefront of the process, ensuring their appropriate application and the validity of their results. Therefore, histologic image analysis and the field of pathology should co-evolve, creating a symbiotic relationship that results in high-quality reproducible, objective data.

Tissue pattern recognition error rates and tumor heterogeneity in gastric cancer

The anatomic pathology discipline is slowly moving toward a digital workflow, where pathologists will evaluate whole-slide images on a computer monitor rather than glass slides through a microscope. One of the driving factors in this workflow is computer-assisted scoring, which depends on appropriate selection of regions of interest. With advances in tissue pattern recognition techniques, a more precise region of the tissue can be evaluated, no longer bound by the pathologist's patience in manually outlining target tissue areas. Pathologists use entire tissues from which to determine a score in a region of interest when making manual immunohistochemistry assessments. Tissue pattern recognition theoretically offers this same advantage; however, error rates exist in any tissue pattern recognition program, and these error rates contribute to errors in the overall score. To provide a real-world example of tissue pattern recognition, 11 HER2-stained upper gastrointestinal malignancies with high heterogeneity were evaluated. HER2 scoring of gastric cancer was chosen due to its increasing importance in gastrointestinal disease. A method is introduced for quantifying the error rates of tissue pattern recognition. The trade-off between fully sampling tumor with a given tissue pattern recognition error rate versus randomly sampling a limited number of fields of view with higher target accuracy was modeled with a Monte-Carlo simulation. Under most scenarios, stereological methods of sampling-limited fields of view outperformed whole-slide tissue pattern recognition approaches for accurate immunohistochemistry analysis. The importance of educating pathologists in the use of statistical sampling is discussed, along with the emerging role of hybrid whole-tissue imaging and stereological approaches.

Whole-slide imaging: routine pathologic diagnosis

Digital pathology systems offer pathologists an alternate, emerging mechanism to manage and interpret information. They offer increasingly fast and scalable hardware platforms for slide scanning and software that facilitates remote viewing, slide conferencing, archiving, and image analysis. Deployed initially and validated largely within the research and biopharmaceutical industries, WSI is increasingly being implemented for direct patient care. Improvements in image quality, scan times, and imageviewing browsers will hopefully allow pathologists to more seamlessly convert to digital pathology, much like our radiology colleagues have done before us. However, WSI creates both opportunities and challenges. Although niche applications of WSI technology for clinical, educational, and research purposes are clearly successful, it is evident that several areas still require attention and careful consideration before more widespread clinical adoption of WSI takes place. These include regulatory issues, development of standards of practice and validation guidelines, workflow modifications, as well as defining situations where WSI technology will really improve practice in a cost-effective way. Current progress on these and other issues, along with improving technology, will no doubt pave the way for increased adoption over the next decade, allowing the pathology community as a whole to harness the true potential of WSI for patient care. The digital decade will likely redefine how pathology is practiced and the role of the pathologist.