Colour in digital pathology: a review

Development of a multi-scanner facility for data acquisition for digital pathology artificial intelligence

Whole slide imaging (WSI) of pathology glass slides using high-resolution scanners has enabled the large-scale application of artificial intelligence (AI) in pathology, to support the detection and diagnosis of disease, potentially increasing efficiency and accuracy in tissue diagnosis. Despite the promise of AI, it has limitations. 'Brittleness' or sensitivity to variation in inputs necessitates that large amounts of data are used for training. AI is often trained on data from different scanners but not usually by replicating the same slide across scanners. The utilisation of multiple WSI instruments to produce digital replicas of the same slides will make more comprehensive datasets and may improve the robustness and generalisability of AI algorithms as well as reduce the overall data requirements of AI training. To this end, the National Pathology Imaging Cooperative (NPIC) has built the AI FORGE (Facilitating Opportunities for Robust Generalisable data Emulation), a unique multi-scanner facility embedded in a clinical site in the NHS to (1) compare scanner performance, (2) replicate digital pathology image datasets across WSI systems, and (3) support the evaluation of clinical AI algorithms. The NPIC AI FORGE currently comprises 15 scanners from nine manufacturers. It can generate approximately 4,000 WSI images per day (approximately 7 TB of image data). This paper describes the process followed to plan and build such a facility. © 2024 The Author(s). The Journal of Pathology published by John Wiley & Sons Ltd on behalf of The Pathological Society of Great Britain and Ireland.

Deep Learning Models Capture Histological Disease Activity in Crohn's Disease and Ulcerative Colitis with High Fidelity

BACKGROUND AND AIMS

Histological disease activity in inflammatory bowel disease [IBD] is associated with clinical outcomes and is an important endpoint in drug development. We developed deep learning models for automating histological assessments in IBD.

METHODS

Histology images of intestinal mucosa from phase 2 and phase 3 clinical trials in Crohn's disease [CD] and ulcerative colitis [UC] were used to train artificial intelligence [AI] models to predict the Global Histology Activity Score [GHAS] for CD and Geboes histopathology score for UC. Three AI methods were compared. AI models were evaluated on held-back testing sets, and model predictions were compared against an expert central reader and five independent pathologists.

RESULTS

The model based on multiple instance learning and the attention mechanism [SA-AbMILP] demonstrated the best performance among competing models. AI-modelled GHAS and Geboes subgrades matched central readings with moderate to substantial agreement, with accuracies ranging from 65% to 89%. Furthermore, the model was able to distinguish the presence and absence of pathology across four selected histological features, with accuracies for colon in both CD and UC ranging from 87% to 94% and for CD ileum ranging from 76% to 83%. For both CD and UC and across anatomical compartments [ileum and colon] in CD, comparable accuracies against central readings were found between the model-assigned scores and scores by an independent set of pathologists.

CONCLUSIONS

Deep learning models based upon GHAS and Geboes scoring systems were effective at distinguishing between the presence and absence of IBD microscopic disease activity.

Registered multi-device/staining histology image dataset for domain-agnostic machine learning models

Variations in color and texture of histopathology images are caused by differences in staining conditions and imaging devices between hospitals. These biases decrease the robustness of machine learning models exposed to out-of-domain data. To address this issue, we introduce a comprehensive histopathology image dataset named PathoLogy Images of Scanners and Mobile phones (PLISM). The dataset consisted of 46 human tissue types stained using 13 hematoxylin and eosin conditions and captured using 13 imaging devices. Precisely aligned image patches from different domains allowed for an accurate evaluation of color and texture properties in each domain. Variation in PLISM was assessed and found to be significantly diverse across various domains, particularly between whole-slide images and smartphones. Furthermore, we assessed the improvement in domain shift using a convolutional neural network pre-trained on PLISM. PLISM is a valuable resource that facilitates the precise evaluation of domain shifts in digital pathology and makes significant contributions towards the development of robust machine learning models that can effectively address challenges of domain shift in histological image analysis.

Immune subtyping of melanoma whole slide images using multiple instance learning

Determining early-stage prognostic markers and stratifying patients for effective treatment are two key challenges for improving outcomes for melanoma patients. Previous studies have used tumour transcriptome data to stratify patients into immune subgroups, which were associated with differential melanoma specific survival and potential predictive biomarkers. However, acquiring transcriptome data is a time-consuming and costly process. Moreover, it is not routinely used in the current clinical workflow. Here, we attempt to overcome this by developing deep learning models to classify gigapixel haematoxylin and eosin (H&E) stained pathology slides, which are well established in clinical workflows, into these immune subgroups. We systematically assess six different multiple instance learning (MIL) frameworks, using five different image resolutions and three different feature extraction methods. We show that pathology-specific self-supervised models using 10x resolution patches generate superior representations for the classification of immune subtypes. In addition, in a primary melanoma dataset, we achieve a mean area under the receiver operating characteristic curve (AUC) of 0.80 for classifying histopathology images into 'high' or 'low immune' subgroups and a mean AUC of 0.82 in an independent TCGA melanoma dataset. Furthermore, we show that these models are able to stratify patients into 'high' and 'low immune' subgroups with significantly different melanoma specific survival outcomes (log rank test, P< 0.005). We anticipate that MIL methods will allow us to find new biomarkers of high importance, act as a tool for clinicians to infer the immune landscape of tumours and stratify patients, without needing to carry out additional expensive genetic tests.

Solution-free and simplified H&E staining using a hydrogel-based stamping technology

Hematoxylin and eosin (H&E) staining has been widely used as a fundamental and essential tool for diagnosing diseases and understanding biological phenomena by observing cellular arrangements and tissue morphological changes. However, conventional staining methods commonly involve solution-based, complex, multistep processes that are susceptible to user-handling errors. Moreover, inconsistent staining results owing to staining artifacts pose real challenges for accurate diagnosis. This study introduces a solution-free H&E staining method based on agarose hydrogel patches that is expected to represent a valuable tool to overcome the limitations of the solution-based approach. Using two agarose gel-based hydrogel patches containing hematoxylin and eosin dyes, H&E staining can be performed through serial stamping processes, minimizing color variation from handling errors. This method allows easy adjustments of the staining color by controlling the stamping time, effectively addressing variations in staining results caused by various artifacts, such as tissue processing and thickness. Moreover, the solution-free approach eliminates the need for water, making it applicable even in environmentally limited middle- and low-income countries, while still achieving a staining quality equivalent to that of the conventional method. In summary, this hydrogel-based H&E staining method can be used by researchers and medical professionals in resource-limited settings as a powerful tool to diagnose and understand biological phenomena.

Digitization of Pathology Labs: A Review of Lessons Learned

Pathology laboratories are increasingly using digital workflows. This has the potential of increasing laboratory efficiency, but the digitization process also involves major challenges. Several reports have been published describing the individual experiences of specific laboratories with the digitization process. However, a comprehensive overview of the lessons learned is still lacking. We provide an overview of the lessons learned for different aspects of the digitization process, including digital case management, digital slide reading, and computer-aided slide reading. We also cover metrics used for monitoring performance and pitfalls and corresponding values observed in practice. The overview is intended to help pathologists, information technology decision makers, and administrators to benefit from the experiences of others and to implement the digitization process in an optimal way to make their own laboratory future-proof.

A workflow for the creation of photorealistic 3D cadaveric models using photogrammetry

Three-dimensional (3D) representations of anatomical specimens are increasingly used as learning resources. Photogrammetry is a well-established technique that can be used to generate 3D models and has only been recently applied to produce visualisations of cadaveric specimens. This study has developed a semi-standardised photogrammetry workflow to produce photorealistic models of human specimens. Eight specimens, each with unique anatomical characteristics, were successfully digitised into interactive 3D models using the described workflow and the strengths and limitations of the technique are described. Various tissue types were reconstructed with apparent preservation of geometry and texture which visually resembled the original specimen. Using this workflow, an institution could digitise their existing cadaveric resources, facilitating the delivery of novel educational experiences.

Assessment of Color Reproducibility and Mitigation of Color Variation in Whole Slide Image Scanners for Toxicologic Pathology

Digital pathology workflows in toxicologic pathology rely on whole slide images (WSIs) from histopathology slides. Inconsistent color reproduction by WSI scanners of different models and from different manufacturers can result in different color representations and inter-scanner color variation in the WSIs. Although pathologists can accommodate a range of color variation during their evaluation of WSIs, color variability can degrade the performance of computational applications in digital pathology. In particular, color variability can compromise the generalization of artificial intelligence applications to large volumes of data from diverse sources. To address these challenges, we developed a process that includes two modules: (1) assessing the color reproducibility of our scanners and the color variation among them and (2) applying color correction to WSIs to minimize the color deviation and variation. Our process ensures consistent color reproduction across WSI scanners and enhances color homogeneity in WSIs, and its flexibility enables easy integration as a post-processing step following scanning by WSI scanners of different models and from different manufacturers.

Interoperable slide microscopy viewer and annotation tool for imaging data science and computational pathology

The exchange of large and complex slide microscopy imaging data in biomedical research and pathology practice is impeded by a lack of data standardization and interoperability, which is detrimental to the reproducibility of scientific findings and clinical integration of technological innovations. We introduce Slim, an open-source, web-based slide microscopy viewer that implements the internationally accepted Digital Imaging and Communications in Medicine (DICOM) standard to achieve interoperability with a multitude of existing medical imaging systems. We showcase the capabilities of Slim as the slide microscopy viewer of the NCI Imaging Data Commons and demonstrate how the viewer enables interactive visualization of traditional brightfield microscopy and highly-multiplexed immunofluorescence microscopy images from The Cancer Genome Atlas and Human Tissue Atlas Network, respectively, using standard DICOMweb services. We further show how Slim enables the collection of standardized image annotations for the development or validation of machine learning models and the visual interpretation of model inference results in the form of segmentation masks, spatial heat maps, or image-derived measurements.

Data-driven color augmentation for H&E stained images in computational pathology

Computational pathology targets the automatic analysis of Whole Slide Images (WSI). WSIs are high-resolution digitized histopathology images, stained with chemical reagents to highlight specific tissue structures and scanned via whole slide scanners. The application of different parameters during WSI acquisition may lead to stain color heterogeneity, especially considering samples collected from several medical centers. Dealing with stain color heterogeneity often limits the robustness of methods developed to analyze WSIs, in particular Convolutional Neural Networks (CNN), the state-of-the-art algorithm for most computational pathology tasks. Stain color heterogeneity is still an unsolved problem, although several methods have been developed to alleviate it, such as Hue-Saturation-Contrast (HSC) color augmentation and stain augmentation methods. The goal of this paper is to present Data-Driven Color Augmentation (DDCA), a method to improve the efficiency of color augmentation methods by increasing the reliability of the samples used for training computational pathology models. During CNN training, a database including over 2 million H&E color variations collected from private and public datasets is used as a reference to discard augmented data with color distributions that do not correspond to realistic data. DDCA is applied to HSC color augmentation, stain augmentation and H&E-adversarial networks in colon and prostate cancer classification tasks. DDCA is then compared with 11 state-of-the-art baseline methods to handle color heterogeneity, showing that it can substantially improve classification performance on unseen data including heterogeneous color variations.

An automated pipeline for extracting histological stain area fraction for voxelwise quantitative MRI-histology comparisons

The acquisition of MRI and histology in the same post-mortem tissue sample enables direct correlation between MRI and histologically-derived parameters. However, there still lacks a standardised automated pipeline to process histology data, with most studies relying on manual intervention. Here, we introduce an automated pipeline to extract a quantitative histological measure for staining density (stain area fraction, SAF) from multiple immunohistochemical (IHC) stains. The pipeline is designed to directly address key IHC artefacts related to tissue staining and slide digitisation. Here, the pipeline was applied to post-mortem human brain data from multiple subjects, relating MRI parameters (FA, MD, RD, AD, R2*, R1) to IHC slides stained for myelin, neurofilaments, microglia and activated microglia. Utilising high-quality MRI-histology co-registrations, we then performed whole-slide voxelwise comparisons (simple correlations, partial correlations and multiple regression analyses) between multimodal MRI- and IHC-derived parameters. The pipeline was found to be reproducible, robust to artefacts and generalisable across multiple IHC stains. Our partial correlation results suggest that some simple MRI-SAF correlations should be interpreted with caution, due to the co-localisation of other tissue features (e.g., myelin and neurofilaments). Further, we find activated microglia-a generic biomarker of inflammation-to consistently be the strongest predictor of high DTI FA and low RD, which may suggest sensitivity of diffusion MRI to aspects of neuroinflammation related to microglial activation, even after accounting for other microstructural changes (demyelination, axonal loss and general microglia infiltration). Together, these results show the utility of this approach in carefully curating IHC data and performing multimodal analyses to better understand microstructural relationships with MRI.

IT/QA and Regulatory Aspects of Digital Pathology: Results of the 8th ESTP International Workshop

Digital toxicologic histopathology has been broadly adopted in preclinical compound development for informal consultation and peer review. There is now increased interest in implementing the technology for good laboratory practice-regulated study evaluations. However, the implementation is not straightforward because systems and work processes require qualification and validation, with consideration also given to security. As a result of the high-throughput, high-volume nature of safety evaluations, computer performance, ergonomics, efficiency, and integration with laboratory information management systems are further key considerations. The European Society of Toxicologic Pathology organized an international expert workshop with participation by toxicologic pathologists, quality assurance/regulatory experts, and information technology experts to discuss qualification and validation of digital histopathology systems in a good laboratory practice environment, and to share the resulting conclusions broadly in the toxicologic pathology community.

H&E Multi-Laboratory Staining Variance Exploration with Machine Learning

In diagnostic histopathology, hematoxylin and eosin (H&E) staining is a critical process that highlights salient histological features. Staining results vary between laboratories regardless of the histopathological task, although the method does not change. This variance can impair the accuracy of algorithms and histopathologists’ time-to-insight. Investigating this variance can help calibrate stain normalization tasks to reverse this negative potential. With machine learning, this study evaluated the staining variance between different laboratories on three tissue types. We received H&E-stained slides from 66 different laboratories. Each slide contained kidney, skin, and colon tissue samples stained by the method routinely used in each laboratory. The samples were digitized and summarized as red, green, and blue channel histograms. Dimensions were reduced using principal component analysis. The data projected by principal components were inserted into the k-means clustering algorithm and the k-nearest neighbors classifier with the laboratories as the target. The k-means silhouette index indicated that K = 2 clusters had the best separability in all tissue types. The supervised classification result showed laboratory effects and tissue-type bias. Both supervised and unsupervised approaches suggested that tissue type also affected inter-laboratory variance. We suggest tissue type to also be considered upon choosing the staining and color-normalization approach.

A Quantitative Digital Analysis of Tissue Immune Components Reveals an Immunosuppressive and Anergic Immune Response with Relevant Prognostic Significance in Glioblastoma

Objectives: Immunostimulatory therapies using immune checkpoint blockers show clinical activity in a subset of glioblastoma (GBM) patients. Several inhibitory mechanisms play a relevant role in the immune response to GBM. With the objective of analyzing the tumor immune microenvironment and its clinical significance, we quantified several relevant immune biomarkers. Design: We studied 76 primary (non-recurrent) GBMs with sufficient clinical follow-up, including a subgroup of patients treated with a dendritic cell vaccine. The IDH-mutation, EGFR-amplification, and MGMT methylation statuses were determined. Several relevant immune biomarkers, including CD163, CD8, PD1, and PDL1, were quantified in representative selected areas by digital image analysis and semiquantitative evaluation. The percentage of each immune expression was calculated with respect to the total number of tumor cells. Results: All GBMs were wild-type IDH, with a subgroup of classical GBMs according to the EGFR amplification (44%). Morphologically, CD163 immunostained microglia and intratumor clusters of macrophages were observed. A significant direct correlation was found between the expression of CD8 and the mechanisms of lymphocyte immunosuppression, in such a way that higher values of CD8 were directly associated with higher values of CD163 (p < 0.001), PDL1 (0.026), and PD1 (0.007). In a multivariate analysis, high expressions of CD8+ (HR = 2.05, 95%CI (1.02−4.13), p = 0.034) and CD163+ cells (HR 2.50, 95%CI (1.29−4.85), p = 0.007), were associated with shorter survival durations. The expression of immune biomarkers was higher in the non-classical (non-EGFR amplified tumors) GBMs. Other relevant prognostic factors were age, receipt of the dendritic cell vaccine, and MGMT methylation status. Conclusions: In accordance with the inverse correlation between CD8 and survival and the direct correlation between effector cells and CD163 macrophages and immune-checkpoint expression, we postulate that CD8 infiltration could be placed in a state of anergy or lymphocytic inefficient activity. Furthermore, the significant inverse correlation between CD163 tissue concentration and survival explains the relevance of this type of immune cell when creating a strong immunosuppressive environment. This information may potentially be used to support the selection of patients for immunotherapy.

Quantitative Evaluation of Inflammatory Markers in Peri-Implantitis and Periodontitis Tissues: Digital vs. Manual Analysis—A Proof of Concept Study

Background and Objectives: In dentistry, the assessment of the histomorphometric features of periodontal (PD) and peri-implant (PI) lesions is important to evaluate their underlying pathogenic mechanism. The present study aimed to compare manual and digital methods of analysis in the evaluation of the inflammatory biomarkers in PI and PD lesions. Materials and Methods: PD and PI inflamed soft tissues were excised and processed for histological and immunohistochemical analyses for CD3+, CD4+, CD8+, CD15+, CD20+, CD68+, and CD138+. The obtained slides were acquired using a digital scanner. For each marker, 4 pictures per sample were extracted and the area fraction of the stained tissue was computed both manually using a 594-point counting grid (MC) and digitally using a dedicated image analysis software (DC). To assess the concordance between MC and DC, two blinded observers analysed a total of 200 pictures either with good quality of staining or with non-specific background noise. The inter and intraobserver concordance was evaluated using the intraclass coefficient and the agreement between MC and DC was assessed using the Bland–Altman plot. The time spent analysing each picture using the two methodologies by both observers was recorded. Further, the amount of each marker was compared between PI and PD with both methodologies. Results: The inter- and intraobserver concordance was excellent, except for images with background noise analysed using DC. MC and DC showed a satisfying concordance. DC was performed in half the time compared to MC. The morphological analysis showed a larger inflammatory infiltrate in PI than PD lesions. The comparison between PI and PD showed differences for CD68+ and CD138+ expression. Conclusions: DC could be used as a reliable and time-saving procedure for the immunohistochemical analysis of PD and PI soft tissues. When non-specific background noise is present, the experience of the pathologist may be still required.

Techniques for digital histological morphometry of the pineal gland

INTRODUCTION

The pineal gland is a small photo-neuro-endocrine organ. This study used human post-mortem pineal glands to microscopically assess immunohistochemical marker intensity and percentage of positivity using known and novel digital techniques.

MATERIALS AND METHODS

An experimental non-inferiority study has been performed on 72 pineal glands harvested from post-mortem examinations. The glands have been stained with glial fibrillary acidic protein (GFAP), synaptophysin (SYN), neuron-specific enolase (NSE), and neurofilament (NF). Slides were digitally scanned. Morphometric data were obtained using optical analysis, CaseViewer, ImageJ, and MorphoRGB RESULTS: Strong and statistically significant correlations were found and plotted using Bland-Altman diagrams between the two image analysis software in the case of mean percentage and intensity of GFAP, NSE, NF, and SYN.

DISCUSSIONS

Software such as SlideViewer and ImageJ, with our novel software MorphoRGB were used to perform histological morphometry of the pineal gland. Digital morphometry of a small organ such as the pineal gland is easy to do by using whole slide imaging (WSI) and digital image analysis software, with potential use in clinical settings. MorphoRGB provides slightly more accurate data than ImageJ and is more user-friendly regarding measurements of parenchyma percentage stained by immunohistochemistry. The results show that MorphoRGB is not inferior in functionality.

CONCLUSIONS

The described morphometric techniques have potential value in current practice, experimental small animal models and human pineal glands, or other small endocrine organs that can be fully included in a whole slide image. The software we used has applications in quantifying immunohistochemical stains.

The need for measurement science in digital pathology

Background

Pathology services experienced a surge in demand during the COVID-19 pandemic. Digitalisation of pathology workflows can help to increase throughput, yet many existing digitalisation solutions use non-standardised workflows captured in proprietary data formats and processed by black-box software, yielding data of varying quality. This study presents the views of a UK-led expert group on the barriers to adoption and the required input of measurement science to improve current practices in digital pathology.

Methods

With an aim to support the UK's efforts in digitalisation of pathology services, this study comprised: (1) a review of existing evidence, (2) an online survey of domain experts, and (3) a workshop with 42 representatives from healthcare, regulatory bodies, pharmaceutical industry, academia, equipment, and software manufacturers. The discussion topics included sample processing, data interoperability, image analysis, equipment calibration, and use of novel imaging modalities.

Findings

The lack of data interoperability within the digital pathology workflows hinders data lookup and navigation, according to 80% of attendees. All participants stressed the importance of integrating imaging and non-imaging data for diagnosis, while 80% saw data integration as a priority challenge. 90% identified the benefits of artificial intelligence and machine learning, but identified the need for training and sound performance metrics.Methods for calibration and providing traceability were seen as essential to establish harmonised, reproducible sample processing, and image acquisition pipelines. Vendor-neutral data standards were seen as a "must-have" for providing meaningful data for downstream analysis. Users and vendors need good practice guidance on evaluation of uncertainty, fitness-for-purpose, and reproducibility of artificial intelligence/machine learning tools. All of the above needs to be accompanied by an upskilling of the pathology workforce.

Conclusions

Digital pathology requires interoperable data formats, reproducible and comparable laboratory workflows, and trustworthy computer analysis software. Despite high interest in the use of novel imaging techniques and artificial intelligence tools, their adoption is slowed down by the lack of guidance and evaluation tools to assess the suitability of these techniques for specific clinical question. Measurement science expertise in uncertainty estimation, standardisation, reference materials, and calibration can help establishing reproducibility and comparability between laboratory procedures, yielding high quality data and providing higher confidence in diagnosis.

Deep Convolutional Neural Networks Detect Tumor Genotype from Pathological Tissue Images in Gastrointestinal Stromal Tumors

Gastrointestinal stromal tumors (GIST) are common mesenchymal tumors, and their effective treatment depends upon the mutational subtype of the KIT/PDGFRA genes. We established deep convolutional neural network (DCNN) models to rapidly predict drug-sensitive mutation subtypes from images of pathological tissue. A total of 5153 pathological images of 365 different GISTs from three different laboratories were collected and divided into training and validation sets. A transfer learning mechanism based on DCNN was used with four different network architectures, to identify cases with drug-sensitive mutations. The accuracy ranged from 87% to 75%. Cross-institutional inconsistency, however, was observed. Using gray-scale images resulted in a 7% drop in accuracy (accuracy 80%, sensitivity 87%, specificity 73%). Using images containing only nuclei (accuracy 81%, sensitivity 87%, specificity 73%) or cytoplasm (accuracy 79%, sensitivity 88%, specificity 67%) produced 6% and 8% drops in accuracy rate, respectively, suggesting buffering effects across subcellular components in DCNN interpretation. The proposed DCNN model successfully inferred cases with drug-sensitive mutations with high accuracy. The contribution of image color and subcellular components was also revealed. These results will help to generate a cheaper and quicker screening method for tumor gene testing.

Colorimetrical uncertainty estimation for the performance assessment of whole slide imaging scanners

Purpose: Whole slide imaging (WSI) scanners produce tissue slide images with a large field of view and a high resolution for pathologists to use in diagnoses. Color performance tests of these color imaging devices are necessary and can use stained tissue slides if the color truth is established using a hyperspectral imaging microscopy system (HIMS). The purpose of this study was to estimate the reproducibility uncertainty of CIELAB coordinates for a reference tissue slide measured by both the HIMS and a WSI scanner. Approach: We compared the color performances of the WSI scanner to those of the reference established by the HIMS using the International Commission on Illumination (Commission Internationale de l'Éclairage, or CIE) 1976 Δ E a b * color difference with the just noticeable color difference (JNCD, Δ E a b * ≤ 2 ), and the results from the overlap of the CIELAB coordinates' uncertainty within the error bar, with a coverage factor k = 2 . The reported uncertainty results from measurements and image registration uncertainties. Results: For the blank area common to the HIMS and the WSI average images, the color agreement was higher using the JNCD condition versus the CIELAB uncertainty overlap criterion (82% and 20% of the pixels in the images, respectively). This difference is explained by the fact that numerous pixels have CIELAB coordinates near one another but corresponding to CIELAB uncertainty values small enough not to overlap. In the colored area of the images, the JNCD condition was met for 0.19% of the pixels in the images, compared with 4.3% for the CIELAB uncertainty overlap criterion. Conclusions: The distribution of uncertainties on the CIELAB coordinates was broader for the HIMS compared with the WSI scanner. The WSI scanner had a systemic error in the color reproduction, which pointed to a potential inadequate color calibration of this device.

Identification of pollen taxa by different microscopy techniques

Melissopalynology is an important analytical method to identify botanical origin of honey. Pollen grain recognition is commonly performed by visual inspection by a trained person. An alternative method for visual inspection is automated pollen analysis based on the image analysis technique. Image analysis transfers visual information to mathematical descriptions. In this work, the suitability of three microscopic techniques for automatic analysis of pollen grains was studied. 2D and 3D morphological characteristics, textural and colour features, and extended depth of focus characteristics were used for the pollen discrimination. In this study, 7 botanical taxa and a total of 2482 pollen grains were evaluated. The highest correct classification rate of 93.05% was achieved using the phase contrast microscopy, followed by the dark field microscopy reaching 91.02%, and finally by the light field microscopy reaching 88.88%. The most significant discriminant characteristics were morphological (2D and 3D) and colour characteristics. Our results confirm the potential of using automatic pollen analysis to discriminate pollen taxa in honey. This work provides the basis for further research where the taxa dataset will be increased, and new descriptors will be studied.

Comparative Assessment of Digital Pathology Systems for Primary Diagnosis

Background:

Despite increasing interest in whole-slide imaging (WSI) over optical microscopy (OM), limited information on comparative assessment of various digital pathology systems (DPSs) is available.

Materials and Methods:

A comprehensive evaluation was undertaken to investigate the technical performance–assessment and diagnostic accuracy of four DPSs with an objective to establish the noninferiority of WSI over OM and find out the best possible DPS for clinical workflow.

Results:

A total of 2376 digital images, 15,775 image reads (OM - 3171 + WSI - 12,404), and 6100 diagnostic reads (OM - 1245, WSI - 4855) were generated across four DPSs (coded as DPS: 1, 2, 3, and 4) using a total 240 cases (604 slides). Onsite technical evaluation revealed successful scan rate: DPS3 < DPS2 < DPS4 < DPS1; mean scanning time: DPS4 < DPS1 < DPS2 < DPS3; and average storage space: DPS3 < DPS2 < DPS1 < DPS4. Overall diagnostic accuracy, when compared with the reference standard for OM and WSI, was 95.44% (including 2.48% minor and 2.08% major discordances) and 93.32% (including 4.28% minor and 2.4% major discordances), respectively. The difference between the clinically significant discordances by WSI versus OM was 0.32%. Major discordances were observed mostly using DPS4 and least in DPS1; however, the difference was statistically insignificant. Almost perfect (κ ≥ 0.8)/substantial (κ = 0.6–0.8) inter/intra-observer agreement between WSI and OM was observed for all specimen types, except cytology. Overall image quality was best for DPS1 followed by DPS4. Mean digital artifact rate was 6.8% (163/2376 digital images) and maximum artifacts were noted in DPS2 (n = 77) followed by DPS3 (n = 36). Most pathologists preferred viewing software of DPS1 and DPS2.

Conclusion:

WSI was noninferior to OM for all specimen types, except for cytology. Each DPS has its own pros and cons; however, DPS1 closely emulated the real-world clinical environment. This evaluation is intended to provide a roadmap to pathologists for the selection of the appropriate DPSs while adopting WSI.

Artificial Intelligence-Based Screening for Mycobacteria in Whole-Slide Images of Tissue Samples

OBJECTIVES

This study aimed to develop and validate a deep learning algorithm to screen digitized acid fast-stained (AFS) slides for mycobacteria within tissue sections.

METHODS

A total of 441 whole-slide images (WSIs) of AFS tissue material were used to develop a deep learning algorithm. Regions of interest with possible acid-fast bacilli (AFBs) were displayed in a web-based gallery format alongside corresponding WSIs for pathologist review. Artificial intelligence (AI)-assisted analysis of another 138 AFS slides was compared to manual light microscopy and WSI evaluation without AI support.

RESULTS

Algorithm performance showed an area under the curve of 0.960 at the image patch level. More AI-assisted reviews identified AFBs than manual microscopy or WSI examination (P < .001). Sensitivity, negative predictive value, and accuracy were highest for AI-assisted reviews. AI-assisted reviews also had the highest rate of matching the original sign-out diagnosis, were less time-consuming, and were much easier for pathologists to perform (P < .001).

CONCLUSIONS

This study reports the successful development and clinical validation of an AI-based digital pathology system to screen for AFBs in anatomic pathology material. AI assistance proved to be more sensitive and accurate, took pathologists less time to screen cases, and was easier to use than either manual microscopy or viewing WSIs.

Preservation and Processing of Intestinal Tissue for the Assessment of Histopathology

The intestine is often examined histologically in connection with allergies and in search for pathological changes. To be able to examine the intestine histologically with a microscope, it must be sampled and processed correctly. For microscopic analysis, the samples have to be cut into thin sections, stained, and mounted on slides. Since it is not possible to cut fresh samples without damaging them, they must first be fixed. The most common method, which is described herein, is the fixation in formalin with subsequent embedding in paraffin and staining of the slides with hematoxylin and eosin (H&E). Hematoxylin solutions (in this case Mayer's hemalum solution) stain the acidic components of the cell, i.e., cell nuclei, blue. The staining with eosin gives a pink staining of cytoplasm. This chapter describes the method of processing intestinal tissue for paraffin-embedding, sectioning, and staining with H&E. Tissue processing can be done in tissue processing machines or manually. We describe the manual processing that is often used for smaller batches of samples.

The Effect of Quality Control on Accuracy of Digital Pathology Image Analysis

Digital slide images produced from routine diagnostic histopathological preparations suffer from variation arising at every step of the processing pipeline. Typically, pathologists compensate for such variation using expert knowledge and experience, which is difficult to replicate in automated solutions. The extent to which inconsistencies affect image analysis is explored in this work, examining in detail, the results from a previously published algorithm automating the generation of tumor:stroma ratio (TSR) in colorectal clinical trial datasets. One dataset consisting of 2,211 cases and 106,268 expert-labelled images is used to identify quality issues, by visually inspecting cases where algorithm-pathologist agreement is lowest. Twelve categories are identified and used to analyze pathologist-algorithm agreement in relation to these categories. Of the 2,211 cases, 701 were found to be free from any image quality issues. Algorithm performance was then assessed, comparing pathologist agreement with image quality classification. It was found that agreement was lowest on poorly differentiated tissue, with a mean TSR difference of 0.25 (sd = 0.24). Removing images that contained quality issues increased accuracy from 80% to 83%, at the expense of reducing the dataset to 33,736 images (32%). Training the algorithm on the optimized dataset, prior to testing on all images saw a decrease in accuracy of 4%, indicating that the optimized dataset did not contain enough variation to generate a fully representative model. The results provide an in-depth perspective on image quality, highlighting the importance of the effects on downstream image analysis.

The Application, Challenges, and Advancement Toward Regulatory Acceptance of Digital Toxicologic Pathology: Results of the 7th ESTP International Expert Workshop (September 20-21, 2019)

With advancements in whole slide imaging technology and improved understanding of the features of pathologist workstations required for digital slide evaluation, many institutions are investigating broad digital pathology adoption. The benefits of digital pathology evaluation include remote access to study or diagnostic case materials and integration of analysis and reporting tools. Diagnosis based on whole slide images is established in human medical pathology, and the use of digital pathology in toxicologic pathology is increasing. However, there has not been broad adoption in toxicologic pathology, particularly in the context of regulatory studies, due to lack of precedence. To address this topic, as well as practical aspects, the European Society of Toxicologic Pathology coordinated an expert international workshop to assess current applications and challenges and outline a set of minimal requirements needed to gain future regulatory acceptance for the use of digital toxicologic pathology workflows in research and development, so that toxicologic pathologists can benefit from digital slide technology.

Experimental Assessment of Color Deconvolution and Color Normalization for Automated Classification of Histology Images Stained with Hematoxylin and Eosin

Histological evaluation plays a major role in cancer diagnosis and treatment. The appearance of H&E-stained images can vary significantly as a consequence of differences in several factors, such as reagents, staining conditions, preparation procedure and image acquisition system. Such potential sources of noise can all have negative effects on computer-assisted classification. To minimize such artefacts and their potentially negative effects several color pre-processing methods have been proposed in the literature-for instance, color augmentation, color constancy, color deconvolution and color transfer. Still, little work has been done to investigate the efficacy of these methods on a quantitative basis. In this paper, we evaluated the effects of color constancy, deconvolution and transfer on automated classification of H&E-stained images representing different types of cancers-specifically breast, prostate, colorectal cancer and malignant lymphoma. Our results indicate that in most cases color pre-processing does not improve the classification accuracy, especially when coupled with color-based image descriptors. Some pre-processing methods, however, can be beneficial when used with some texture-based methods like Gabor filters and Local Binary Patterns.

Color lens-free imaging using multi-wavelength illumination based phase retrieval

Accurate image reconstruction in color lens-free imaging has proven challenging. The color image reconstruction of a sample is impacted not only by how strongly the illumination intensity is absorbed at a given spectral range, but also by the lack of phase information recorded on the image sensor. We present a compact and cost-effective approach of addressing the need for phase retrieval to enable robust color image reconstruction in lens-free imaging. The amplitude images obtained at transparent wavelength bands are used to estimate the phase in highly absorbed wavelength bands. The accurate phase information, obtained through our iterative algorithm, removes the color artefacts due to twin-image noise in the reconstructed image and improves image reconstruction quality to allow accurate color reconstruction. This could enable the technique to be applied for imaging of stained pathology slides, an important tool in medical diagnostics.

Color standardization and optimization in whole slide imaging

Whole slide imaging (WSI) has various uses, including the development of decision support systems, image analysis, education, conferences, and remote diagnostics. It is also used to develop artificial intelligence using machine learning methods. In the clinical setting, however, many issues have hindered the implementation of WSI. These issues are becoming more important as WSI is gaining wider use in clinical practice, particularly with the implementation of artificial intelligence in pathological diagnosis. One of the most important issues is the standardization of color for WSI, which is an important component of digital pathology. In this paper, we review the major factors of color variation and how to evaluate and modify color variation to establish color standardization. There are five major reasons for color variation, which include specimen thickness, staining, scanner, viewer, and display. Recognizing that the color is not standardized is the first step towards standardization, and it is difficult to ascertain whether the appropriate color of the WSI is displayed at the reviewers' end.

A Point-of-Use Quality Assurance Tool for Digital Pathology Remote Working

Pathology services are facing pressures due to the COVID-19 pandemic. Digital pathology has the capability to meet some of these unprecedented challenges by allowing remote diagnoses to be made at home, during periods of social distancing or self-isolation. However, while digital pathology allows diagnoses to be made on standard computer screens, unregulated home environments may not be conducive for optimal viewing conditions. There is also a paucity of experimental evidence available to support the minimum display requirements for digital pathology. This study presents a Point-of-Use Quality Assurance (POUQA) tool for remote assessment of viewing conditions for reporting digital pathology slides. The tool is a psychophysical test combining previous work from successfully implemented quality assurance tools in both pathology and radiology to provide a minimally intrusive display screen validation task, before viewing digital slides. The test is specific to pathology assessment in that it requires visual discrimination between colors derived from hematoxylin and eosin staining, with a perceptual difference of ±1 delta E (dE). This tool evaluates the transfer of a 1 dE signal through the digital image display chain, including the observers’ contrast and color responses within the test color range. The web-based system has been rapidly developed and deployed as a response to the COVID-19 pandemic and may be used by anyone in the world to help optimize flexible working conditions at: http://www. virtualpathology.leeds.ac.uk/res earch/systems/pouqa/.

Colorimetrical uncertainty of a hyperspectral imaging microscopy system for assessing whole-slide imaging devices

A whole-slide imaging (WSI) device is a color medical imaging system whose application in digital pathology is to digitalize stained tissue samples into electronic images for pathologists to diagnose without using a conventional light microscope. Testing the color performance of a WSI device usually implies a color target with known truth that is compared with the device output to estimate color differences. Using stained tissue samples as color targets is challenging because the cellular features cannot be measured with ordinary spectroradiometers unless a hyperspectral imaging microscopy system (HIMS) is used. The goal of this study is to determine the colorimetrical uncertainty of such a reference HIMS that is designed to assess the color performance of WSI devices. A set of optical filters are used for that purpose. The color truth, in terms of spectral transmittance in the visible band, of the optical filters is measured by a reference spectroradiometer. The spectral transmittance is combined with a standard illuminant to generate colorimetrical measures using the CIEXYZ and CIELAB formulas. The differences between the reference HIMS and the reference spectroradiometer are evaluated using the CIE 1976 color difference formulas.

Digital Microscopy, Image Analysis, and Virtual Slide Repository

Advancements in technology and digitization have ushered in novel ways of enhancing tissue-based research via digital microscopy and image analysis. Whole slide imaging scanners enable digitization of histology slides to be stored in virtual slide repositories and to be viewed via computers instead of microscopes. Easier and faster sharing of histologic images for teaching and consultation, improved storage and preservation of quality of stained slides, and annotation of features of interest in the digital slides are just a few of the advantages of this technology. Combined with the development of software for digital image analysis, digital slides further pave the way for the development of tools that extract quantitative data from tissue-based studies. This review introduces digital microscopy and pathology, and addresses technical and scientific considerations in slide scanning, quantitative image analysis, and slide repositories. It also highlights the current state of the technology and factors that need to be taken into account to insure optimal utility, including preanalytical considerations and the importance of involving a pathologist in all major steps along the digital microscopy and pathology workflow.

Quantitative Histopathology of Stained Tissues using Color Spatial Light Interference Microscopy (cSLIM)

Tissue biopsy evaluation in the clinic is in need of quantitative disease markers for diagnosis and, most importantly, prognosis. Among the new technologies, quantitative phase imaging (QPI) has demonstrated promise for histopathology because it reveals intrinsic tissue nanoarchitecture through the refractive index. However, a vast majority of past QPI investigations have relied on imaging unstained tissues, which disrupts the established specimen processing. Here we present color spatial light interference microscopy (cSLIM) as a new whole-slide imaging modality that performs interferometric imaging on stained tissue, with a color detector array. As a result, cSLIM yields in a single scan both the intrinsic tissue phase map and the standard color bright-field image, familiar to the pathologist. Our results on 196 breast cancer patients indicate that cSLIM can provide stain-independent prognostic information from the alignment of collagen fibers in the tumor microenvironment. The effects of staining on the tissue phase maps were corrected by a mathematical normalization. These characteristics are likely to reduce barriers to clinical translation for the new cSLIM technology.

Whole-Section Tumor Micro-Architecture Analysis by a Two-Dimensional Phasor-Based Approach Applied to Polarization-Dependent Second Harmonic Imaging

Second Harmonic Generation (SHG) microscopy has gained much interest in the histopathology field since it allows label-free imaging of tissues simultaneously providing information on their morphology and on the collagen microarchitecture, thereby highlighting the onset of pathologies and diseases. A wide request of image analysis tools is growing, with the aim to increase the reliability of the analysis of the huge amount of acquired data and to assist pathologists in a user-independent way during their diagnosis. In this light, we exploit here a set of phasor-parameters that, coupled to a 2-dimensional phasor-based approach (μMAPPS, Microscopic Multiparametric Analysis by Phasor projection of Polarization-dependent SHG signal) and a clustering algorithm, allow to automatically recover different collagen microarchitectures in the tissues extracellular matrix. The collagen fibrils microscopic parameters (orientation and anisotropy) are analyzed at a mesoscopic level by quantifying their local spatial heterogeneity in histopathology sections (few mm in size) from two cancer xenografts in mice, in order to maximally discriminate different collagen organizations, allowing in this case to identify the tumor area with respect to the surrounding skin tissue. We show that the "fibril entropy" parameter, which describes the tissue order on a selected spatial scale, is the most effective in enlightening the tumor edges, opening the possibility of their automatic segmentation. Our method, therefore, combined with tissue morphology information, has the potential to become a support to standard histopathology in diseases diagnosis.

Development and Evaluation of a Novel Point-of-Use Quality Assurance Tool for Digital Pathology

Context.—

Flexible working at diverse or remote sites is a major advantage when reporting using digital pathology, but currently there is no method to validate the clinical diagnostic setting within digital microscopy.

Objective.—

To develop a preliminary Point-of-Use Quality Assurance (POUQA) tool designed specifically to validate the diagnostic setting for digital microscopy.

Design.—

We based the POUQA tool on the red, green, and blue (RGB) values of hematoxylin-eosin. The tool used 144 hematoxylin-eosin–colored, 5×5-cm patches with a superimposed random letter with subtly lighter RGB values from the background color, with differing levels of difficulty. We performed an initial evaluation across 3 phases within 2 pathology departments: 1 in the United Kingdom and 1 in Sweden.

Results.—

In total, 53 experiments were conducted across all phases resulting in 7632 test images viewed in all. Results indicated that the display, the user's visual system, and the environment each independently impacted performance. Performance was improved with reduction in natural light and through use of medical-grade displays.

Conclusions.—

The use of a POUQA tool for digital microscopy is essential to afford flexible working while ensuring patient safety. The color-contrast test provides a standardized method of comparing diagnostic settings for digital microscopy. With further planned development, the color-contrast test may be used to create a “Verified Login” for diagnostic setting validation.

Three-dimensional histopathological reconstruction of bladder tumours

BACKGROUND

Histopathological analysis is the cornerstone in bladder cancer (BCa) diagnosis. These analysis suffer from a moderate observer agreement in the staging of bladder cancer. Three-dimensional reconstructions have the potential to support the pathologists in visualizing spatial arrangements of structures, which may improve the interpretation of specimen. The aim of this study is to present three-dimensional (3D) reconstructions of histology images.

METHODS

En-bloc specimens of transurethral bladder tumour resections were formalin fixed and paraffin embedded. Specimens were cut into sections of 4 μm and stained with Hematoxylin and Eosin (H&E). With a Phillips IntelliSite UltraFast scanner, glass slides were digitized at 20x magnification. The digital images were aligned by performing rigid and affine image alignment. The tumour and the muscularis propria (MP) were manually delineated to create 3D segmentations. In conjunction with a 3D display, the results were visualized with the Vesalius3D interactive visualization application for a 3D workstation.

RESULTS

En-bloc resection was performed in 21 BCa patients. Per case, 26-30 sections were included for the reconstruction into a 3D volume. Five cases were excluded due to export problems, size of the dataset or condition of the tissue block. Qualitative evaluation suggested an accurate registration for 13 out of 16 cases. The segmentations allowed full 3D visualization and evaluation of the spatial relationship of the BCa tumour and the MP.

CONCLUSION

Digital scanning of en-bloc resected specimens allows a full-fledged 3D reconstruction and analysis and has a potential role to support pathologists in the staging of BCa.

Fast-Response and Reusable Oxytetracycline Colorimetric Strips Based on Nickel (II) Ions Immobilized Carboxymethylcellulose/Polyacrylonitrile Nanofibrous Membranes

Driven by economic interests, the abuse of antibiotics has become a significant concern for humans worldwide. As one of the most commonly used antibiotics, oxytetracycline (OTC) residue in animal-derived foods occurs occasionally, which has caused danger to humanity. However, there is still no simple and efficient solution to detect OTC residue. Here, an easily-operated colorimetric strategy for OTC detection was developed based on nickel ions (Ni2+) immobilized carboxymethylcellulose/polyacrylonitrile nanofibrous membranes (Ni@CMC/PAN NFMs). Owing to numerous O- and N-containing groups OTC has a strong tendency to complex with Ni2+ on the strips, inducing a color change from light green to yellow visible to the naked eye. The NFMs structural features, CMC functionalization process, and Ni2+ immobilization amount was carefully regulated to assure OTC detection whilst maintaining the inherent characteristics of NFMs. With the benefits of the large specific surface area (SSA) and small pore size of NFMs, the strips not only exhibited a rapid response (2 min), and low detection limit (5 nM) but also performed with good reversibility and selectivity concerning OTC detection over other antibiotics. The successful development of such enchanting nanofibrous materials may provide a new comprehension into the design and improvement of colorimetric strips.

Label-free quantitative evaluation of breast tissue using Spatial Light Interference Microscopy (SLIM)

Breast cancer is the most common type of cancer among women worldwide. The standard histopathology of breast tissue, the primary means of disease diagnosis, involves manual microscopic examination of stained tissue by a pathologist. Because this method relies on qualitative information, it can result in inter-observer variation. Furthermore, for difficult cases the pathologist often needs additional markers of malignancy to help in making a diagnosis, a need that can potentially be met by novel microscopy methods. We present a quantitative method for label-free breast tissue evaluation using Spatial Light Interference Microscopy (SLIM). By extracting tissue markers of malignancy based on the nanostructure revealed by the optical path-length, our method provides an objective, label-free and potentially automatable method for breast histopathology. We demonstrated our method by imaging a tissue microarray consisting of 68 different subjects −34 with malignant and 34 with benign tissues. Three-fold cross validation results showed a sensitivity of 94% and specificity of 85% for detecting cancer. Our disease signatures represent intrinsic physical attributes of the sample, independent of staining quality, facilitating classification through machine learning packages since our images do not vary from scan to scan or instrument to instrument.

Adversarial Stain Transfer for Histopathology Image Analysis

It is generally recognized that color information is central to the automatic and visual analysis of histopathology tissue slides. In practice, pathologists rely on color, which reflects the presence of specific tissue components, to establish a diagnosis. Similarly, automatic histopathology image analysis algorithms rely on color or intensity measures to extract tissue features. With the increasing access to digitized histopathology images, color variation and its implications have become a critical issue. These variations are the result of not only a variety of factors involved in the preparation of tissue slides but also in the digitization process itself. Consequently, different strategies have been proposed to alleviate stain-related tissue inconsistencies in automatic image analysis systems. Such techniques generally rely on collecting color statistics to perform color matching across images. In this work, we propose a different approach for stain normalization that we refer to as stain transfer. We design a discriminative image analysis model equipped with a stain normalization component that transfers stains across datasets. Our model comprises a generative network that learns data set-specific staining properties and image-specific color transformations as well as a task-specific network (e.g., classifier or segmentation network). The model is trained end-to-end using a multi-objective cost function. We evaluate the proposed approach in the context of automatic histopathology image analysis on three data sets and two different analysis tasks: tissue segmentation and classification. The proposed method achieves superior results in terms of accuracy and quality of normalized images compared to various baselines.

Validation of Digital Microscopy Compared With Light Microscopy for the Diagnosis of Canine Cutaneous Tumors

Integration of new technologies, such as digital microscopy, into a highly standardized laboratory routine requires the validation of its performance in terms of reliability, specificity, and sensitivity. However, a validation study of digital microscopy is currently lacking in veterinary pathology. The aim of the current study was to validate the usability of digital microscopy in terms of diagnostic accuracy, speed, and confidence for diagnosing and differentiating common canine cutaneous tumor types and to compare it to classical light microscopy. Therefore, 80 histologic sections including 17 different skin tumor types were examined twice as glass slides and twice as digital whole-slide images by 6 pathologists with different levels of experience at 4 time points. Comparison of both methods found digital microscopy to be noninferior for differentiating individual tumor types within the category epithelial and mesenchymal tumors, but diagnostic concordance was slightly lower for differentiating individual round cell tumor types by digital microscopy. In addition, digital microscopy was associated with significantly shorter diagnostic time, but diagnostic confidence was lower and technical quality was considered inferior for whole-slide images compared with glass slides. Of note, diagnostic performance for whole-slide images scanned at 200× magnification was noninferior in diagnostic performance for slides scanned at 400×. In conclusion, digital microscopy differs only minimally from light microscopy in few aspects of diagnostic performance and overall appears adequate for the diagnosis of individual canine cutaneous tumors with minor limitations for differentiating individual round cell tumor types and grading of mast cell tumors.