Software‐assisted decision support in digital histopathology

Humanized monoacylglycerol acyltransferase 2 mice develop metabolic dysfunction-associated steatohepatitis

Mice lacking monoacylglycerol acyltransferase 2 (mMGAT21) are resistant to diet-induced fatty liver, suggesting hMOGAT2 inhibition is a viable option for treating metabolic dysfunction-associated steatotic liver disease (MASLD)/metabolic dysfunction-associated steatohepatitis (MASH). We generated humanized hMOGAT2 mice (HuMgat2) for use in pre-clinical studies testing the efficacy of hMOGAT2 inhibitors for treating MASLD/MASH. HuMgat2 mice developed MASH when fed a steatotic diet. Computer-aided histology revealed the presence of hepatocyte cell ballooning, immune cell infiltration, and fibrosis. Hepatocytes accumulated Mallory-Denk bodies containing phosphorylated p62/sequestosome-1-ubiquitinated protein aggregates likely caused by defects in autophagy. Metainflammation and apoptotic cell death were seen in the livers of HuMgat2 mice. Treating HuMgat2 mice with elafibranor reduced several MASH phenotypes. RNASeq analysis predicted changes in bile acid transporter expression that correlated with altered bile acid metabolism indicative of cholestasis. Our results suggest that HuMgat2 mice will serve as a pre-clinical model for testing hMOGAT2 inhibitor efficacy and toxicity and allow for the study of hMOGAT2 in the context of MASH.

Personalized cancer vaccine design using AI-powered technologies

Immunotherapy has ushered in a new era of cancer treatment, yet cancer remains a leading cause of global mortality. Among various therapeutic strategies, cancer vaccines have shown promise by activating the immune system to specifically target cancer cells. While current cancer vaccines are primarily prophylactic, advancements in targeting tumor-associated antigens (TAAs) and neoantigens have paved the way for therapeutic vaccines. The integration of artificial intelligence (AI) into cancer vaccine development is revolutionizing the field by enhancing various aspect of design and delivery. This review explores how AI facilitates precise epitope design, optimizes mRNA and DNA vaccine instructions, and enables personalized vaccine strategies by predicting patient responses. By utilizing AI technologies, researchers can navigate complex biological datasets and uncover novel therapeutic targets, thereby improving the precision and efficacy of cancer vaccines. Despite the promise of AI-powered cancer vaccines, significant challenges remain, such as tumor heterogeneity and genetic variability, which can limit the effectiveness of neoantigen prediction. Moreover, ethical and regulatory concerns surrounding data privacy and algorithmic bias must be addressed to ensure responsible AI deployment. The future of cancer vaccine development lies in the seamless integration of AI to create personalized immunotherapies that offer targeted and effective cancer treatments. This review underscores the importance of interdisciplinary collaboration and innovation in overcoming these challenges and advancing cancer vaccine development.

Lesion Localization and Pathological Diagnosis of Ovine Pulmonary Adenocarcinoma Based on MASK R-CNN

Ovine pulmonary adenocarcinoma (OPA) is a contagious lung tumour caused by the Jaagsiekte Sheep Retrovirus (JSRV). Histopathological diagnosis is the gold standard for OPA diagnosis. However, interpretation of traditional pathology images is complex and operator dependent. The mask regional convolutional neural network (Mask R-CNN) has emerged as a valuable tool in pathological diagnosis. This study utilized 54 typical OPA whole slide images (WSI) to extract 7167 typical lesion images containing OPA to construct a Common Objects in Context (COCO) dataset for OPA pathological images. The dataset was categorized into training and test sets (8:2 ratio) for model training and validation. Mean average specificity (mASp) and average sensitivity (ASe) were used to evaluate model performance. Six WSI-level pathological images (three OPA and three non-OPA images), not included in the dataset, were used for anti-peeking model validation. A random selection of 500 images, not included in the dataset establishment, was used to compare the performance of the model with assessment by pathologists. Accuracy, sensitivity, specificity, and concordance rate were evaluated. The model achieved a mASp of 0.573 and an ASe of 0.745, demonstrating effective lesion detection and alignment with expert annotation. In Anti-Peeking verification, the model showed good performance in locating OPA lesions and distinguished OPA from non-OPA pathological images. In the random 500-image diagnosis, the model achieved 92.8% accuracy, 100% sensitivity, and 88% specificity. The agreement rates between junior and senior pathologists were 100% and 96.5%, respectively. In conclusion, the Mask R-CNN-based OPA diagnostic model developed for OPA facilitates rapid and accurate diagnosis in practical applications.

Deep learning for automated scoring of immunohistochemically stained tumour tissue sections - Validation across tumour types based on patient outcomes

We aimed to develop deep learning (DL) models to detect protein expression in immunohistochemically (IHC) stained tissue-sections, and to compare their accuracy and performance with manually scored clinically relevant proteins in common cancer types. Five cancer patient cohorts (colon, two prostate, breast, and endometrial) were included. We developed separate DL models for scoring IHC-stained tissue-sections with nuclear, cytoplasmic, and membranous staining patterns. For training, we used images with annotations of cells with positive and negative staining from the colon cohort stained for Ki-67 and PMS2 (nuclear model), the prostate cohort 1 stained for PTEN (cytoplasmic model) and β-catenin (membranous model). The nuclear DL model was validated for MSH6 in the colon, MSH6 and PMS2 in the endometrium, Ki-67 and CyclinB1 in prostate, and oestrogen and progesterone receptors in the breast cancer cohorts. The cytoplasmic DL model was validated for PTEN and Mapre2, and the membranous DL model for CD44 and Flotillin1, all in prostate cohorts. When comparing the results of manual and DL scores in the validation sets, using manual scores as the ground truth, we observed an average correct classification rate of 91.5 % (76.9-98.5 %) for the nuclear model, 85.6 % (73.3-96.6 %) for the cytoplasmic model, and 78.4 % (75.5-84.3 %) for the membranous model. In survival analyses, manual and DL scores showed similar prognostic impact, with similar hazard ratios and p-values for all DL models. Our findings demonstrate that DL models offer a promising alternative to manual IHC scoring, providing efficiency and reproducibility across various data sources and markers.

Latent Diffusion Models with Image-Derived Annotations for Enhanced AI-Assisted Cancer Diagnosis in Histopathology

Artificial Intelligence (AI)-based image analysis has immense potential to support diagnostic histopathology, including cancer diagnostics. However, developing supervised AI methods requires large-scale annotated datasets. A potentially powerful solution is to augment training data with synthetic data. Latent diffusion models, which can generate high-quality, diverse synthetic images, are promising. However, the most common implementations rely on detailed textual descriptions, which are not generally available in this domain. This work proposes a method that constructs structured textual prompts from automatically extracted image features. We experiment with the PCam dataset, composed of tissue patches only loosely annotated as healthy or cancerous. We show that including image-derived features in the prompt, as opposed to only healthy and cancerous labels, improves the Fréchet Inception Distance (FID) by 88.6. We also show that pathologists find it challenging to detect synthetic images, with a median sensitivity/specificity of 0.55/0.55. Finally, we show that synthetic data effectively train AI models.

A Deep Learning-Based Approach to Estimate Paneth Cell Granule Area in Celiac Disease

CONTEXT.—

Changes in Paneth cell numbers can be associated with chronic inflammatory diseases of the gastrointestinal tract. So far, no consensus has been achieved on the number of Paneth cells and their relevance to celiac disease (CD).

OBJECTIVES.—

To compare crypt and Paneth cell granule areas between patients with CD and those without CD (non-CD) using an artificial intelligence-based solution.

DESIGN.—

Hematoxylin-eosin-stained sections of duodenal biopsies from 349 patients at the McGill University Health Centre were analyzed. Of these, 185 had a history of CD and 164 were controls. Slides were digitized, and NoCodeSeg, a code-free workflow using open-source software (QuPath, DeepMIB), was implemented to train deep learning models to segment crypts and Paneth cell granules. The total area of the entire analyzed tissue, epithelium, crypts, and Paneth cell granules was documented for all slides, and comparisons were performed.

RESULTS.—

A mean intersection-over-union score of 88.76% and 91.30% was achieved for crypt areas and Paneth cell granule segmentations, respectively. On normalization to total tissue area, the crypt to total tissue area in CD was increased and the Paneth cell granule area to total tissue area decreased when compared to non-CD controls.

CONCLUSIONS.—

Crypt hyperplasia was confirmed in CD compared to non-CD controls. The area of Paneth cell granules, an indirect measure of Paneth cell function, decreased with increasing severity of CD. More importantly, our study analyzed complete hematoxylin-eosin slide sections using an efficient and easy to use coding-free artificial intelligence workflow.

Correlation Between ImageJ and Conventional Manual Scoring Methods for Programmed Death-Ligand 1 Immuno-Histochemically Stained Sections

Background: One of the most frequently used methods for quantifying PD-L1 (programmed cell death-ligand 1) expression in tumor tissue is IHC (immunohistochemistry). This may predict the patient's response to anti-PD1/PD-L1 therapy in cancer. Methods: ImageJ software was used to score IHC-stained sections for PD-L1 and compare the results with the conventional manual method. Results: In diffuse large B cell lymphoma, no significant difference between the scores obtained by the conventional method and ImageJ scores obtained using the option "RGB" or "Brightness/Contrast." On the other hand, a significant difference was found between the conventional and HSB scoring methods. ImageJ faced some challenges in analyzing head and neck squamous cell carcinoma tissues because of tissue heterogenicity. A significant difference was found between the conventional and ImageJ scores using HSB or RGB but not with the "Brightness/Contrast" option. Scores obtained by ImageJ analysis after taking images using 20 × objective lens gave significantly higher readings compared to 40 × magnification. A significant difference between camera-captured images' scores and scanner whole slide images' scores was observed. Conclusion: ImageJ can be used to score homogeneous tissues. In the case of highly heterogeneous tissues, it is advised to use the conventional method rather than ImageJ scoring.

Personalizable AI platform for universal access to research and diagnosis in digital pathology

BACKGROUND AND MOTIVATION

Digital pathology has been evolving over the last years, proposing significant workflow advantages that have fostered its adoption in professional environments. Patient clinical and image data are readily available in remote data banks that can be consumed efficiently over standard communication technologies. The appearance of new imaging techniques and advanced artificial intelligence algorithms has significantly reduced the burden on medical professionals by speeding up the screening process. Despite these advancements, the usage of digital pathology in professional environments has been slowed down by poor interoperability between services resulting from a lack of standard interfaces and integrative solutions. This work addresses this issue by proposing a cloud-based digital pathology platform built on standard and open interfaces.

METHODS

The work proposes and describes a vendor-neutral platform that provides interfaces for managing digital slides, and medical reports, and integrating digital image analysis services compatible with existing standards. The solution integrates the open-source plugin-based Dicoogle PACS for interoperability and extensibility, which grants the proposed solution great feature customization.

RESULTS

The solution was developed in collaboration with iPATH research project partners, including the validation by medical pathologists. The result is a pure Web collaborative framework that supports both research and production environments. A total of 566 digital slides from different pathologies were successfully uploaded to the platform. Using the integration interfaces, a mitosis detection algorithm was successfully installed into the platform, and it was trained with 2400 annotations collected from breast carcinoma images.

CONCLUSION

Interoperability is a key factor when discussing digital pathology solutions, as it facilitates their integration into existing institutions' information systems. Moreover, it improves data sharing and integration of third-party services such as image analysis services, which have become relevant in today's digital pathology workflow. The proposed solution fully embraces the DICOM standard for digital pathology, presenting an interoperable cloud-based solution that provides great feature customization thanks to its extensible architecture.

Strategic Approach to Heterogeneity Analysis of Cutaneous Adnexal Carcinomas Using Computational Pathology and Genomics

Cutaneous adnexal tumors are neoplasms that arise from skin appendages. Their morphologic diversity and phenotypic variability with rare progression to malignancy make them difficult to diagnose and classify, and there is currently no established treatment strategy. To overcome these difficulties, this study investigated the transcription factor SOX9 expression, morphology, and genetics of skin adnexal tumors for understanding their biology, especially their histogenesis. We showed that cutaneous adnexal tumors and their nontumor counterparts of skin and appendages exhibit expression patterns similar to that of SOX9. Its expression intensity and pattern, as well as histopathologic evaluation of tumors, were analyzed using digital images of 69 normal skin adnexal 9-type organs and 185 skin adnexal 29-type tumors as references. It was possible to distinguish basal cell carcinoma from squamous cell carcinoma, sebaceous carcinoma, and pilomatrixoma with significant differences, along with porocarcinoma from squamous cell carcinoma. Furthermore, unsupervised machine learning "computational pathology" was used to derive a multiregion whole-exome sequencing fusion method termed "genocomputed pathology." The genocomputed pathology of three representable adnexal carcinomas (porocarcinoma, hidradenocarcinoma, and spiradenocarcinoma) was evaluated for total nine cases. We showed that there was more heterogeneity than expected within the tumors as well as the coexistence of components lacking driver fusion genes. The presence or absence of potential driver genes, such as PIK3CA, YAP1, and PTEN, in each region was identified, highlighting a therapeutic strategy for cutaneous adnexal carcinoma encompassing heterogeneous tumors.

Application of multiple-finding segmentation utilizing Mask R-CNN-based deep learning in a rat model of drug-induced liver injury

Drug-induced liver injury (DILI) presents significant diagnostic challenges, and recently artificial intelligence-based deep learning technology has been used to predict various hepatic findings. In this study, we trained a set of Mask R-CNN-based deep algorithms to learn and quantify typical toxicant induced-histopathological lesions, portal area, and connective tissue in Sprague Dawley rats. We compared a set of single-finding models (SFMs) and a combined multiple-finding model (MFM) for their ability to simultaneously detect, classify, and quantify multiple hepatic findings on rat liver slide images. All of the SFMs yielded mean average precision (mAP) values above 85%, suggesting that the models had been successfully established. The MFM showed better performance than the SFMs, with a total mAP value of 92.46%. We compared the model predictions for slide images with ground-truth annotations generated by an accredited pathologist. For the MFM, the overall and individual finding predictions were highly correlated with the annotated areas, with R-squared values of 0.852, 0.952, 0.999, 0.990, and 0.958 being obtained for portal area, infiltration, necrosis, vacuolation, and connective tissue (including fibrosis), respectively. Our results indicate that the proposed MFM could be a useful tool for detecting and predicting multiple hepatic findings in basic non-clinical study settings.

Building a nonclinical pathology laboratory of the future for pharmaceutical research excellence

We describe a roadmap for a fully digital artificial intelligence (AI)-augmented nonclinical pathology laboratory across three continents. Underpinning the design are Good Laboratory Practice (GLP)-validated laboratory information management systems (LIMS), whole slide-scanners (WSS), image management systems (IMS), and a digital microscope intended for use by the nonclinical pathologist. Digital diagnostics are supported by tools that include AI-based virtual staining and deep learning-based decision support. Implemented during the COVID-19 pandemic, the initial digitized workflow largely mitigated disruption of pivotal nonclinical studies required to support pharmaceutical clinical testing. We believe that this digital transformation of our nonclinical pathology laboratories will promote efficiency and innovation in the future and enhance the quality and speed of drug development decision making.

Pathologists' first opinions on barriers and facilitators of computational pathology adoption in oncological pathology: an international study

Computational pathology (CPath) algorithms detect, segment or classify cancer in whole slide images, approaching or even exceeding the accuracy of pathologists. Challenges have to be overcome before these algorithms can be used in practice. We therefore aim to explore international perspectives on the future role of CPath in oncological pathology by focusing on opinions and first experiences regarding barriers and facilitators. We conducted an international explorative eSurvey and semi-structured interviews with pathologists utilizing an implementation framework to classify potential influencing factors. The eSurvey results showed remarkable variation in opinions regarding attitude, understandability and validation of CPath. Interview results showed that barriers focused on the quality of available evidence, while most facilitators concerned strengths of CPath. A lack of consensus was present for multiple factors, such as the determination of sufficient validation using CPath, the preferred function of CPath within the digital workflow and the timing of CPath introduction in pathology education. The diversity in opinions illustrates variety in influencing factors in CPath adoption. A next step would be to quantitatively determine important factors for adoption and initiate validation studies. Both should include clear case descriptions and be conducted among a more homogenous panel of pathologists based on sub specialization.

Seven-colour multiplex immunochemistry/immunofluorescence and whole slide imaging of frozen sections

Multiplex Immunochemistry/Immunofluorescence (mIHC/IF) aims to visualise multiple biomarkers in a single tissue section and is especially powerful when used on slide scanners coupled with digital analysis tools. mIHC/IF is commonly employed in immuno-oncology to characterise features of the tumour microenvironment (TME) and correlate them with clinical parameters to guide prognostication and therapy. However, mIHC/IF can be applied to a wide range of organisms in any physiological or disease context. Recent innovation has extended the number of markers that can be detected using slide scanners well beyond the 3-4 markers typically reported in traditional fluorescence microscopy. However, these methods often require sequential antibody staining and stripping, and are not compatible with frozen tissue sections. Using fluorophore-conjugated antibodies, we have established a simple mIHC/IF imaging workflow that enables simultaneous staining and detection of seven markers in a single section of frozen tissue. Coupled with automated whole slide imaging and digital quantification, our data efficiently revealed the tumour-immune complexity in metastatic melanoma. Computational image analysis quantified the immune and stromal cell populations present in the TME as well as their spatial interactions. This imaging workflow can also be performed with an indirect labelling panel consisting of primary and secondary antibodies. Our new methods, combined with digital quantification, will provide a valuable tool for high-quality mIHC/IF assays in immuno-oncology research and other translational studies, especially in circumstances where frozen sections are required for detection of particular markers, or for applications where frozen sections may be preferred, such as spatial transcriptomics.

Artificial intelligence and digital biomarker in precision pathology guiding immune therapy selection and precision oncology

BACKGROUND

The currently available immunotherapies already changed the strategy how many cancers are treated from first to last line. Understanding even the most complex heterogeneity in tumor tissue and mapping the spatial cartography of the tumor immunity allows the best and optimized selection of immune modulating agents to (re-)activate the patient's immune system and direct it against the individual cancer in the most effective way.

RECENT FINDINGS

Primary cancer and metastases maintain a high degree of plasticity to escape any immune surveillance and continue to evolve depending on many intrinsic and extrinsic factors In the field of immune-oncology (IO) immune modulating agents are recognized as practice changing therapeutic modalities. Recent studies have shown that an optimal and lasting efficacy of IO therapeutics depends on the understanding of the spatial communication network and functional context of immune and cancer cells within the tumor microenvironment. Artificial intelligence (AI) provides an insight into the immune-cancer-network through the visualization of very complex tumor and immune interactions in cancer tissue specimens and allows the computer-assisted development and clinical validation of such digital biomarker.

CONCLUSIONS

The successful implementation of AI-supported digital biomarker solutions guides the clinical selection of effective immune therapeutics based on the retrieval and visualization of spatial and contextual information from cancer tissue images and standardized data. As such, computational pathology (CP) turns into "precision pathology" delivering individual therapy response prediction. Precision Pathology does not only include digital and computational solutions but also high levels of standardized processes in the routine histopathology workflow and the use of mathematical tools to support clinical and diagnostic decisions as the basic principle of a "precision oncology".

A Deep Neural Network-Based Decision Support Tool for the Detection of Lymph Node Metastases in Colorectal Cancer Specimens

The identification of lymph node metastases in colorectal cancer (CRC) specimens is crucial for the planning of postoperative treatment and can be a time-consuming task for pathologists. In this study, we developed a deep neural network (DNN) algorithm for the detection of metastatic CRC in digitized histologic sections of lymph nodes and evaluated its performance as a diagnostic support tool. First, the DNN algorithm was trained using pixel-level annotations of cancerous areas on 758 whole slide images (360 with cancerous areas). The algorithm's performance was evaluated on 74 whole slide images (43 with cancerous areas). Second, the algorithm was evaluated as a decision support tool on 288 whole slide images covering 1517 lymph node sections, randomized in 16 batches. Two senior pathologists (C.K. and C.O.) assessed each batch with and without the help of the algorithm in a 2 × 2 crossover design, with a washout period of 1 month in between. The time needed for the evaluation of each node section was recorded. The DNN algorithm achieved a median pixel-level accuracy of 0.952 on slides with cancerous areas and 0.996 on slides with benign samples. N+ disease (metastases, micrometastases, or tumor deposits) was present in 103 of the 1517 sections. The algorithm highlighted cancerous areas in 102 of these sections, with a sensitivity of 0.990. Assisted by the algorithm, the median time needed for evaluation was significantly shortened for both pathologists (median time for pathologist 1, 26 vs 14 seconds; P < .001; 95% CI, 11.0-12.0; median time for pathologist 2, 25 vs 23 seconds; P < .001; 95% CI, 2.0-4.0). Our DNN showed high accuracy for detecting metastatic CRC in digitized histologic sections of lymph nodes. This decision support tool has the potential to improve the diagnostic workflow by shortening the time needed for the evaluation of lymph nodes in CRC specimens without impairing diagnostic accuracy.

Computationally efficient adaptive decompression for whole slide image processing

Whole slide image (WSI) analysis is increasingly being adopted as an important tool in modern pathology. Recent deep learning-based methods have achieved state-of-the-art performance on WSI analysis tasks such as WSI classification, segmentation, and retrieval. However, WSI analysis requires a significant amount of computation resources and computation time due to the large dimensions of WSIs. Most of the existing analysis approaches require the complete decompression of the whole image exhaustively, which limits the practical usage of these methods, especially for deep learning-based workflows. In this paper, we present compression domain processing-based computation efficient analysis workflows for WSIs classification that can be applied to state-of-the-art WSI classification models. The approaches leverage the pyramidal magnification structure of WSI files and compression domain features that are available from the raw code stream. The methods assign different decompression depths to the patches of WSIs based on the features directly retained from compressed patches or partially decompressed patches. Patches from the low-magnification level are screened by attention-based clustering, resulting in different decompression depths assigned to the high-magnification level patches at different locations. A finer-grained selection based on compression domain features from the file code stream is applied to select further a subset of the high-magnification patches that undergo a full decompression. The resulting patches are fed to the downstream attention network for final classification. Computation efficiency is achieved by reducing unnecessary access to the high zoom level and expensive full decompression. With the number of decompressed patches reduced, the time and memory costs of downstream training and inference procedures are also significantly reduced. Our approach achieves a 7.2× overall speedup, and the memory cost is reduced by 1.1 orders of magnitudes, while the resulting model accuracy is comparable to the original workflow.

Improving quality control in the routine practice for histopathological interpretation of gastrointestinal endoscopic biopsies using artificial intelligence

BACKGROUND

Colorectal and gastric cancer are major causes of cancer-related deaths. In Korea, gastrointestinal (GI) endoscopic biopsy specimens account for a high percentage of histopathologic examinations. Lack of a sufficient pathologist workforce can cause an increase in human errors, threatening patient safety. Therefore, we developed a digital pathology total solution combining artificial intelligence (AI) classifier models and pathology laboratory information system for GI endoscopic biopsy specimens to establish a post-analytic daily fast quality control (QC) system, which was applied in clinical practice for a 3-month trial run by four pathologists.

METHODS AND FINDINGS

Our whole slide image (WSI) classification framework comprised patch-generator, patch-level classifier, and WSI-level classifier. The classifiers were both based on DenseNet (Dense Convolutional Network). In laboratory tests, the WSI classifier achieved accuracy rates of 95.8% and 96.0% in classifying histopathological WSIs of colorectal and gastric endoscopic biopsy specimens, respectively, into three classes (Negative for dysplasia, Dysplasia, and Malignant). Classification by pathologic diagnosis and AI prediction were compared and daily reviews were conducted, focusing on discordant cases for early detection of potential human errors by the pathologists, allowing immediate correction, before the pathology report error is conveyed to the patients. During the 3-month AI-assisted daily QC trial run period, approximately 7-10 times the number of slides compared to that in the conventional monthly QC (33 months) were reviewed by pathologists; nearly 100% of GI endoscopy biopsy slides were double-checked by the AI models. Further, approximately 17-30 times the number of potential human errors were detected within an average of 1.2 days.

CONCLUSIONS

The AI-assisted daily QC system that we developed and established demonstrated notable improvements in QC, in quantitative, qualitative, and time utility aspects. Ultimately, we developed an independent AI-assisted post-analytic daily fast QC system that was clinically applicable and influential, which could enhance patient safety.

Differentiation of pancreatic ductal adenocarcinoma and chronic pancreatitis using graph neural networks on histopathology and collagen fiber features

Pancreatic ductal adenocarcinoma (PDAC) is one of the most lethal human cancers. However, the symptoms and radiographic appearance of chronic pancreatitis (CP) mimics that of PDAC, and sometimes the 2 entities can also be difficult to differentiate microscopically. The need for accurate differentiation of PDAC and CP has become a major topic in pancreatic pathology. These 2 diseases can present similar histomorphological features, such as excessive deposition of fibrotic stroma in the tissue microenvironment and inflammatory cell infiltration. In this paper, we present a quantitative analysis pipeline empowered by graph neural networks (GNN) capable of automatic detection and differentiation of PDAC and CP in human histological specimens. Modeling histological images as graphs and deploying graph convolutions can enable the capture of histomorphological features at different scales, ranging from nuclear size to the organization of ducts. The analysis pipeline combines image features computed from co-registered hematoxylin and eosin (H&E) images and Second-Harmonic Generation (SHG) microscopy images, with the SHG images enabling the extraction of collagen fiber morphological features. Evaluating the analysis pipeline on a human tissue micro-array dataset consisting of 786 cores and a tissue region dataset consisting of 268 images, it attained 86.4% accuracy with an average area under the curve (AUC) of 0.954 and 88.9% accuracy with an average AUC of 0.957, respectively. Moreover, incorporating topological features of collagen fibers computed from SHG images into the model further increases the classification accuracy on the tissue region dataset to 91.3% with an average AUC of 0.962, suggesting that collagen characteristics are diagnostic features in PDAC and CP detection and differentiation.

Application of Artificial Intelligence in Pathology: Trends and Challenges

Given the recent success of artificial intelligence (AI) in computer vision applications, many pathologists anticipate that AI will be able to assist them in a variety of digital pathology tasks. Simultaneously, tremendous advancements in deep learning have enabled a synergy with artificial intelligence (AI), allowing for image-based diagnosis on the background of digital pathology. There are efforts for developing AI-based tools to save pathologists time and eliminate errors. Here, we describe the elements in the development of computational pathology (CPATH), its applicability to AI development, and the challenges it faces, such as algorithm validation and interpretability, computing systems, reimbursement, ethics, and regulations. Furthermore, we present an overview of novel AI-based approaches that could be integrated into pathology laboratory workflows.

A deep learning model for molecular label transfer that enables cancer cell identification from histopathology images

Deep-learning classification systems have the potential to improve cancer diagnosis. However, development of these computational approaches so far depends on prior pathological annotations and large training datasets. The manual annotation is low-resolution, time-consuming, highly variable and subject to observer variance. To address this issue, we developed a method, H&E Molecular neural network (HEMnet). HEMnet utilizes immunohistochemistry as an initial molecular label for cancer cells on a H&E image and trains a cancer classifier on the overlapping clinical histopathological images. Using this molecular transfer method, HEMnet successfully generated and labeled 21,939 tumor and 8782 normal tiles from ten whole-slide images for model training. After building the model, HEMnet accurately identified colorectal cancer regions, which achieved 0.84 and 0.73 of ROC AUC values compared to p53 staining and pathological annotations, respectively. Our validation study using histopathology images from TCGA samples accurately estimated tumor purity, which showed a significant correlation (regression coefficient of 0.8) with the estimation based on genomic sequencing data. Thus, HEMnet contributes to addressing two main challenges in cancer deep-learning analysis, namely the need to have a large number of images for training and the dependence on manual labeling by a pathologist. HEMnet also predicts cancer cells at a much higher resolution compared to manual histopathologic evaluation. Overall, our method provides a path towards a fully automated delineation of any type of tumor so long as there is a cancer-oriented molecular stain available for subsequent learning. Software, tutorials and interactive tools are available at: https://github.com/BiomedicalMachineLearning/HEMnet.

Understanding the ethical and legal considerations of Digital Pathology

Digital Pathology (DP) is a platform which has the potential to develop a truly integrated and global pathology community. The generation of DP data at scale creates novel challenges for the histopathology community in managing, processing, and governing the use of these data. The current understanding of, and confidence in, the legal and ethical aspects of DP by pathologists is unknown. We developed an electronic survey (e-survey), comprising 22 questions, with input from the Royal College of Pathologists (RCPath) Digital Pathology Working Group. The e-survey was circulated via e-mail and social media (Twitter) through the RCPath Digital Pathology Working Group network, RCPath Trainee Committee network, the Pathology image data Lake for Analytics, Knowledge and Education (PathLAKE) digital pathology consortium, National Pathology Imaging Co-operative (NPIC), local contacts, and to the membership of both The Pathological Society of Great Britain and Ireland and the British Division of the International Academy of Pathology (BDIAP). Between 14 July 2020 and 6 September 2020, we collected 198 responses representing a cross section of histopathologists, including individuals with experience of DP research. We ascertained that, in the UK, DP is being used for diagnosis, research, and teaching, and that the platform is enabling data sharing. Our survey demonstrated that there is often a lack of confidence and understanding of the key issues of consent, legislation, and ethical guidelines. Of 198 respondents, 82 (41%) did not know when the use of digital scanned slide images would fall under the relevant legislation and 93 (47%) were 'Not confident at all' in their interpretation of consent for scanned slide images in research. With increasing uptake of DP, a working knowledge of these areas is essential but histopathologists often express a lack of confidence in these topics. The need for specific training in these areas is highlighted by the findings of this study.

Deep learning in cancer diagnosis, prognosis and treatment selection

Deep learning is a subdiscipline of artificial intelligence that uses a machine learning technique called artificial neural networks to extract patterns and make predictions from large data sets. The increasing adoption of deep learning across healthcare domains together with the availability of highly characterised cancer datasets has accelerated research into the utility of deep learning in the analysis of the complex biology of cancer. While early results are promising, this is a rapidly evolving field with new knowledge emerging in both cancer biology and deep learning. In this review, we provide an overview of emerging deep learning techniques and how they are being applied to oncology. We focus on the deep learning applications for omics data types, including genomic, methylation and transcriptomic data, as well as histopathology-based genomic inference, and provide perspectives on how the different data types can be integrated to develop decision support tools. We provide specific examples of how deep learning may be applied in cancer diagnosis, prognosis and treatment management. We also assess the current limitations and challenges for the application of deep learning in precision oncology, including the lack of phenotypically rich data and the need for more explainable deep learning models. Finally, we conclude with a discussion of how current obstacles can be overcome to enable future clinical utilisation of deep learning.

Whole-slide imaging, tissue image analysis, and artificial intelligence in veterinary pathology: An updated introduction and review

Since whole-slide imaging has been commercially available for over 2 decades, digital pathology has become a constantly expanding aspect of the pathology profession that will continue to significantly impact how pathologists conduct their craft. While some aspects, such as whole-slide imaging for archiving, consulting, and teaching, have gained broader acceptance, other facets such as quantitative tissue image analysis and artificial intelligence-based assessments are still met with some reservations. While most vendors in this space have focused on diagnostic applications, that is, viewing one or few slides at a time, some are developing solutions tailored more specifically to the various aspects of veterinary pathology including updated diagnostic, discovery, and research applications. This has especially advanced the use of digital pathology in toxicologic pathology and drug development, for primary reads as well as peer reviews. It is crucial that pathologists gain a deeper understanding of digital pathology and tissue image analysis technology and their applications in order to fully use these tools in a way that enhances and improves the pathologist's assessment as well as work environment. This review focuses on an updated introduction to the basics of digital pathology and image analysis and introduces emerging topics around artificial intelligence and machine learning.

[Standardized diagnosis of pancreatic head carcinoma]

Most pancreatic ductal adenocarcinomas are localized in the pancreatic head. Due to the complex anatomic relationships with the surrounding organs and vascular structures in the retroperitoneal space and to the presence of numerous transection margins and dissection planes, pancreatic head resections belong to the most complex specimens concerning grossing and sampling for histopathologic analysis.Here we discuss current guidelines for standardized grossing and reporting of pancreatic cancer, with special reference to the assessment of the resection margin status. The importance of standardized reporting for the sake of completeness, comprehensibility, comparability, and quality control as well as for the integration of pathology reports in interdisciplinary digital workflows and artificial intelligence applications will be emphasized.

Tissue Multiplex Analyte Detection in Anatomic Pathology - Pathways to Clinical Implementation

Background: Multiplex tissue analysis has revolutionized our understanding of the tumor microenvironment (TME) with implications for biomarker development and diagnostic testing. Multiplex labeling is used for specific clinical situations, but there remain barriers to expanded use in anatomic pathology practice.

Methods: We review immunohistochemistry (IHC) and related assays used to localize molecules in tissues, with reference to United States regulatory and practice landscapes. We review multiplex methods and strategies used in clinical diagnosis and in research, particularly in immuno-oncology. Within the framework of assay design and testing phases, we examine the suitability of multiplex immunofluorescence (mIF) for clinical diagnostic workflows, considering its advantages and challenges to implementation.

Results: Multiplex labeling is poised to radically transform pathologic diagnosis because it can answer questions about tissue-level biology and single-cell phenotypes that cannot be addressed with traditional IHC biomarker panels. Widespread implementation will require improved detection chemistry, illustrated by InSituPlex technology (Ultivue, Inc., Cambridge, MA) that allows coregistration of hematoxylin and eosin (H&E) and mIF images, greater standardization and interoperability of workflow and data pipelines to facilitate consistent interpretation by pathologists, and integration of multichannel images into digital pathology whole slide imaging (WSI) systems, including interpretation aided by artificial intelligence (AI). Adoption will also be facilitated by evidence that justifies incorporation into clinical practice, an ability to navigate regulatory pathways, and adequate health care budgets and reimbursement. We expand the brightfield WSI system “pixel pathway” concept to multiplex workflows, suggesting that adoption might be accelerated by data standardization centered on cell phenotypes defined by coexpression of multiple molecules.

Conclusion: Multiplex labeling has the potential to complement next generation sequencing in cancer diagnosis by allowing pathologists to visualize and understand every cell in a tissue biopsy slide. Until mIF reagents, digital pathology systems including fluorescence scanners, and data pipelines are standardized, we propose that diagnostic labs will play a crucial role in driving adoption of multiplex tissue diagnostics by using retrospective data from tissue collections as a foundation for laboratory-developed test (LDT) implementation and use in prospective trials as companion diagnostics (CDx).

Novel approach for quantification of multiple immunofluorescent signals using histograms and 2D plot profiling of whole-section panoramic images

We describe a novel approach for quantification and colocalization of immunofluorescence (IF) signals of multiple markers on high-resolution panoramic images of serial histological sections utilizing standard staining techniques and readily available software for image processing and analysis. Human gingiva samples stained with primary antibodies against the common leukocyte antigen CD45 and factors related to heparan sulfate glycosaminoglycans (HS GAG) were used. Expression domains and spatial gradients of IF signals were quantified by histograms and 2D plot profiles, respectively. The importance of histomorphometric profiling of tissue samples and IF signal thresholding is elaborated. This approach to quantification of IF staining utilizes pixel (px) counts and comparison of px grey value (GV) or luminance. No cell counting is applied either to determine the cellular content of a given histological section nor the number of cells positive to the primary antibody of interest. There is no selection of multiple Regions-Of-Interest (ROIs) since the entire histological section is quantified. Although the standard IF staining protocol is applied, the data output enables colocalization of multiple markers (up to 30) from a given histological sample. This can serve as an alternative for colocalization of IF staining of multiple primary antibodies based on repeating cycles of staining of the same histological section since those techniques require non standard staining protocols and sophisticated equipment that can be out of reach for small laboratories in academic settings. Combined with the data from ontological bases, this approach to quantification of IF enables creation of in silico virtual disease models.

Conventional and semi-automatic histopathological analysis of tumor cell content for multigene sequencing of lung adenocarcinoma

Background

Targeted genetic profiling of tissue samples is paramount to detect druggable genetic aberrations in patients with non-squamous non-small cell lung cancer (NSCLC). Accurate upfront estimation of tumor cell content (TCC) is a crucial pre-analytical step for reliable testing and to avoid false-negative results. As of now, TCC is usually estimated on hematoxylin-eosin (H&E) stained tissue sections by a pathologist, a methodology that may be prone to substantial intra- and interobserver variability. Here we the investigate suitability of digital pathology for TCC estimation in a clinical setting by evaluating the concordance between semi-automatic and conventional TCC quantification.

Methods

TCC was analyzed in 120 H&E and thyroid transcription factor 1 (TTF-1) stained high-resolution images by 19 participants with different levels of pathological expertise as well as by applying two semi-automatic digital pathology image analysis tools (HALO and QuPath).

Results

Agreement of TCC estimations [intra-class correlation coefficients (ICC)] between the two software tools (H&E: 0.87; TTF-1: 0.93) was higher compared to that between conventional observers (0.48; 0.47). Digital TCC estimations were in good agreement with the average of human TCC estimations (0.78; 0.96). Conventional TCC estimators tended to overestimate TCC, especially in H&E stainings, in tumors with solid patterns and in tumors with an actual TCC close to 50%.

Conclusions

Our results determine factors that influence TCC estimation. Computer-assisted analysis can improve the accuracy of TCC estimates prior to molecular diagnostic workflows. In addition, we provide a free web application to support self-training and quality improvement initiatives at other institutions.

Artificial intelligence and computational pathology

Data processing and learning has become a spearhead for the advancement of medicine, with pathology and laboratory medicine has no exception. The incorporation of scientific research through clinical informatics, including genomics, proteomics, bioinformatics, and biostatistics, into clinical practice unlocks innovative approaches for patient care. Computational pathology is burgeoning subspecialty in pathology that promises a better-integrated solution to whole-slide images, multi-omics data, and clinical informatics. However, computational pathology faces several challenges, including the ability to integrate raw data from different sources, limitation of hardware processing capacity, and a lack of specific training programs, as well as issues on ethics and larger societal acceptable practices that are still solidifying. The establishment of the entire industry of computational pathology requires far-reaching changes of the three essential elements connecting patients and doctors: the local laboratory, the scan center, and the central cloud hub/portal for data processing and retrieval. Computational pathology, unlocked through information integration and advanced digital communication networks, has the potential to improve clinical workflow efficiency, diagnostic quality, and ultimately create personalized diagnosis and treatment plans for patients. This review describes clinical perspectives and discusses the statistical methods, clinical applications, potential obstacles, and future directions of computational pathology.

Artificial intelligence and digital pathology: Opportunities and implications for immuno-oncology

The field of immuno-oncology has expanded rapidly over the past decade, but key questions remain. How does tumour-immune interaction regulate disease progression? How can we prospectively identify patients who will benefit from immunotherapy? Identifying measurable features of the tumour immune-microenvironment which have prognostic or predictive value will be key to making meaningful gains in these areas. Recent developments in deep learning enable big-data analysis of pathological samples. Digital approaches allow data to be acquired, integrated and analysed far beyond what is possible with conventional techniques, and to do so efficiently and at scale. This has the potential to reshape what can be achieved in terms of volume, precision and reliability of output, enabling data for large cohorts to be summarised and compared. This review examines applications of artificial intelligence (AI) to important questions in immuno-oncology (IO). We discuss general considerations that need to be taken into account before AI can be applied in any clinical setting. We describe AI methods that have been applied to the field of IO to date and present several examples of their use.

"Omics" in traumatic brain injury: novel approaches to a complex disease

BACKGROUND

To date, there is neither any pharmacological treatment with efficacy in traumatic brain injury (TBI) nor any method to halt the disease progress. This is due to an incomplete understanding of the vast complexity of the biological cascades and failure to appreciate the diversity of secondary injury mechanisms in TBI. In recent years, techniques for high-throughput characterization and quantification of biological molecules that include genomics, proteomics, and metabolomics have evolved and referred to as omics.

METHODS

In this narrative review, we highlight how omics technology can be applied to potentiate diagnostics and prognostication as well as to advance our understanding of injury mechanisms in TBI.

RESULTS

The omics platforms provide possibilities to study function, dynamics, and alterations of molecular pathways of normal and TBI disease states. Through advanced bioinformatics, large datasets of molecular information from small biological samples can be analyzed in detail and provide valuable knowledge of pathophysiological mechanisms, to include in prognostic modeling when connected to clinically relevant data. In such a complex disease as TBI, omics enables broad categories of studies from gene compositions associated with susceptibility to secondary injury or poor outcome, to potential alterations in metabolites following TBI.

CONCLUSION

The field of omics in TBI research is rapidly evolving. The recent data and novel methods reviewed herein may form the basis for improved precision medicine approaches, development of pharmacological approaches, and individualization of therapeutic efforts by implementing mathematical "big data" predictive modeling in the near future.

Challenges in the Development, Deployment, and Regulation of Artificial Intelligence in Anatomic Pathology

Significant advances in artificial intelligence (AI), deep learning, and other machine-learning approaches have been made in recent years, with applications found in almost every industry, including health care. AI has proved to be capable of completing a spectrum of mundane to complex medically oriented tasks previously performed only by boarded physicians, most recently assisting with the detection of cancers difficult to find on histopathology slides. Although computers will not replace pathologists any time soon, properly designed AI-based tools hold great potential for increasing workflow efficiency and diagnostic accuracy in the practice of pathology. Recent trends, such as data augmentation, crowdsourcing for generating annotated data sets, and unsupervised learning with molecular and/or clinical outcomes versus human diagnoses as a source of ground truth, are eliminating the direct role of pathologists in algorithm development. Proper integration of AI-based systems into anatomic-pathology practice will necessarily require fully digital imaging platforms, an overhaul of legacy information-technology infrastructures, modification of laboratory/pathologist workflows, appropriate reimbursement/cost-offsetting models, and ultimately, the active participation of pathologists to encourage buy-in and oversight. Regulations tailored to the nature and limitations of AI are currently in development and, when instituted, are expected to promote safe and effective use. This review addresses the challenges in AI development, deployment, and regulation to be overcome prior to its widespread adoption in anatomic pathology.

Recent Advances in Pathology: the 2020 Annual Review Issue of The Journal of Pathology

This year's Annual Review Issue of The Journal of Pathology contains 18 invited reviews on current research areas in pathology. The subject areas reflect the broad range of topics covered by the journal and this year encompass the development and application of software in digital histopathology, implementation of biomarkers in pathology practice; genetics and epigenetics, and stromal influences in disease. The reviews are authored by experts in their field and provide comprehensive updates in the chosen areas, in which there has been considerable recent progress in our understanding of disease. © 2020 Pathological Society of Great Britain and Ireland. Published by John Wiley & Sons, Ltd.

Piloting a Deep Learning Model for Predicting Nuclear BAP1 Immunohistochemical Expression of Uveal Melanoma from Hematoxylin-and-Eosin Sections

Background

Uveal melanoma (UM) is the most common primary intraocular malignancy in adults. Monosomy 3 and BAP1 mutation are strong prognostic factors predicting metastatic risk in UM. Nuclear BAP1 (nBAP1) expression is a close immunohistochemical surrogate for both genetic alterations. Not all laboratories perform routine BAP1 immunohistochemistry or genetic testing, and rely mainly on clinical information and anatomic/morphologic analyses for UM prognostication. The purpose of our study was to pilot deep learning (DL) techniques to predict nBAP1 expression on whole slide images (WSIs) of hematoxylin and eosin (H&E) stained UM sections.

Methods

One hundred forty H&E-stained UMs were scanned at 40 × magnification, using commercially available WSI image scanners. The training cohort comprised 66 BAP1⁺ and 74 BAP1⁻ UM, with known chromosome 3 status and clinical outcomes. Nonoverlapping areas of three different dimensions (512 × 512, 1024 × 1024, and 2048 × 2048 pixels) for comparison were extracted from tumor regions in each WSI, and were resized to 256 × 256 pixels. Deep convolutional neural networks (Resnet18 pre-trained on Imagenet) and auto-encoder-decoders (U-Net) were trained to predict nBAP1 expression of these patches. Trained models were tested on the patches cropped from a test cohort of WSIs of 16 BAP1⁺ and 28 BAP1⁻ UM cases.

Results

The trained model with best performance achieved area under the curve values of 0.90 for patches and 0.93 for slides on the test set.

Conclusions

Our results show the effectiveness of DL for predicting nBAP1 expression in UM on the basis of H&E sections only.

Translational Relevance

Our pilot demonstrates a high capacity of artificial intelligence-related techniques for automated prediction on the basis of histomorphology, and may be translatable into routine histology laboratories.

Role of digital pathology in diagnostic histopathology in the response to COVID-19: results from a survey of experience in a UK tertiary referral hospital

The COVID-19 pandemic has challenged our diagnostic services at a time when many histopathology departments already faced a diminishing workforce and increasing workload. Digital pathology (DP) has been hailed as a potential solution to at least some of the challenges faced. We present a survey of pathologists within a UK National Health Service cellular pathology department with access to DP, in which we ascertain the role of DP in clinical services during this current pandemic and explore challenges encountered. This survey indicates an increase in uptake of diagnostic DP during this period, with increased remote access. Half of respondents agreed that DP had facilitated maintenance of diagnostic practice. While challenges have been encountered, these are remediable, and none have impacted on the uptake of DP during this period. We conclude that in our institution, DP has demonstrated current and future potential to increase resilience in diagnostic practice and have highlighted some of the challenges that need to be considered.