Artificial intelligence and computational pathology

Cellular senescence in cancer: clinical detection and prognostic implications

Cellular senescence is a state of stable cell-cycle arrest with secretory features in response to cellular stress. Historically, it has been considered as an endogenous evolutionary homeostatic mechanism to eliminate damaged cells, including damaged cells which are at risk of malignant transformation, thereby protecting against cancer. However, accumulation of senescent cells can cause long-term detrimental effects, mainly through the senescence-associated secretory phenotype, and paradoxically contribute to age-related diseases including cancer. Besides its role as tumor suppressor, cellular senescence is increasingly being recognized as an in vivo response in cancer patients to various anticancer therapies. Its role in cancer is ambiguous and even controversial, and senescence has recently been promoted as an emerging hallmark of cancer because of its hallmark-promoting capabilities. In addition, the prognostic implications of cellular senescence have been underappreciated due to the challenging detection and sparse in and ex vivo evidence of cellular senescence in cancer patients, which is only now catching up. In this review, we highlight the approaches and current challenges of in and ex vivo detection of cellular senescence in cancer patients, and we discuss the prognostic implications of cellular senescence based on in and ex vivo evidence in cancer patients.

Accurate diagnostic tissue segmentation and concurrent disease subtyping with small datasets

Purpose

To provide a flexible, end-to-end platform for visually distinguishing diseased from undiseased tissue in a medical image, in particular pathology slides, and classifying diseased regions by subtype. Highly accurate results are obtained using small training datasets and reduced-scale source images that can be easily shared.

Approach

An ensemble of lightweight convolutional neural networks (CNNs) is trained on different subsets of images derived from a relatively small number of annotated whole-slide histopathology images (WSIs). The WSIs are first reduced in scale in a manner that preserves anatomic features critical to analysis while also facilitating convenient handling and storage. The segmentation and subtyping tasks are performed sequentially on the reduced-scale images using the same basic workflow: generating and sifting tiles from the image, then classifying each tile with an ensemble of appropriately trained CNNs. For segmentation, the CNN predictions are combined using a function to favor a selected similarity metric, and a mask or map for a a candidate image is produced from tiles whose combined predictions exceed a decision boundary. For subtyping, the resulting mask is applied to the candidate image, and new tiles are derived from the unoccluded regions. These are classified by the subtyping CNNs to produce an overall subtype prediction.

Results and conclusion

This approach was applied successfully to two very different datasets of large WSIs, one (PAIP2020) involving multiple subtypes of colorectal cancer and the other (CAMELYON16) single-type breast cancer metastases. Scored using standard similarity metrics, the segmentations outperformed more complex models typifying the state of the art.

Breast Cancer Dataset, Classification and Detection Using Deep Learning

Incorporating scientific research into clinical practice via clinical informatics, which includes genomics, proteomics, bioinformatics, and biostatistics, improves patients' treatment. Computational pathology is a growing subspecialty with the potential to integrate whole slide images, multi-omics data, and health informatics. Pathology and laboratory medicine are critical to diagnosing cancer. This work will review existing computational and digital pathology methods for breast cancer diagnosis with a special focus on deep learning. The paper starts by reviewing public datasets related to breast cancer diagnosis. Additionally, existing deep learning methods for breast cancer diagnosis are reviewed. The publicly available code repositories are introduced as well. The paper is closed by highlighting challenges and future works for deep learning-based diagnosis.

Computational Pipeline for Glomerular Segmentation and Association of the Quantified Regions with Prognosis of Kidney Function in IgA Nephropathy

The histopathological findings of the glomeruli from whole slide images (WSIs) of a renal biopsy play an important role in diagnosing and grading kidney disease. This study aimed to develop an automated computational pipeline to detect glomeruli and to segment the histopathological regions inside of the glomerulus in a WSI. In order to assess the significance of this pipeline, we conducted a multivariate regression analysis to determine whether the quantified regions were associated with the prognosis of kidney function in 46 cases of immunoglobulin A nephropathy (IgAN). The developed pipelines showed a mean intersection over union (IoU) of 0.670 and 0.693 for five classes (i.e., background, Bowman's space, glomerular tuft, crescentic, and sclerotic regions) against the WSI of its facility, and 0.678 and 0.609 against the WSI of the external facility. The multivariate analysis revealed that the predicted sclerotic regions, even those that were predicted by the external model, had a significant negative impact on the slope of the estimated glomerular filtration rate after biopsy. This is the first study to demonstrate that the quantified sclerotic regions that are predicted by an automated computational pipeline for the segmentation of the histopathological glomerular components on WSIs impact the prognosis of kidney function in patients with IgAN.

Computer-Assisted Annotation of Digital H&E/SOX10 Dual Stains Generates High-Performing Convolutional Neural Network for Calculating Tumor Burden in H&E-Stained Cutaneous Melanoma

Deep learning for the analysis of H&E stains requires a large annotated training set. This may form a labor-intensive task involving highly skilled pathologists. We aimed to optimize and evaluate computer-assisted annotation based on digital dual stains of the same tissue section. H&E stains of primary and metastatic melanoma (N = 77) were digitized, re-stained with SOX10, and re-scanned. Because images were aligned, annotations of SOX10 image analysis were directly transferred to H&E stains of the training set. Based on 1,221,367 annotated nuclei, a convolutional neural network for calculating tumor burden (CNN_TB) was developed. For primary melanomas, precision of annotation was 100% (95%CI, 99% to 100%) for tumor cells and 99% (95%CI, 98% to 100%) for normal cells. Due to low or missing tumor-cell SOX10 positivity, precision for normal cells was markedly reduced in lymph-node and organ metastases compared with primary melanomas (p < 0.001). Compared with stereological counts within skin lesions, mean difference in tumor burden was 6% (95%CI, -1% to 13%, p = 0.10) for CNN_TB and 16% (95%CI, 4% to 28%, p = 0.02) for pathologists. Conclusively, the technique produced a large annotated H&E training set with high quality within a reasonable timeframe for primary melanomas and subcutaneous metastases. For these lesion types, the training set generated a high-performing CNN_TB, which was superior to the routine assessments of pathologists.

A hierarchical machine learning model based on Glioblastoma patients' clinical, biomedical, and image data to analyze their treatment plans

AIM OF STUDY

Glioblastoma Multiforme (GBM) is an aggressive brain cancer in adults that kills most patients in the first year due to ineffective treatment. Different clinical, biomedical, and image data features are needed to analyze GBM, increasing complexities. Besides, they lead to weak performances for machine learning models due to ignoring physicians' knowledge. Therefore, this paper proposes a hierarchical model based on Fuzzy C-mean (FCM) clustering, Wrapper feature selection, and twelve classifiers to analyze treatment plans.

METHODOLOGY/APPROACH

The proposed method finds the effectiveness of previous and current treatment plans, hierarchically determining the best decision for future treatment plans for GBM patients using clinical data, biomedical data, and different image data. A case study is presented based on the Cancer Genome Atlas Glioblastoma Multiforme dataset to prove the effectiveness of the proposed model. This dataset is analyzed using data preprocessing, experts' knowledge, and a feature reduction method based on the Principal Component Analysis. Then, the FCM clustering method is utilized to reinforce classifier learning.

OUTCOMES OF STUDY

The proposed model finds the best combination of Wrapper feature selection and classifier for each cluster based on different measures, including accuracy, sensitivity, specificity, precision, F-score, and G-mean according to a hierarchical structure. It has the best performance among other reinforced classifiers. Besides, this model is compatible with real-world medical processes for GBM patients based on clinical, biomedical, and image data.

A Deep Learning Approach to Capture the Essence of Candida albicans Morphologies

We present deep learning-based approaches for exploring the complex array of morphologies exhibited by the opportunistic human pathogen Candida albicans. Our system, entitled Candescence, automatically detects C. albicans cells from differential image contrast microscopy and labels each detected cell with one of nine morphologies. This ranges from yeast white and opaque forms to hyphal and pseudohyphal filamentous morphologies. The software is based upon a fully convolutional one-stage (FCOS) object detector, a deep learning technique that uses an extensive set of images that we manually annotated with the location and morphology of each cell. We developed a novel cumulative curriculum-based learning strategy that stratifies our images by difficulty from simple yeast forms to complex filamentous architectures. Candescence achieves very good performance (~85% recall; 81% precision) on this difficult learning set, where some images contain hundreds of cells with substantial intermixing between the predicted classes. To capture the essence of each C. albicans morphology and how they intermix, we used a second technique from deep learning entitled generative adversarial networks. The resultant models allow us to identify and explore technical variables, developmental trajectories, and morphological switches. Importantly, the model allows us to quantitatively capture morphological plasticity observed with genetically modified strains or strains grown in different media and environments. We envision Candescence as a community meeting point for quantitative explorations of C. albicans morphology. IMPORTANCE The fungus Candida albicans can "shape shift" between 12 morphologies in response to environmental variables. The cytoprotective capacity provided by this polymorphism makes C. albicans a formidable pathogen to treat clinically. Microscopy images of C. albicans colonies can contain hundreds of cells in different morphological states. Manual annotation of images can be difficult, especially as a result of densely packed and filamentous colonies and of technical artifacts from the microscopy itself. Manual annotation is inherently subjective, depending on the experience and opinion of annotators. Here, we built a deep learning approach entitled Candescence to parse images in an automated, quantitative, and objective fashion: each cell in an image is located and labeled with its morphology. Candescence effectively replaces simple rules based on visual phenotypes (size, shape, and shading) with neural circuitry capable of capturing subtle but salient features in images that may be too complex for human annotators.

BRACS: A Dataset for BReAst Carcinoma Subtyping in H&E Histology Images

Breast cancer is the most commonly diagnosed cancer and registers the highest number of deaths for women. Advances in diagnostic activities combined with large-scale screening policies have significantly lowered the mortality rates for breast cancer patients. However, the manual inspection of tissue slides by pathologists is cumbersome, time-consuming and is subject to significant inter- and intra-observer variability. Recently, the advent of whole-slide scanning systems has empowered the rapid digitization of pathology slides and enabled the development of Artificial Intelligence (AI)-assisted digital workflows. However, AI techniques, especially Deep Learning, require a large amount of high-quality annotated data to learn from. Constructing such task-specific datasets poses several challenges, such as data-acquisition level constraints, time-consuming and expensive annotations and anonymization of patient information. In this paper, we introduce the BReAst Carcinoma Subtyping (BRACS) dataset, a large cohort of annotated Hematoxylin and Eosin (H&E)-stained images to advance AI development in the automatic characterization of breast lesions. BRACS contains 547 Whole-Slide Images (WSIs) and 4539 Regions Of Interest (ROIs) extracted from the WSIs. Each WSI and respective ROIs are annotated by the consensus of three board-certified pathologists into different lesion categories. Specifically, BRACS includes three lesion types, i.e., benign, malignant and atypical, which are further subtyped into seven categories. It is, to the best of our knowledge, the largest annotated dataset for breast cancer subtyping both at WSI and ROI levels. Furthermore, by including the understudied atypical lesions, BRACS offers a unique opportunity for leveraging AI to better understand their characteristics. We encourage AI practitioners to develop and evaluate novel algorithms on the BRACS dataset to further breast cancer diagnosis and patient care.

Database URL: https://www.bracs.icar.cnr.it/

Current and Emerging Molecular Markers of Liver Diseases: A Pathogenic Perspective

In the past decade, with the rapid development of molecular medicine and the application of more sophisticated methods for disease diagnosis and treatment, a number of molecular markers have become available for liver diseases. Pathogenesis-related markers are likely to be effectively discovered and rigorously validated, due to the unique biological links to diseases. The present study reviews the predominant clinical and research articles in the previous decade to provide a pathogenic perspective of current and emerging biomarkers for liver diseases, including hepatocellular neoplasms (e.g. hepatocellular carcinoma), non-neoplastic hepatocellular diseases, intrahepatic biliary diseases, and other liver diseases. Although it remains challenging to cover all markers for the diagnosis and prognosis of liver diseases, current and emerging molecular markers in clinical practice and under investigation are reviewed in a wide spectrum of liver diseases, in order to help clinicians and researchers identify liver disease markers for reference.

Histology segmentation using active learning on regions of interest in oral cavity squamous cell carcinoma

In digital pathology, deep learning has been shown to have a wide range of applications, from cancer grading to segmenting structures like glomeruli. One of the main hurdles for digital pathology to be truly effective is the size of the dataset needed for generalization to address the spectrum of possible morphologies. Small datasets limit classifiers' ability to generalize. Yet, when we move to larger datasets of whole slide images (WSIs) of tissue, these datasets may cause network bottlenecks as each WSI at its original magnification can be upwards of 100 000 by 100 000 pixels, and over a gigabyte in file size. Compounding this problem, high quality pathologist annotations are difficult to obtain, as the volume of necessary annotations to create a classifier that can generalize would be extremely costly in terms of pathologist-hours. In this work, we use Active Learning (AL), a process for iterative interactive training, to create a modified U-net classifier on the region of interest (ROI) scale. We then compare this to Random Learning (RL), where images for addition to the dataset for retraining are randomly selected. Our hypothesis is that AL shows benefits for generating segmentation results versus randomly selecting images to annotate. We show that after 3 iterations, that AL, with an average Dice coefficient of 0.461, outperforms RL, with an average Dice Coefficient of 0.375, by 0.086.

Energy System 4.0: Digitalization of the Energy Sector with Inclination towards Sustainability

The United Nations' sustainable development goals have emphasized implementing sustainability to ensure environmental security for the future. Affordable energy, clean energy, and innovation in infrastructure are the relevant sustainable development goals that are applied to the energy sector. At present, digital technologies have a significant capability to realize the target of sustainability in energy. With this motivation, the study aims to discuss the significance of different digital technologies such as the Internet of Things (IoT), artificial intelligence (AI), edge computing, blockchain, and big data and their implementation in the different stages of energy such as generation, distribution, transmission, smart grid, and energy trading. The study also discusses the different architecture that has been implemented by previous studies for smart grid computing. Additionally, we addressed IoT-based microgrids, IoT services in electrical equipment, and blockchain-based energy trading. Finally, the article discusses the challenges and recommendations for the effective implementation of digital technologies in the energy sector for meeting sustainability. Big data for energy analytics, digital twins in smart grid modeling, virtual power plants with Metaverse, and green IoT are the major vital recommendations that are discussed in this study for future enhancement.

Identification of technology frontiers of artificial intelligence-assisted pathology based on patent citation network

OBJECTIVES

This paper aimed to identify the technology frontiers of artificial intelligence-assisted pathology based on patent citation network.

METHODS

Patents related to artificial intelligence-assisted pathology were searched and collected from the Derwent Innovation Index (DII), which were imported into Derwent Data Analyzer (DDA, Clarivate Derwent, New York, NY, USA) for authority control, and imported into the freely available computer program Ucinet 6 for drawing the patent citation network. The patent citation network according to the citation relationship could describe the technology development context in the field of artificial intelligence-assisted pathology. The patent citations were extracted from the collected patent data, selected highly cited patents to form a co-occurrence matrix, and built a patent citation network based on the co-occurrence matrix in each period. Text clustering is an unsupervised learning method, an important method in text mining, where similar documents are grouped into clusters. The similarity between documents are determined by calculating the distance between them, and the two documents with the closest distance are combined. The method of text clustering was used to identify the technology frontiers based on the patent citation network, which was according to co-word analysis of the title and abstract of the patents in this field.

RESULTS

1704 patents were obtained in the field of artificial intelligence-assisted pathology, which had been currently undergoing three stages, namely the budding period (1992-2000), the development period (2001-2015), and the rapid growth period (2016-2021). There were two technology frontiers in the budding period (1992-2000), namely systems and methods for image data processing in computerized tomography (CT), and immunohistochemistry (IHC), five technology frontiers in the development period (2001-2015), namely spectral analysis methods of biomacromolecules, pathological information system, diagnostic biomarkers, molecular pathology diagnosis, and pathological diagnosis antibody, and six technology frontiers in the rapid growth period (2016-2021), namely digital pathology (DP), deep learning (DL) algorithms-convolutional neural networks (CNN), disease prediction models, computational pathology, pathological image analysis method, and intelligent pathological system.

CONCLUSIONS

Artificial intelligence-assisted pathology was currently in a rapid development period, and computational pathology, DL and other technologies in this period all involved the study of algorithms. Future research hotspots in this field would focus on algorithm improvement and intelligent diagnosis in order to realize the precise diagnosis. The results of this study presented an overview of the characteristics of research status and development trends in the field of artificial intelligence-assisted pathology, which could help readers broaden innovative ideas and discover new technological opportunities, and also served as important indicators for government policymaking.

Artificial Intelligence-Assisted Renal Pathology: Advances and Prospects

Digital imaging and advanced microscopy play a pivotal role in the diagnosis of kidney diseases. In recent years, great achievements have been made in digital imaging, providing novel approaches for precise quantitative assessments of nephropathology and relieving burdens of renal pathologists. Developing novel methods of artificial intelligence (AI)-assisted technology through multidisciplinary interaction among computer engineers, renal specialists, and nephropathologists could prove beneficial for renal pathology diagnoses. An increasing number of publications has demonstrated the rapid growth of AI-based technology in nephrology. In this review, we offer an overview of AI-assisted renal pathology, including AI concepts and the workflow of processing digital image data, focusing on the impressive advances of AI application in disease-specific backgrounds. In particular, this review describes the applied computer vision algorithms for the segmentation of kidney structures, diagnosis of specific pathological changes, and prognosis prediction based on images. Lastly, we discuss challenges and prospects to provide an objective view of this topic.

Analysis of neuroendocrine clones in NSCLCs using an immuno-guided laser-capture microdissection-based approach

Clonal evolution and lineage plasticity are key contributors to tumor heterogeneity and response to treatment in cancer. However, capturing signal transduction events in coexisting clones remains challenging from a technical perspective. In this study, we developed and tested a signal-transduction-based workflow to isolate and profile coexisting clones within a complex cellular system like non-small cell lung cancers (NSCLCs). Cooccurring clones were isolated under immunohistochemical guidance using laser-capture microdissection, and cell signaling activation portraits were measured using the reverse-phase protein microarray. To increase the translational potential of this work and capture druggable vulnerabilities within different clones, we measured expression/activation of a panel of key drug targets and downstream substrates of FDA-approved or investigational agents. We isolated intermixed clones, including poorly represented ones (<5% of cells), within the tumor microecology and identified molecular characteristics uniquely attributable to cancer cells that undergo lineage plasticity and neuroendocrine transdifferentiation in NSCLCs.

Development of a Fully Automated Method to Obtain Reproducible Lymphocyte Counts in Patients With Colorectal Cancer

Colorectal cancer (CRC) is the third most common cancer worldwide. Although clinical outcome varies among patients diagnosed within the same TNM stage it is the cornerstone in treatment decisions as well as follow-up programmes. Tumor-infiltrating lymphocytes have added value when evaluating survival outcomes. The aim of this study was to develop a fully automated method for quantification of subsets of T lymphocytes in the invasive margin and central tumor in patients with CRC based on Deep Learning powered artificial intelligence. The study cohort consisted of 163 consecutive patients with a primary diagnosis of CRC followed by a surgical resection. Double-labeling immunohistochemical staining with cytokeratin in combination with CD3 or CD8, respectively, was performed on 1 representative slide from each patient. Visiopharm Quantitative Digital Pathology software was used to develop Application Protocol Packages for visualization of architectural details (background, normal epithelium, cancer epithelium, surrounding tissue), identification of central tumor and invasive margin as well as subsequent quantitative analysis of immune cells. Fully automated counts for CD3 and CD8 positive T cells were obtained in 93% and 92% of the cases, respectively. In the remaining cases, manual editing was required. In conclusion, the development of a fully automated method for counting CD3 + and CD8 + lymphocytes in a cohort of patients with CRC provided excellent results eliminating not only observer variability in lymphocyte counts but also in identifying the regions of interest for the quantitative analysis. Validation of the performance of the Application Protocol Packages including clinical correlation is needed.

Computational pathology in ovarian cancer

Histopathologic evaluations of tissue sections are key to diagnosing and managing ovarian cancer. Pathologists empirically assess and integrate visual information, such as cellular density, nuclear atypia, mitotic figures, architectural growth patterns, and higher-order patterns, to determine the tumor type and grade, which guides oncologists in selecting appropriate treatment options. Latent data embedded in pathology slides can be extracted using computational imaging. Computers can analyze digital slide images to simultaneously quantify thousands of features, some of which are visible with a manual microscope, such as nuclear size and shape, while others, such as entropy, eccentricity, and fractal dimensions, are quantitatively beyond the grasp of the human mind. Applications of artificial intelligence and machine learning tools to interpret digital image data provide new opportunities to explore and quantify the spatial organization of tissues, cells, and subcellular structures. In comparison to genomic, epigenomic, transcriptomic, and proteomic patterns, morphologic and spatial patterns are expected to be more informative as quantitative biomarkers of complex and dynamic tumor biology. As computational pathology is not limited to visual data, nuanced subvisual alterations that occur in the seemingly “normal” pre-cancer microenvironment could facilitate research in early cancer detection and prevention. Currently, efforts to maximize the utility of computational pathology are focused on integrating image data with other -omics platforms that lack spatial information, thereby providing a new way to relate the molecular, spatial, and microenvironmental characteristics of cancer. Despite a dire need for improvements in ovarian cancer prevention, early detection, and treatment, the ovarian cancer field has lagged behind other cancers in the application of computational pathology. The intent of this review is to encourage ovarian cancer research teams to apply existing and/or develop additional tools in computational pathology for ovarian cancer and actively contribute to advancing this important field.

The Significance of External Quality Assessment Schemes for Molecular Testing in Clinical Laboratories

Simple Summary

Patients and clinicians often rely on the outcome of laboratory tests, but can we really trust these test results? Good quality management is key for laboratories to guarantee reliable test results. This review focusses on external quality assessment (EQA) schemes which are a tool for laboratories to examine and improve the quality of their testing routines. In this review, an overview of the role and importance of EQA schemes for clinical laboratories is given, and different types of EQA schemes and EQA providers available on the market are discussed, as well as recent developments in the EQA landscape.

Abstract

External quality assessment (EQA) schemes are a tool for clinical laboratories to evaluate and manage the quality of laboratory practice with the support of an independent party (i.e., an EQA provider). Depending on the context, there are different types of EQA schemes available, as well as various EQA providers, each with its own field of expertise. In this review, an overview of the general requirements for EQA schemes and EQA providers based on international guidelines is provided. The clinical and scientific value of these kinds of schemes for clinical laboratories, clinicians and patients are highlighted, in addition to the support EQA can provide to other types of laboratories, e.g., laboratories affiliated to biotech companies. Finally, recent developments and challenges in laboratory medicine and quality management, for example, the introduction of artificial intelligence in the laboratory and the shift to a more individual-approach instead of a laboratory-focused approach, are discussed. EQA schemes should represent current laboratory practice as much as possible, which poses the need for EQA providers to introduce latest laboratory innovations in their schemes and to apply up-to-date guidelines. By incorporating these state-of-the-art techniques, EQA aims to contribute to continuous learning.

Automated histological classification for digital pathology images of colonoscopy specimen via deep learning

Colonoscopy is an effective tool to detect colorectal lesions and needs the support of pathological diagnosis. This study aimed to develop and validate deep learning models that automatically classify digital pathology images of colon lesions obtained from colonoscopy-related specimen. Histopathological slides of colonoscopic biopsy or resection specimens were collected and grouped into six classes by disease category: adenocarcinoma, tubular adenoma (TA), traditional serrated adenoma (TSA), sessile serrated adenoma (SSA), hyperplastic polyp (HP), and non-specific lesions. Digital photographs were taken of each pathological slide to fine-tune two pre-trained convolutional neural networks, and the model performances were evaluated. A total of 1865 images were included from 703 patients, of which 10% were used as a test dataset. For six-class classification, the mean diagnostic accuracy was 97.3% (95% confidence interval [CI], 96.0–98.6%) by DenseNet-161 and 95.9% (95% CI 94.1–97.7%) by EfficientNet-B7. The per-class area under the receiver operating characteristic curve (AUC) was highest for adenocarcinoma (1.000; 95% CI 0.999–1.000) by DenseNet-161 and TSA (1.000; 95% CI 1.000–1.000) by EfficientNet-B7. The lowest per-class AUCs were still excellent: 0.991 (95% CI 0.983–0.999) for HP by DenseNet-161 and 0.995 for SSA (95% CI 0.992–0.998) by EfficientNet-B7. Deep learning models achieved excellent performances for discriminating adenocarcinoma from non-adenocarcinoma lesions with an AUC of 0.995 or 0.998. The pathognomonic area for each class was appropriately highlighted in digital images by saliency map, particularly focusing epithelial lesions. Deep learning models might be a useful tool to help the diagnosis for pathologic slides of colonoscopy-related specimens.

Advantages of manual and automatic computer-aided compared to traditional histopathological diagnosis of melanoma: A pilot study

BACKGROUND

Cutaneous malignant melanoma (CMM) accounts for the highest mortality rate among all skin cancers. Traditional histopathologic diagnosis may be limited by the pathologists' subjectivity. Second-opinion strategies and multidisciplinary consultations are usually performed to overcome this issue. An available solution in the future could be the use of automated solutions based on a computational algorithm that could help the pathologist in everyday practice. The aim of this pilot study was to investigate the potential diagnostic aid of a machine-based algorithm in the histopathologic diagnosis of CMM.

METHODS

We retrospectively examined excisional biopsies of 50 CMM and 20 benign congenital compound nevi. Hematoxylin and eosin (H&E) stained WSI were reviewed independently by two expert dermatopathologists. A fully automated pipeline for WSI processing to support the estimation and prioritization of the melanoma areas was developed.

RESULTS

The spatial distribution of the nuclei in the sample provided a multi-scale overview of the tumor. A global overview of the lesion's silhouette was achieved and, by increasing the magnification, the topological distribution of the nuclei and the most informative areas of interest for the CMM diagnosis were identified and highlighted. These silhouettes allow the histopathologist to discriminate between nevus and CMM with an accuracy of 96% without any extra information.

CONCLUSION

In this study we proposed an easy-to-use model that produces segmentations of CMM silhouettes at fine detail level.

The state of the art for artificial intelligence in lung digital pathology

Lung diseases carry a significant burden of morbidity and mortality worldwide. The advent of digital pathology (DP) and an increase in computational power have led to the development of artificial intelligence (AI)-based tools that can assist pathologists and pulmonologists in improving clinical workflow and patient management. While previous works have explored the advances in computational approaches for breast, prostate, and head and neck cancers, there has been a growing interest in applying these technologies to lung diseases as well. The application of AI tools on radiology images for better characterization of indeterminate lung nodules, fibrotic lung disease, and lung cancer risk stratification has been well documented. In this article, we discuss methodologies used to build AI tools in lung DP, describing the various hand-crafted and deep learning-based unsupervised feature approaches. Next, we review AI tools across a wide spectrum of lung diseases including cancer, tuberculosis, idiopathic pulmonary fibrosis, and COVID-19. We discuss the utility of novel imaging biomarkers for different types of clinical problems including quantification of biomarkers like PD-L1, lung disease diagnosis, risk stratification, and prediction of response to treatments such as immune checkpoint inhibitors. We also look briefly at some emerging applications of AI tools in lung DP such as multimodal data analysis, 3D pathology, and transplant rejection. Lastly, we discuss the future of DP-based AI tools, describing the challenges with regulatory approval, developing reimbursement models, planning clinical deployment, and addressing AI biases. © 2022 The Authors. The Journal of Pathology published by John Wiley & Sons Ltd on behalf of The Pathological Society of Great Britain and Ireland.

Pilot study to evaluate tools to collect pathologist annotations for validating machine learning algorithms

Purpose: Validation of artificial intelligence (AI) algorithms in digital pathology with a reference standard is necessary before widespread clinical use, but few examples focus on creating a reference standard based on pathologist annotations. This work assesses the results of a pilot study that collects density estimates of stromal tumor-infiltrating lymphocytes (sTILs) in breast cancer biopsy specimens. This work will inform the creation of a validation dataset for the evaluation of AI algorithms fit for a regulatory purpose. Approach: Collaborators and crowdsourced pathologists contributed glass slides, digital images, and annotations. Here, "annotations" refer to any marks, segmentations, measurements, or labels a pathologist adds to a report, image, region of interest (ROI), or biological feature. Pathologists estimated sTILs density in 640 ROIs from hematoxylin and eosin stained slides of 64 patients via two modalities: an optical light microscope and two digital image viewing platforms. Results: The pilot study generated 7373 sTILs density estimates from 29 pathologists. Analysis of annotations found the variability of density estimates per ROI increases with the mean; the root mean square differences were 4.46, 14.25, and 26.25 as the mean density ranged from 0% to 10%, 11% to 40%, and 41% to 100%, respectively. The pilot study informs three areas of improvement for future work: technical workflows, annotation platforms, and agreement analysis methods. Upgrades to the workflows and platforms will improve operability and increase annotation speed and consistency. Conclusions: Exploratory data analysis demonstrates the need to develop new statistical approaches for agreement. The pilot study dataset and analysis methods are publicly available to allow community feedback. The development and results of the validation dataset will be publicly available to serve as an instructive tool that can be replicated by developers and researchers.

Rethinking ImageNet Pre-training for Computational Histopathology

Transfer learning from ImageNet pretrained weights is widely used when training Deep Learning models on a Histopathology dataset. However, the visual features of the two domains are different. Rather than ImageNet pretrained weights, pre-training on a Histopathology dataset may provide better initialization. To prove this hypothesis, we train two commonly used Deep Learning model architectures - ResNet and DenseNet on a complex Histopathology classification dataset, and compare transfer learning performance with ImageNet pretrained weights. Based on the fine-tuning on three histopathology datasets including two different stains (H&E and IHC), we show that the domain specific pretrained weights are better suited for transfer learning. This is reflected by higher performance, lower training time as well as better feature reuse. Clinical Relevance - The paper establishes merit of using Histopathology domain specific pretrained weights rather than ImageNet pretrained weights.

Lymphocytes in the neighborhood: good or bad for the kidney?

Lupus nephritis (LN) is common in people with systemic lupus erythematosus (SLE) and advances, almost invariably, to end-stage renal disease (ESRD). In this issue of the JCI, Abraham, Durkee, et al. presented a large-scale immune cell landscape of kidney biopsies from patients with LN by combining multiplexed confocal microscopy imaging with customized computer vision and quantification. The presence of diverse CD4^– T cells in small neighborhoods, but not of B cells or CD4⁺ T cells in large neighborhoods, is linked to the development of ESRD. Unexpectedly, B cells in the kidney heralded a good prognosis. The precise location of different types of immune cells allows inference on possible interactions between different immune cells and also between immune and kidney-resident cells. The data have important implications on the development of prognostic tools and effective targeted therapies in patients with LN.

Improving feature extraction from histopathological images through a fine-tuning ImageNet model

Background

Due to lack of annotated pathological images, transfer learning has been the predominant approach in the field of digital pathology. Pre-trained neural networks based on ImageNet database are often used to extract "off-the-shelf" features, achieving great success in predicting tissue types, molecular features, and clinical outcomes, etc. We hypothesize that fine-tuning the pre-trained models using histopathological images could further improve feature extraction, and downstream prediction performance.

Methods

We used 100 000 annotated H&E image patches for colorectal cancer (CRC) to fine-tune a pre-trained Xception model via a 2-step approach. The features extracted from fine-tuned Xception (FTX-2048) model and Image-pretrained (IMGNET-2048) model were compared through: (1) tissue classification for H&E images from CRC, same image type that was used for fine-tuning; (2) prediction of immune-related gene expression, and (3) gene mutations for lung adenocarcinoma (LUAD). Five-fold cross validation was used for model performance evaluation. Each experiment was repeated 50 times.

Findings

The extracted features from the fine-tuned FTX-2048 exhibited significantly higher accuracy (98.4%) for predicting tissue types of CRC compared to the "off-the-shelf" features directly from Xception based on ImageNet database (96.4%) (P value = 2.2 × 10-6). Particularly, FTX-2048 markedly improved the accuracy for stroma from 87% to 94%. Similarly, features from FTX-2048 boosted the prediction of transcriptomic expression of immune-related genes in LUAD. For the genes that had significant relationships with image features (P < 0.05, n = 171), the features from the fine-tuned model improved the prediction for the majority of the genes (139; 81%). In addition, features from FTX-2048 improved prediction of mutation for 5 out of 9 most frequently mutated genes (STK11, TP53, LRP1B, NF1, and FAT1) in LUAD.

Conclusions

We proved the concept that fine-tuning the pretrained ImageNet neural networks with histopathology images can produce higher quality features and better prediction performance for not only the same-cancer tissue classification where similar images from the same cancer are used for fine-tuning, but also cross-cancer prediction for gene expression and mutation at patient level.

Artificial Intelligence-Assisted Image Analysis of Acetaminophen-Induced Acute Hepatic Injury in Sprague-Dawley Rats

Although drug-induced liver injury (DILI) is a major target of the pharmaceutical industry, we currently lack an efficient model for evaluating liver toxicity in the early stage of its development. Recent progress in artificial intelligence-based deep learning technology promises to improve the accuracy and robustness of current toxicity prediction models. Mask region-based CNN (Mask R-CNN) is a detection-based segmentation model that has been used for developing algorithms. In the present study, we applied a Mask R-CNN algorithm to detect and predict acute hepatic injury lesions induced by acetaminophen (APAP) in Sprague-Dawley rats. To accomplish this, we trained, validated, and tested the model for various hepatic lesions, including necrosis, inflammation, infiltration, and portal triad. We confirmed the model performance at the whole-slide image (WSI) level. The training, validating, and testing processes, which were performed using tile images, yielded an overall model accuracy of 96.44%. For confirmation, we compared the model’s predictions for 25 WSIs at 20× magnification with annotated lesion areas determined by an accredited toxicologic pathologist. In individual WSIs, the expert-annotated lesion areas of necrosis, inflammation, and infiltration tended to be comparable with the values predicted by the algorithm. The overall predictions showed a high correlation with the annotated area. The R square values were 0.9953, 0.9610, and 0.9445 for necrosis, inflammation plus infiltration, and portal triad, respectively. The present study shows that the Mask R-CNN algorithm is a useful tool for detecting and predicting hepatic lesions in non-clinical studies. This new algorithm might be widely useful for predicting liver lesions in non-clinical and clinical settings.

Establishment of a whole slide imaging-based frozen section service at a cancer center

Background

In recent years, there has been a surge of interest in clinical digital pathology (DP). Hardware and software platforms have matured and become more affordable, and advances in artificial intelligence promise to transform the practice of pathology. At our institution, we are launching a stepwise process of DP adoption which will eventually encompass our entire workflow. Out of necessity, we began by establishing a whole slide imaging (WSI)-based frozen section service.

Methods

We proceeded in a systematic manner by first assembling a team of key stakeholders. We carefully evaluated the various options for digitizing frozen sections before deciding that a WSI-based solution made the most sense for us. We used a formalized evaluation system to quantify performance metrics that were relevant to us. After deciding on a WSI-based system, we likewise carefully considered the various whole slide scanners and digital slide management systems available before making decisions.

Results

During formal evaluation by pathologists, the WSI-based system outperformed competing platforms. Although implementation was relatively complex, we have been happy with the results and have noticed significant improvements in our frozen section turnaround time. Our users have been happy with the slide management system, which we plan on utilizing in future DP efforts.

Conclusions

There are various options for digitizing frozen section slides. Although WSI-based systems are more complex and expensive than some alternatives, they perform well and may make sense for institutions with a pre-existing or planned larger DP infrastructure.

Artificial intelligence in prostate cancer: Definitions, current research, and future directions

Multiple novel modalities tasking artificial intelligence based computational pathology applications and integrating other variables, such as risk factors, tumor microenvironment, genomic testing data, laboratory findings, clinical history, and radiology findings, will improve diagnostic consistency and generate a synergistic diagnostic workflow. In this article, we present the concise and contemporary review on the utilization of artificial intelligence in prostate cancer and identify areas for possible future applications.

The Challenges and Opportunities of Translational Pathology

Translational pathology has not caught up with the quality and quantity of translational medicine. Thus, challenges and opportunities related to translational pathology are discussed here. Pathologists should actively participate in reverse translational research that seeks mechanistic insights to explain clinical findings and/or solve clinical problems. Challenges in translational pathology include ambiguity in defining translational pathology, pathologists' mindsets about translational research, lack of sufficient workforce and immature publication outlets. However, with collective wisdom and support of various stakeholders, we can expand the pool of pathologist scientists, build a translational pathology community and drive innovations in medicine through computational, molecular genetic/genomic and digital pathology approaches.

An analysis of pathologists' viewing processes as they diagnose whole slide digital images

Although pathologists have their own viewing habits while diagnosing, viewing behaviors leading to the most accurate diagnoses are under-investigated. Digital whole slide imaging has enabled investigators to analyze pathologists' visual interpretation of histopathological features using mouse and viewport tracking techniques. In this study, we provide definitions for basic viewing behavior variables and investigate the association of pathologists' characteristics and viewing behaviors, and how they relate to diagnostic accuracy when interpreting whole slide images. We use recordings of 32 pathologists' actions while interpreting a set of 36 digital whole slide skin biopsy images (5 sets of 36 cases; 180 cases total). These viewport tracking data include the coordinates of a viewport scene on pathologists' screens, the magnification level at which that viewport was viewed, as well as a timestamp. We define a set of variables to quantify pathologists' viewing behaviors such as zooming, panning, and interacting with a consensus reference panel's selected region of interest (ROI). We examine the association of these viewing behaviors with pathologists' demographics, clinical characteristics, and diagnostic accuracy using cross-classified multilevel models. Viewing behaviors differ based on clinical experience of the pathologists. Pathologists with a higher caseload of melanocytic skin biopsy cases and pathologists with board certification and/or fellowship training in dermatopathology have lower average zoom and lower variance of zoom levels. Viewing behaviors associated with higher diagnostic accuracy include higher average and variance of zoom levels, a lower magnification percentage (a measure of consecutive zooming behavior), higher total interpretation time, and higher amount of time spent viewing ROIs. Scanning behavior, which refers to panning with a fixed zoom level, has marginally significant positive association with accuracy. Pathologists' training, clinical experience, and their exposure to a range of cases are associated with their viewing behaviors, which may contribute to their diagnostic accuracy. Research in computational pathology integrating digital imaging and clinical informatics opens up new avenues for leveraging viewing behaviors in medical education and training, potentially improving patient care and the effectiveness of clinical workflow.

Automated Cancer Diagnostics via Analysis of Optical and Chemical Images by Deep and Shallow Learning

Optical microscopy has long been the gold standard to analyse tissue samples for the diagnostics of various diseases, such as cancer. The current diagnostic workflow is time-consuming and labour-intensive, and manual annotation by a qualified pathologist is needed. With the ever-increasing number of tissue blocks and the complexity of molecular diagnostics, new approaches have been developed as complimentary or alternative solutions for the current workflow, such as digital pathology and mass spectrometry imaging (MSI). This study compares the performance of a digital pathology workflow using deep learning for tissue recognition and an MSI approach utilising shallow learning to annotate formalin-fixed and paraffin-embedded (FFPE) breast cancer tissue microarrays (TMAs). Results show that both deep learning algorithms based on conventional optical images and MSI-based shallow learning can provide automated diagnostics with F1-scores higher than 90%, with the latter intrinsically built on biochemical information that can be used for further analysis.

Usual interstitial pneumonia: a clinically significant pattern, but not the final word

Usual interstitial pneumonia (UIP) is a concept that is deeply entrenched in clinical practice and the prognostic significance of UIP is well established, but the field continues to suffer from the lack of a true gold standard for diagnosing fibrotic interstitial lung disease (ILD). The meaning and usage of UIP have shifted over time and this term is prone to misinterpretation and poor diagnostic agreement. For pathologists, it is worth reflecting on the limitations of UIP and our true role in the care of patients with ILD, a controversial topic explored in two point-counterpoint editorials published simultaneously in this journal. Current diagnostic guidelines are ambiguous and difficult to apply in clinical practice. Further complicating matters for the pathologist is the paradigm shift that occurred with the advent of anti-fibrotic agents, necessitating increased focus on the most likely etiology of fibrosis rather than simply the pattern of fibrosis when pulmonologists select appropriate therapy. Despite the wealth of information locked in tissue samples that could provide novel insights into fibrotic ILDs, pulmonologists increasingly shy away from obtaining biopsies, likely because pathologists no longer provide sufficient value to offset the risks of a biopsy procedure, and pathologic assessment is insufficiently reliable to meaningfully inform therapeutic decisionmaking. To increase the value of biopsies, pathologists must first recognize the problems with UIP as a diagnostic term. Second, pathologists must realize that the primary goal of a biopsy is to determine the most likely etiology to target with therapy, requiring a shift in diagnostic focus. Third, pathologists must devise and validate new classifications and criteria that are evidence-based, biologically relevant, easy to use, and predictive of outcome and treatment response. Only after the limitations of UIP are understood will pathologists provide maximum diagnostic value from biopsies to clinicians today and advance the field forward.

BestCyte® Cell Sorter Imaging System: Primary and Adjudicative Whole Slide Image Rescreening Review Times of 500 ThinPrep Pap Test Thin-layers - An Intra-observer, Time-Surrogate Analysis of Diagnostic Confidence Potentialities

Background

The novel Artificial Intelligence-driven BestCyte® Cell Sorter Imaging System (BestCyte) enables hybrid digital screening through classification and sorting of tiles depicting cells in 8 galleries or whole slide image (WSI) reviews.

Objectives

(1) Analyze expenditures of time (minutes) for primary BestCyte cell sorter screening and adjudicative WSI rescreening of 500 blinded, randomized ThinPrep thin-layers to determine review times per Bethesda nomenclature; (2) Analyze review times for NILM qualifier diagnoses reflecting increasing interpretive complexity (i.e., Inflammation, Reactive/Repair, Bacterial cytolysis, Bacterial vaginosis, Atrophy, and Atrophic vaginitis); (3) Challenge accuracy of primary diagnoses (Downgraded, Upheld, and Upgraded) following adjudicative WSI rescreening to assess correlated review times as surrogate indicators of diagnostic confidence in BestCyte functionality (i.e., learning curve); and (4) Correlate primary and adjudicative diagnoses to calculate intra-observer reproducibility Kappa coefficients per Bethesda nomenclature.

Results

Of 500 thin-layers, the mean [primary/adjudicative rescreening review times (minutes)] were: Overall study [1.38/3.94], NILM [1.23/3.02], ASCUS [1.18/2.53], ASC-H [1.73/4.86], AGUS [1.84/6.34], LSIL [1.49/4.16], HSIL [1.52/4.10], CA [0.65/2.57]. Of 500 primary Bethesda diagnoses: 2 (0.40%) downgraded; 483 (96.6%) upheld; 15 (3.00%) upgraded after adjudicative WSI rescreening. Of 354 NILM diagnoses: 0 downgraded; 344 (97.2%) upheld; 10 (2.82%) upgraded. Of 34 ASCUS diagnoses: 2 (5.88%) downgraded; 28 (82.4%) upheld; 4 (11.8%) upgraded. Of 17 ASC-H diagnoses: 0 downgraded; 16 (94.1%) upheld; 1 (5.88%) upgraded. Of AGUS (n=1), LSIL (n=24), HSIL (n=52), CA (n=1), UNSAT (n=17): 100% upheld. Kappa coefficients with 95% (Confidence Intervals): Overall study 0.9305 (0.8983–0.9627), NILM 0.9429 (0.9110–0.9748), ASCUS 0.8378 (0.7393–0.9363), ASC-H 0.9112 (0.8113–0.9999), AGUS 1.0 (1.0–1.0), LSIL 0.9189 (0.8400–0.9978), HSIL 0.9894 (0.9685–0.9999), CA 1.0 (1.0–1.0), UNSAT 1.0 (1.0–1.0). Primary BestCyte cell image review time trends for NILM, ASCUS, LSIL, and HSIL, revealed plateaus relative to decreasing respective adjudicative WSI rescreening times.

Conclusions

Given innovative robustness, BestCyte accommodates interpretive fundamentals, enabling shorter ThinPrep thin-layer review times with optimal intra-observer concordance per Bethesda nomenclature through classifying, ranking, sorting, and displaying clinically relevant cells efficiently in galleries. BestCyte fosters continuously optimizing diagnostic confidence learning curves; may supplant manual microscopy for primary screening.

Applications of machine learning in routine laboratory medicine: Current state and future directions

Machine learning is able to leverage large amounts of data to infer complex patterns that are otherwise beyond the capabilities of rule-based systems and human experts. Its application to laboratory medicine is particularly exciting, as laboratory testing provides much of the foundation for clinical decision making. In this article, we provide a brief introduction to machine learning for the medical professional in addition to a comprehensive literature review outlining the current state of machine learning as it has been applied to routine laboratory medicine. Although still in its early stages, machine learning has been used to automate laboratory tasks, optimize utilization, and provide personalized reference ranges and test interpretation. The published literature leads us to believe that machine learning will be an area of increasing importance for the laboratory practitioner. We envision the laboratory of the future will utilize these methods to make significant improvements in efficiency and diagnostic precision.

Application and performance of artificial intelligence technology in cytopathology

Deep learning algorithms and artificial intelligence (AI) are making great progress in their capacity to evaluate and interpret image data recent advancements in computer vision and machine learning. The first use of AI in a pathology lab was in cytopathology, when a computer-assisted Pap test screening was created. Initially designed to diagnose rather than screen, there was a lot of disagreement concerning their wide use to clinical specimens. However, whole-slide imaging of both gynaecological and non-gynaecological histopathology have been the subject of recent AI work. An overview of the literature on AI in cytopathology is provided in this brief review. To be more precise, it intends to emphasize the relevance of applications of AI algorithms to gynaecological and non-gynaecologic cytology. Between January 2000 and December 2021, a search on artificial intelligence in cytopathology was conducted in several well-known databases, including PubMed, Web of Science, Scopus, Embase, and Google Scholar. Only full-text papers that could be accessed online were evaluated.

Developing image analysis methods for digital pathology

The potential to use quantitative image analysis and artificial intelligence is one of the driving forces behind digital pathology. However, despite novel image analysis methods for pathology being described across many publications, few become widely adopted and many are not applied in more than a single study. The explanation is often straightforward: software implementing the method is simply not available, or is too complex, incomplete, or dataset‐dependent for others to use. The result is a disconnect between what seems already possible in digital pathology based upon the literature, and what actually is possible for anyone wishing to apply it using currently available software. This review begins by introducing the main approaches and techniques involved in analysing pathology images. I then examine the practical challenges inherent in taking algorithms beyond proof‐of‐concept, from both a user and developer perspective. I describe the need for a collaborative and multidisciplinary approach to developing and validating meaningful new algorithms, and argue that openness, implementation, and usability deserve more attention among digital pathology researchers. The review ends with a discussion about how digital pathology could benefit from interacting with and learning from the wider bioimage analysis community, particularly with regard to sharing data, software, and ideas. © 2022 The Author. The Journal of Pathology published by John Wiley & Sons Ltd on behalf of The Pathological Society of Great Britain and Ireland.

Artificial intelligence significantly improves the diagnostic accuracy of deep myxoid soft tissue lesions in histology

Deep myxoid soft tissue lesions have posed a diagnostic challenge for pathologists due to significant histological overlap and regional heterogeneity, especially when dealing with small biopsies which have profoundly low accuracy. However, accurate diagnosis is important owing to difference in biological behaviors and response to adjuvant therapy, that will guide the extent of surgery and the need for neo-adjuvant therapy. Herein, we trained two convolutional neural network models based on a total of 149,130 images representing diagnoses of extra skeletal myxoid chondrosarcoma, intramuscular myxoma, low-grade fibromyxoid sarcoma, myxofibrosarcoma and myxoid liposarcoma. Both AI models outperformed all the pathologists, with a significant improvement of accuracy up to 97% compared to average pathologists of 69.7% (p < 0.00001), corresponding to 90% reduction in error rate. The area under curve of the best AI model was on average 0.9976. It could assist pathologists in clinical practice for accurate diagnosis of deep soft tissue myxoid lesions, and guide clinicians for precise and optimal treatment for patients.

Flap failure prediction in microvascular tissue reconstruction using machine learning algorithms

BACKGROUND

Microvascular tissue reconstruction is a well-established, commonly used technique for a wide variety of the tissue defects. However, flap failure is associated with an additional hospital stay, medical cost burden, and mental stress. Therefore, understanding of the risk factors associated with this event is of utmost importance.

AIM

To develop machine learning-based predictive models for flap failure to identify the potential factors and screen out high-risk patients.

METHODS

Using the data set of 946 consecutive patients, who underwent microvascular tissue reconstruction of free flap reconstruction for head and neck, breast, back, and extremity, we established three machine learning models including random forest classifier, support vector machine, and gradient boosting. Model performances were evaluated by the indicators such as area under the curve of receiver operating characteristic curve, accuracy, precision, recall, and F1 score. A multivariable regression analysis was performed for the most critical variables in the random forest model.

RESULTS

Post-surgery, the flap failure event occurred in 34 patients (3.6%). The machine learning models based on various preoperative and intraoperative variables were successfully developed. Among them, the random forest classifier reached the best performance in receiver operating characteristic curve, with an area under the curve score of 0.770 in the test set. The top 10 variables in the random forest were age, body mass index, ischemia time, smoking, diabetes, experience, prior chemotherapy, hypertension, insulin, and obesity. Interestingly, only age, body mass index, and ischemic time were statistically associated with the outcomes.

CONCLUSION

Machine learning-based algorithms, especially the random forest classifier, were very important in categorizing patients at high risk of flap failure. The occurrence of flap failure was a multifactor-driven event and was identified with numerous factors that warrant further investigation. Importantly, the successful application of machine learning models may help the clinician in decision-making, understanding the underlying pathologic mechanisms of the disease, and improving the long-term outcome of patients.

Artificial neural network in diagnostic cytology

The artificial neural network (ANN) is a computer software design or model that simulates the biological neural network of the human brain. Instead of biological neurons, ANN is composed of many layers of nodes that carry the signal and process it to make the final decision. ANN is a modern technology that is widely used in different fields of science. The ANN is reshaping the medical system and the various areas of pathology. In this paper, the basic concept and applications of ANN in cytology have been discussed. In this paper, the various articles published on ANN in the field of cytology have been systemically reviewed. The ANN is relatively less used in cytology. After introducing convolutional neural network and whole slide scanners in the commercial market, it is now essential to have thorough knowledge in this field to start diagnostic application of ANN.

Assessment of deep learning algorithms to predict histopathological diagnosis of breast cancer: first Moroccan prospective study on a private dataset

Objective

Breast cancer is a critical public health issue and a leading cause of cancer-related deaths among women worldwide. Its early diagnosis and detection can effectively help in increasing the chances of survival rate. For this reason, the diagnosis and classification of breast cancer using Deep learning algorithms have attracted a lot of attention. Therefore, our study aimed to design a computational approach based on deep convolutional neural networks for an efficient classification of breast cancer histopathological images by using our own created dataset. We collected overall 328 digital slides, from 116 of surgical breast specimens diagnosed with invasive breast carcinoma of non-specific type, and referred to the histopathology department of the National Institute of Oncology in Rabat, Morocco. We used two models of deep neural network architectures in order to accurately classify the images into one of three categories: normal tissue-benign lesions, in situ carcinoma or invasive carcinoma.

Results

Both Resnet50 and Xception models achieved comparable results, with a small advantage to Xception extracted features. We reported high degrees of overall correct classification accuracy (88%), and sensitivity (95%) for detection of carcinoma cases, which is important for diagnostic pathology workflow in order to assist pathologists for diagnosing breast cancer with precision. The results of the present study showed that the designed classification model has a good generalization performance in predicting diagnosis of breast cancer, in spite of the limited size of the data. To our knowledge, this approach can be highly compared with other common methods in the automated analysis of breast cancer images reported in literature.

A promising deep learning-assistive algorithm for histopathological screening of colorectal cancer

Colorectal cancer is one of the most common cancers worldwide, accounting for an annual estimated 1.8 million incident cases. With the increasing number of colonoscopies being performed, colorectal biopsies make up a large proportion of any histopathology laboratory workload. We trained and validated a unique artificial intelligence (AI) deep learning model as an assistive tool to screen for colonic malignancies in colorectal specimens, in order to improve cancer detection and classification; enabling busy pathologists to focus on higher order decision-making tasks. The study cohort consists of Whole Slide Images (WSI) obtained from 294 colorectal specimens. Qritive’s unique composite algorithm comprises both a deep learning model based on a Faster Region Based Convolutional Neural Network (Faster-RCNN) architecture for instance segmentation with a ResNet-101 feature extraction backbone that provides glandular segmentation, and a classical machine learning classifier. The initial training used pathologists’ annotations on a cohort of 66,191 image tiles extracted from 39 WSIs. A subsequent application of a classical machine learning-based slide classifier sorted the WSIs into ‘low risk’ (benign, inflammation) and ‘high risk’ (dysplasia, malignancy) categories. We further trained the composite AI-model’s performance on a larger cohort of 105 resections WSIs and then validated our findings on a cohort of 150 biopsies WSIs against the classifications of two independently blinded pathologists. We evaluated the area under the receiver-operator characteristic curve (AUC) and other performance metrics. The AI model achieved an AUC of 0.917 in the validation cohort, with excellent sensitivity (97.4%) in detection of high risk features of dysplasia and malignancy. We demonstrate an unique composite AI-model incorporating both a glandular segmentation deep learning model and a classical machine learning classifier, with excellent sensitivity in picking up high risk colorectal features. As such, AI plays a role as a potential screening tool in assisting busy pathologists by outlining the dysplastic and malignant glands.

Grading of adult diffuse gliomas according to the 2021 WHO Classification of Tumors of the Central Nervous System

The grading of gliomas based on histological features has been a subject of debate for several decades. A consensus has not yet been reached because of technical limitations and inter-observer variations. While the traditional grading system has failed to stratify the risk of IDH-mutant astrocytoma, canonical histological and proliferative markers may be applicable to the risk stratification of IDH-wild-type astrocytoma. Numerous studies have examined molecular markers in order to obtain more clinically relevant information that will improve the risk stratification of gliomas. The CDKN2A/B homozygous deletion for IDH-mutant astrocytoma and the following three criteria for IDH-wild-type astrocytoma: the concurrent gain of whole chromosome 7 and loss of whole chromosome 10, TERT promoter mutations, and EGFR amplification, were identified as independent molecular markers of the worst clinical outcomes. Therefore, the 2021 World Health Organization (WHO) Classification of Tumors of the Central Nervous System adopted these molecular markers into the revised grading criteria of IDH-mutant and -wild-type astrocytoma, respectively, as a grading system within tumor types. Of note, several recent studies have shown that some low-grade IDH-wild-type astrocytoma lacking both the molecular glioblastoma signature and genetic alterations typical of pediatric-type gliomas may demonstrate a relatively indolent clinical course, suggesting the existence of lower-grade adult IDH-wild-type astrocytoma. In terms of oligodendroglioma, IDH-mutant, and 1p/19q codeleted, consistent makers that predict poor outcomes have not yet been identified, and, thus, the current criteria have remained unchanged. Molecular testing to fulfill the revised WHO criteria is, however, not always available worldwide, and in that case, an integrated diagnosis combining all available complementary information is highly recommended. This review discusses controversial issues surrounding legacy grading systems and newly identified potential genetic markers of adult diffuse gliomas and provides perspectives on future grading systems.

Harnessing multimodal data integration to advance precision oncology

Advances in quantitative biomarker development have accelerated new forms of data-driven insights for patients with cancer. However, most approaches are limited to a single mode of data, leaving integrated approaches across modalities relatively underdeveloped. Multimodal integration of advanced molecular diagnostics, radiological and histological imaging, and codified clinical data presents opportunities to advance precision oncology beyond genomics and standard molecular techniques. However, most medical datasets are still too sparse to be useful for the training of modern machine learning techniques, and significant challenges remain before this is remedied. Combined efforts of data engineering, computational methods for analysis of heterogeneous data and instantiation of synergistic data models in biomedical research are required for success. In this Perspective, we offer our opinions on synthesizing complementary modalities of data with emerging multimodal artificial intelligence methods. Advancing along this direction will result in a reimagined class of multimodal biomarkers to propel the field of precision oncology in the coming decade.

Development of a semi-automated method for tumour budding assessment in colorectal cancer and comparison with manual methods

AIMS

Tumour budding (TB) is an established prognostic feature in multiple cancers but is not routinely assessed in pathology practice. Efforts to standardise and automate assessment have shifted from haematoxylin and eosin (H&E)-stained images towards cytokeratin immunohistochemistry. The aim of this study was to compare manual H&E and cytokeratin assessment methods with a semi-automated approach built within QuPath open-source software.

METHODS AND RESULTS

TB was assessed in cores from the advancing tumour edge in a cohort of stage II/III colon cancers (n = 186). The total numbers of buds detected with each method were as follows: manual H&E, n = 503; manual cytokeratin, n = 2290; and semi-automated, n = 5138. More than four times the number of buds were identified manually with cytokeratin assessment than with H&E assessment. One thousand seven hundred and thirty-four individual buds were identified with both manual and semi-automated assessments applied to cytokeratin images, representing 75.7% of the buds identified manually (n = 2290) and 33.7% of the buds detected with the semi-automated method (n = 5138). Higher semi-automated TB scores were due to any discrete area of cytokeratin immunopositivity within an accepted area range being identified as a bud, regardless of shape or crispness of definition, and to the inclusion of tumour cell clusters within glandular lumina ('luminal pseudobuds'). Although absolute numbers differed, semi-automated and manual bud counts were strongly correlated across cores (ρ = 0.81, P < 0.0001). All methods of TB assessment demonstrated poorer survival associated with higher TB scores.

CONCLUSIONS

We present a new QuPath-based approach to TB assessment, which compares favourably with established methods and offers a freely available, rapid and transparent tool that is also applicable to whole slide images.

Real-Time, Two-Color Stimulated Raman Scattering Imaging of Mouse Brain for Tissue Diagnosis

Stimulated Raman scattering (SRS) microscopy has emerged as a powerful optical imaging technique for tissue diagnosis. In recent years, two-color SRS has been shown to be able to provide hematoxylin and eosin (H&E)-equivalent images that allow fast and reliable diagnosis of brain cancer. Such capability has enabled exciting intraoperative cancer diagnosis applications. Two-color SRS imaging of tissue can be done with either a picosecond or femtosecond laser source. Femtosecond lasers have the advantage of enabling flexible imaging modes, including fast hyperspectral imaging and real-time, two-color SRS imaging. A spectral-focusing approach with chirped laser pulses is typically used with femtosecond lasers to achieve high spectral resolution. Two-color SRS acquisition can be realized with orthogonal modulation and lock-in detection. The complexity of pulse chirping, modulation, and characterization is a bottleneck for the widespread adoption of this method. This article provides a detailed protocol to demonstrate the implementation and optimization of spectral-focusing SRS and real-time, two-color imaging of mouse brain tissue in the epi-mode. This protocol can be used for a broad range of SRS imaging applications that leverage the high speed and spectroscopic imaging capability of SRS.

Development of Planar Illumination Strategies for Solving Mysteries in the Sub-Cellular Realm

Optical microscopy has vastly expanded the frontiers of structural and functional biology, due to the non-invasive probing of dynamic volumes in vivo. However, traditional widefield microscopy illuminating the entire field of view (FOV) is adversely affected by out-of-focus light scatter. Consequently, standard upright or inverted microscopes are inept in sampling diffraction-limited volumes smaller than the optical system's point spread function (PSF). Over the last few decades, several planar and structured (sinusoidal) illumination modalities have offered unprecedented access to sub-cellular organelles and 4D (3D + time) image acquisition. Furthermore, these optical sectioning systems remain unaffected by the size of biological samples, providing high signal-to-noise (SNR) ratios for objective lenses (OLs) with long working distances (WDs). This review aims to guide biologists regarding planar illumination strategies, capable of harnessing sub-micron spatial resolution with a millimeter depth of penetration.

Integrating artificial intelligence in pathology: a qualitative interview study of users' experiences and expectations

Recent progress in the development of artificial intelligence (AI) has sparked enthusiasm for its potential use in pathology. As pathology labs are currently starting to shift their focus towards AI implementation, a better understanding how AI tools can be optimally aligned with the medical and social context of pathology daily practice is urgently needed. Strikingly, studies often fail to mention the ways in which AI tools should be integrated in the decision-making processes of pathologists, nor do they address how this can be achieved in an ethically sound way. Moreover, the perspectives of pathologists and other professionals within pathology concerning the integration of AI within pathology remains an underreported topic. This article aims to fill this gap in the literature and presents the first in-depth interview study in which professionals' perspectives on the possibilities, conditions and prerequisites of AI integration in pathology are explicated. The results of this study have led to the formulation of three concrete recommendations to support AI integration, namely: (1) foster a pragmatic attitude toward AI development, (2) provide task-sensitive information and training to health care professionals working in pathology departments and (3) take time to reflect upon users' changing roles and responsibilities.