Deep learning and pathomics analyses reveal cell nuclei as important features for mutation prediction of BRAF-mutated melanomas

MetFinder: A Tool for Automated Quantitation of Metastatic Burden in Histological Sections From Preclinical Models

As efforts to study the mechanisms of melanoma metastasis and novel therapeutic approaches multiply, researchers need accurate, high-throughput methods to evaluate the effects on tumor burden resulting from specific interventions. We show that automated quantification of tumor content from whole slide images is a compelling solution to assess in vivo experiments. In order to increase the outflow of data collection from preclinical studies, we assembled a large dataset with annotations and trained a deep neural network for the quantitative analysis of melanoma tumor content on histopathological sections of murine models. After assessing its performance in segmenting these images, the tool obtained consistent results with an orthogonal method (bioluminescence) of measuring metastasis in an experimental setting. This AI-based algorithm, made freely available to academic laboratories through a web-interface called MetFinder, promises to become an asset for melanoma researchers and pathologists interested in accurate, quantitative assessment of metastasis burden.

Immune subtyping of melanoma whole slide images using multiple instance learning

Determining early-stage prognostic markers and stratifying patients for effective treatment are two key challenges for improving outcomes for melanoma patients. Previous studies have used tumour transcriptome data to stratify patients into immune subgroups, which were associated with differential melanoma specific survival and potential predictive biomarkers. However, acquiring transcriptome data is a time-consuming and costly process. Moreover, it is not routinely used in the current clinical workflow. Here, we attempt to overcome this by developing deep learning models to classify gigapixel haematoxylin and eosin (H&E) stained pathology slides, which are well established in clinical workflows, into these immune subgroups. We systematically assess six different multiple instance learning (MIL) frameworks, using five different image resolutions and three different feature extraction methods. We show that pathology-specific self-supervised models using 10x resolution patches generate superior representations for the classification of immune subtypes. In addition, in a primary melanoma dataset, we achieve a mean area under the receiver operating characteristic curve (AUC) of 0.80 for classifying histopathology images into 'high' or 'low immune' subgroups and a mean AUC of 0.82 in an independent TCGA melanoma dataset. Furthermore, we show that these models are able to stratify patients into 'high' and 'low immune' subgroups with significantly different melanoma specific survival outcomes (log rank test, P< 0.005). We anticipate that MIL methods will allow us to find new biomarkers of high importance, act as a tool for clinicians to infer the immune landscape of tumours and stratify patients, without needing to carry out additional expensive genetic tests.

The role of artificial intelligence and convolutional neural networks in the management of melanoma: a clinical, pathological, and radiological perspective

Clinical dermatoscopy and pathological slide assessment are essential in the diagnosis and management of patients with cutaneous melanoma. For those presenting with stage IIC disease and beyond, radiological investigations are often considered. The dermatoscopic, whole slide and radiological images used during clinical care are often stored digitally, enabling artificial intelligence (AI) and convolutional neural networks (CNN) to learn, analyse and contribute to the clinical decision-making. A keyword search of the Medline database was performed to assess the progression, capabilities and limitations of AI and CNN and its use in diagnosis and management of cutaneous melanoma. Full-text articles were reviewed if they related to dermatoscopy, pathological slide assessment or radiology. Through analysis of 95 studies, we demonstrate that diagnostic accuracy of AI/CNN can be superior (or at least equal) to clinicians. However, variability in image acquisition, pre-processing, segmentation, and feature extraction remains challenging. With current technological abilities, AI/CNN and clinicians synergistically working together are better than one another in all subspecialty domains relating to cutaneous melanoma. AI has the potential to enhance the diagnostic capabilities of junior dermatology trainees, primary care skin cancer clinicians and general practitioners. For experienced clinicians, AI provides a cost-efficient second opinion. From a pathological and radiological perspective, CNN has the potential to improve workflow efficiency, allowing clinicians to achieve more in a finite amount of time. Until the challenges of AI/CNN are reliably met, however, they can only remain an adjunct to clinical decision-making.

Review of Molecular Technologies for Investigating Canine Cancer

Genetic molecular testing is starting to gain traction as part of standard clinical practice for dogs with cancer due to its multi-faceted benefits, such as potentially being able to provide diagnostic, prognostic and/or therapeutic information. However, the benefits and ultimate success of genomic analysis in the clinical setting are reliant on the robustness of the tools used to generate the results, which continually expand as new technologies are developed. To this end, we review the different materials from which tumour cells, DNA, RNA and the relevant proteins can be isolated and what methods are available for interrogating their molecular profile, including analysis of the genetic alterations (both somatic and germline), transcriptional changes and epigenetic modifications (including DNA methylation/acetylation and microRNAs). We also look to the future and the tools that are currently being developed, such as using artificial intelligence (AI) to identify genetic mutations from histomorphological criteria. In summary, we find that the molecular genetic characterisation of canine neoplasms has made a promising start. As we understand more of the genetics underlying these tumours and more targeted therapies become available, it will no doubt become a mainstay in the delivery of precision veterinary care to dogs with cancer.

Self-supervised artificial intelligence predicts recurrence, metastasis and disease specific death from primary cutaneous squamous cell carcinoma at diagnosis

Primary cutaneous squamous cell carcinoma (cSCC) is responsible for ~10,000 deaths annually in the United States. Stratification of risk of poor outcome (PO) including recurrence, metastasis and disease specific death (DSD) at initial biopsy would significantly impact clinical decision-making during the initial post operative period where intervention has been shown to be most effective. In this multi-institutional study, we developed a state-of-the-art self-supervised deep-learning approach with interpretability power and demonstrated its ability to predict poor outcomes of cSCCs at the time of initial biopsy. By highlighting histomorphological phenotypes, our approach demonstrates that poor differentiation and deep invasion correlate with poor prognosis. Our approach is particularly efficient at defining poor outcome risk in Brigham and Women's Hospital (BWH) T2a and American Joint Committee on Cancer (AJCC) T2 cSCCs. This bridges a significant gap in our ability to assess risk among T2a/T2 cSCCs and may be useful in defining patients at highest risk of poor outcome at the time of diagnosis. Early identification of highest-risk patients could signal implementation of more stringent surveillance, rigorous diagnostic work up and identify patients who might best respond to early postoperative adjunctive treatment.

An update on methods for detection of prognostic and predictive biomarkers in melanoma

The approval of immunotherapy for stage II-IV melanoma has underscored the need for improved immune-based predictive and prognostic biomarkers. For resectable stage II-III patients, adjuvant immunotherapy has proven clinical benefit, yet many patients experience significant adverse events and may not require therapy. In the metastatic setting, single agent immunotherapy cures many patients but, in some cases, more intensive combination therapies against specific molecular targets are required. Therefore, the establishment of additional biomarkers to determine a patient's disease outcome (i.e., prognostic) or response to treatment (i.e., predictive) is of utmost importance. Multiple methods ranging from gene expression profiling of bulk tissue, to spatial transcriptomics of single cells and artificial intelligence-based image analysis have been utilized to better characterize the immune microenvironment in melanoma to provide novel predictive and prognostic biomarkers. In this review, we will highlight the different techniques currently under investigation for the detection of prognostic and predictive immune biomarkers in melanoma.

Deep learning in computational dermatopathology of melanoma: A technical systematic literature review

Deep learning (DL) has become one of the major approaches in computational dermatopathology, evidenced by a significant increase in this topic in the current literature. We aim to provide a structured and comprehensive overview of peer-reviewed publications on DL applied to dermatopathology focused on melanoma. In comparison to well-published DL methods on non-medical images (e.g., classification on ImageNet), this field of application comprises a specific set of challenges, such as staining artifacts, large gigapixel images, and various magnification levels. Thus, we are particularly interested in the pathology-specific technical state-of-the-art. We also aim to summarize the best performances achieved thus far with respect to accuracy, along with an overview of self-reported limitations. Accordingly, we conducted a systematic literature review of peer-reviewed journal and conference articles published between 2012 and 2022 in the databases ACM Digital Library, Embase, IEEE Xplore, PubMed, and Scopus, expanded by forward and backward searches to identify 495 potentially eligible studies. After screening for relevance and quality, a total of 54 studies were included. We qualitatively summarized and analyzed these studies from technical, problem-oriented, and task-oriented perspectives. Our findings suggest that the technical aspects of DL for histopathology in melanoma can be further improved. The DL methodology was adopted later in this field, and still lacks the wider adoption of DL methods already shown to be effective for other applications. We also discuss upcoming trends toward ImageNet-based feature extraction and larger models. While DL has achieved human-competitive accuracy in routine pathological tasks, its performance on advanced tasks is still inferior to wet-lab testing (for example). Finally, we discuss the challenges impeding the translation of DL methods to clinical practice and provide insight into future research directions.

Artificial Intelligence to Predict the BRAF V595E Mutation in Canine Urinary Bladder Urothelial Carcinomas

In dogs, the BRAF mutation (V595E) is common in bladder and prostate cancer and represents a specific diagnostic marker. Recent advantages in artificial intelligence (AI) offer new opportunities in the field of tumour marker detection. While AI histology studies have been conducted in humans to detect BRAF mutation in cancer, comparable studies in animals are lacking. In this study, we used commercially available AI histology software to predict BRAF mutation in whole slide images (WSI) of bladder urothelial carcinomas (UC) stained with haematoxylin and eosin (HE), based on a training (n = 81) and a validation set (n = 96). Among 96 WSI, 57 showed identical PCR and AI-based BRAF predictions, resulting in a sensitivity of 58% and a specificity of 63%. The sensitivity increased substantially to 89% when excluding small or poor-quality tissue sections. Test reliability depended on tumour differentiation (p < 0.01), presence of inflammation (p < 0.01), slide quality (p < 0.02) and sample size (p < 0.02). Based on a small subset of cases with available adjacent non-neoplastic urothelium, AI was able to distinguish malignant from benign epithelium. This is the first study to demonstrate the use of AI histology to predict BRAF mutation status in canine UC. Despite certain limitations, the results highlight the potential of AI in predicting molecular alterations in routine tissue sections.

Artificial intelligence in digital pathology of cutaneous lymphomas: a review of the current state and future perspectives

Primary cutaneous lymphomas (CLs) represent a heterogeneous group of T-cell lymphomas and B-cell lymphomas that present in the skin without evidence of extracutaneous involvement at time of diagnosis. CLs are largely distinct from their systemic counterparts in clinical presentation, histopathology, and biological behavior and, therefore, require different therapeutic management. Additional diagnostic burden is added by the fact that several benign inflammatory dermatoses mimic CL subtypes, requiring clinicopathological correlation for definitive diagnosis. Due to the heterogeneity and rarity of CL, adjunct diagnostic tools are welcomed, especially by pathologists without expertise in this field or with limited access to a centralized specialist panel. The transition into digital pathology workflows enables artificial intelligence (AI)-based analysis of patients' whole-slide pathology images (WSIs). AI can be used to automate manual processes in histopathology but, more importantly, can be applied to complex diagnostic tasks, especially suitable for rare disease like CL. To date, AI-based applications for CL have been minimally explored in literature. However, in other skin cancers and systemic lymphomas, disciplines that are recognized here as the building blocks for CLs, several studies demonstrated promising results using AI for disease diagnosis and subclassification, cancer detection, specimen triaging, and outcome prediction. Additionally, AI allows discovery of novel biomarkers or may help to quantify established biomarkers. This review summarizes and blends applications of AI in pathology of skin cancer and lymphoma and proposes how these findings can be applied to diagnostics of CL.

Recent Advances in Melanoma Diagnosis and Prognosis Using Machine Learning Methods

PURPOSE OF REVIEW

The purpose was to summarize the current role and state of artificial intelligence and machine learning in the diagnosis and management of melanoma.

RECENT FINDINGS

Deep learning algorithms can identify melanoma from clinical, dermoscopic, and whole slide pathology images with increasing accuracy. Efforts to provide more granular annotation to datasets and to identify new predictors are ongoing. There have been many incremental advances in both melanoma diagnostics and prognostic tools using artificial intelligence and machine learning. Higher quality input data will further improve these models' capabilities.

Multimodal integration of image, epigenetic and clinical data to predict BRAF mutation status in melanoma

BACKGROUND

In machine learning, multimodal classifiers can provide more generalised performance than unimodal classifiers. In clinical practice, physicians usually also rely on a range of information from different examinations for diagnosis. In this study, we used BRAF mutation status prediction in melanoma as a model system to analyse the contribution of different data types in a combined classifier because BRAF status can be determined accurately by sequencing as the current gold standard, thus nearly eliminating label noise.

METHODS

We trained a deep learning-based classifier by combining individually trained random forests of image, clinical and methylation data to predict BRAF-V600 mutation status in primary and metastatic melanomas of The Cancer Genome Atlas cohort.

RESULTS

With our multimodal approach, we achieved an area under the receiver operating characteristic curve of 0.80, whereas the individual classifiers yielded areas under the receiver operating characteristic curve of 0.63 (histopathologic image data), 0.66 (clinical data) and 0.66 (methylation data) on an independent data set.

CONCLUSIONS

Our combined approach can predict BRAF status to some extent by identifying BRAF-V600 specific patterns at the histologic, clinical and epigenetic levels. The multimodal classifiers have improved generalisability in predicting BRAF mutation status.

Facts and Hopes on the Use of Artificial Intelligence for Predictive Immunotherapy Biomarkers in Cancer

Immunotherapy by immune checkpoint inhibitors has become a standard treatment strategy for many types of solid tumors. However, the majority of patients with cancer will not respond, and predicting response to this therapy is still a challenge. Artificial intelligence (AI) methods can extract meaningful information from complex data, such as image data. In clinical routine, radiology or histopathology images are ubiquitously available. AI has been used to predict the response to immunotherapy from radiology or histopathology images, either directly or indirectly via surrogate markers. While none of these methods are currently used in clinical routine, academic and commercial developments are pointing toward potential clinical adoption in the near future. Here, we summarize the state of the art in AI-based image biomarkers for immunotherapy response based on radiology and histopathology images. We point out limitations, caveats, and pitfalls, including biases, generalizability, and explainability, which are relevant for researchers and health care providers alike, and outline key clinical use cases of this new class of predictive biomarkers.

Deep Learning for Skin Melanocytic Tumors in Whole-Slide Images: A Systematic Review

The rise of Artificial Intelligence (AI) has shown promising performance as a support tool in clinical pathology workflows. In addition to the well-known interobserver variability between dermatopathologists, melanomas present a significant challenge in their histological interpretation. This study aims to analyze all previously published studies on whole-slide images of melanocytic tumors that rely on deep learning techniques for automatic image analysis. Embase, Pubmed, Web of Science, and Virtual Health Library were used to search for relevant studies for the systematic review, in accordance with the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) checklist. Articles from 2015 to July 2022 were included, with an emphasis placed on the used artificial intelligence methods. Twenty-eight studies that fulfilled the inclusion criteria were grouped into four groups based on their clinical objectives, including pathologists versus deep learning models (n = 10), diagnostic prediction (n = 7); prognosis (n = 5), and histological features (n = 6). These were then analyzed to draw conclusions on the general parameters and conditions of AI in pathology, as well as the necessary factors for better performance in real scenarios.

Diagnostic and Prognostic Deep Learning Applications for Histological Assessment of Cutaneous Melanoma

Melanoma is among the most devastating human malignancies. Accurate diagnosis and prognosis are essential to offer optimal treatment. Histopathology is the gold standard for establishing melanoma diagnosis and prognostic features. However, discrepancies often exist between pathologists, and analysis is costly and time-consuming. Deep-learning algorithms are deployed to improve melanoma diagnosis and prognostication from histological images of melanoma. In recent years, the development of these machine-learning tools has accelerated, and machine learning is poised to become a clinical tool to aid melanoma histology. Nevertheless, a review of the advances in machine learning in melanoma histology was lacking. We performed a comprehensive literature search to provide a complete overview of the recent advances in machine learning in the assessment of melanoma based on hematoxylin eosin digital pathology images. In our work, we review 37 recent publications, compare the methods and performance of the reviewed studies, and highlight the variety of promising machine-learning applications in melanoma histology.

Computational pathology in ovarian cancer

Histopathologic evaluations of tissue sections are key to diagnosing and managing ovarian cancer. Pathologists empirically assess and integrate visual information, such as cellular density, nuclear atypia, mitotic figures, architectural growth patterns, and higher-order patterns, to determine the tumor type and grade, which guides oncologists in selecting appropriate treatment options. Latent data embedded in pathology slides can be extracted using computational imaging. Computers can analyze digital slide images to simultaneously quantify thousands of features, some of which are visible with a manual microscope, such as nuclear size and shape, while others, such as entropy, eccentricity, and fractal dimensions, are quantitatively beyond the grasp of the human mind. Applications of artificial intelligence and machine learning tools to interpret digital image data provide new opportunities to explore and quantify the spatial organization of tissues, cells, and subcellular structures. In comparison to genomic, epigenomic, transcriptomic, and proteomic patterns, morphologic and spatial patterns are expected to be more informative as quantitative biomarkers of complex and dynamic tumor biology. As computational pathology is not limited to visual data, nuanced subvisual alterations that occur in the seemingly “normal” pre-cancer microenvironment could facilitate research in early cancer detection and prevention. Currently, efforts to maximize the utility of computational pathology are focused on integrating image data with other -omics platforms that lack spatial information, thereby providing a new way to relate the molecular, spatial, and microenvironmental characteristics of cancer. Despite a dire need for improvements in ovarian cancer prevention, early detection, and treatment, the ovarian cancer field has lagged behind other cancers in the application of computational pathology. The intent of this review is to encourage ovarian cancer research teams to apply existing and/or develop additional tools in computational pathology for ovarian cancer and actively contribute to advancing this important field.

When blockchain meets artificial intelligence: An application to cancer histopathology

A recent study by Saldanha et al. demonstrates that blockchain-based models outcompeted local models and performed similarly with merged models to predict molecular features from cancer histopathology images. The results reveal the capability of decentralized models in molecular diagnosis of cancer.

Pediatric Sarcomas: The Next Generation of Molecular Studies

Pediatric sarcomas constitute one of the largest groups of childhood cancers, following hematopoietic, neural, and renal lesions. Partly because of their diversity, they continue to offer challenges in diagnosis and treatment. In spite of the diagnostic, nosologic, and therapeutic gains made with genetic technology, newer means for investigation are needed. This article reviews emerging technology being used to study human neoplasia and how these methods might be applicable to pediatric sarcomas. Methods reviewed include single cell RNA sequencing (scRNAseq), spatial multi-omics, high-throughput functional genomics, and clustered regularly interspersed short palindromic sequence-Cas9 (CRISPR-Cas9) technology. In spite of these advances, the field continues to be challenged by a dearth of properly annotated materials, particularly from recurrences and metastases and pre- and post-treatment samples.