Deep learning in computational dermatopathology of melanoma: A technical systematic literature review

Artificial Intelligence in Dermatopathology: updates, strengths, and challenges

Artificial Intelligence (AI) has evolved to become a significant force in various domains, including medicine. We explore the role of AI in pathology, with a specific focus on dermatopathology and neoplastic dermatopathology. AI, encompassing Machine Learning (ML) and Deep Learning (DL), has demonstrated its potential in tasks ranging from diagnostic applications on Whole Slide Imaging (WSI) to predictive and prognostic functions in skin pathology. In dermatopathology, studies have assessed AI's ability to identify skin lesions, classify melanomas, and improve diagnostic accuracy. Results indicate that AI, particularly Convolutional Neural Networks (CNNs), can outperform human pathologists in terms of sensitivity and specificity. Moreover, AI aids in predicting disease outcomes, identifying aggressive tumors, and differentiating between various skin conditions. Neoplastic dermatopathology showcases AI's prowess in classifying melanocytic lesions, discriminating between melanomas and nevi, and aiding dermatopathologists in making accurate diagnoses. Studies emphasize the reproducibility and diagnostic aid that AI provides, especially in challenging cases. In inflammatory and lymphoproliferative dermatopathology, limited research exists, but studies show attempts to use AI to differentiate conditions like Mycosis Fungoides and eczema. While some results are promising, further exploration is needed in these areas. We highlight the extraordinary interest AI has garnered in the scientific community and its potential to assist clinicians and pathologists. Despite the advancements, we have stress edthe importance of collaboration between medical professionals, computer scientists, bioinformaticians, and engineers to harness AI's benefits while acknowledging its limitations and risks. The integration of AI into dermatopathology holds great promise, positioning it as a valuable tool rather than as a replacement for human expertise.

Integrating Spatial and Morphological Characteristics into Melanoma Prognosis: A Computational Approach

In this study, the prognostic value of cellular morphology and spatial configurations in melanoma has been examined, aiming to complement traditional prognostic indicators like mitotic activity and tumor thickness. Through a computational pipeline using machine learning and deep learning methods, we quantified nuclei sizes within different spatial regions and analyzed their prognostic significance using univariate and multivariate Cox models. Nuclei sizes in the invasive band demonstrated a significant hazard ratio (HR) of 1.1 (95% CI: 1.03, 1.18). Similarly, the nuclei sizes of tumor cells and Ki67 S100 co-positive cells in the invasive band achieved HRs of 1.07 (95% CI: 1.02, 1.13) and 1.09 (95% CI: 1.04, 1.16), respectively. Our findings reveal that nuclei sizes, particularly in the invasive band, are potentially prognostic factors. Correlation analyses further demonstrated a meaningful relationship between cellular morphology and tumor progression, notably showing that nuclei size within the invasive band correlates substantially with tumor thickness. These results suggest the potential of integrating spatial and morphological analyses into melanoma prognostication.

Algorithm-assisted diagnosis of Hirschsprung's disease - evaluation of robustness and comparative image analysis on data from various labs and slide scanners

BACKGROUND

Differences in the preparation, staining and scanning of digital pathology slides create significant pre-analytic variability. Algorithm-assisted tools must be able to contend with this variability in order to be applicable in clinical practice. In a previous study, a decision support algorithm was developed to assist in the diagnosis of Hirschsprung's disease. In the current study, we tested the robustness of this algorithm while assessing for pre-analytic factors which may affect its performance.

METHODS

The decision support algorithm was used on digital pathology slides obtained from four different medical centers (A-D) and scanned by three different scanner models (by Philips, Hamamatsu and 3DHISTECH). A total of 192 cases and 1782 slides were used in this study. RGB histograms were constructed to compare images from the various medical centers and scanner models and highlight the differences in color and contrast.

RESULTS

The algorithm was able to correctly identify ganglion cells in 99.2% of cases, from all medical centers (All scanned by the Philips slide scanner) as well as 95.5% and 100% of the slides scanned by the 3DHISTECH and Hamamatsu brand slide scanners, respectively. The total error rate for center D was lower than the other medical centers (3.9% vs 7.1%, 10.8% and 6% for centers A-C, respectively), the vast majority of errors being false positives (3.45% vs 0.45% false negatives). The other medical centers showed a higher rate of false negatives in relation to false positives (6.81% vs 0.29%, 9.8% vs 1.2% and 5.37% vs 0.63% for centers A-C, respectively). The total error rates for the Philips, Hamamatsu and 3DHISTECH brand scanners were 3.9%, 3.2% and 9.8%, respectively. RGB histograms demonstrated significant differences in pixel value distribution between the four medical centers, as well as between the 3DHISTECH brand scanner when compared to the Philips and Hamamatsu brand scanners.

CONCLUSIONS

The results reported in this paper suggest that the algorithm-based decision support system has sufficient robustness to be applicable for clinical practice. In addition, the novel method used in its development - Hierarchial-Contexual Analysis (HCA) may be applicable to the development of algorithm-assisted tools in other diseases, for which available datasets are limited. Validation of any given algorithm-assisted support system should nonetheless include data from as many medical centers and scanner models as possible.

A Systematic Comparison of Task Adaptation Techniques for Digital Histopathology

Due to an insufficient amount of image annotation, artificial intelligence in computational histopathology usually relies on fine-tuning pre-trained neural networks. While vanilla fine-tuning has shown to be effective, research on computer vision has recently proposed improved algorithms, promising better accuracy. While initial studies have demonstrated the benefits of these algorithms for medical AI, in particular for radiology, there is no empirical evidence for improved accuracy in histopathology. Therefore, based on the ConvNeXt architecture, our study performs a systematic comparison of nine task adaptation techniques, namely, DELTA, L2-SP, MARS-PGM, Bi-Tuning, BSS, MultiTune, SpotTune, Co-Tuning, and vanilla fine-tuning, on five histopathological classification tasks using eight datasets. The results are based on external testing and statistical validation and reveal a multifaceted picture: some techniques are better suited for histopathology than others, but depending on the classification task, a significant relative improvement in accuracy was observed for five advanced task adaptation techniques over the control method, i.e., vanilla fine-tuning (e.g., Co-Tuning: P(≫) = 0.942, d = 2.623). Furthermore, we studied the classification accuracy for three of the nine methods with respect to the training set size (e.g., Co-Tuning: P(≫) = 0.951, γ = 0.748). Overall, our results show that the performance of advanced task adaptation techniques in histopathology is affected by influencing factors such as the specific classification task or the size of the training dataset.

Annotation protocol and crowdsourcing multiple instance learning classification of skin histological images: The CR-AI4SkIN dataset

Digital Pathology (DP) has experienced a significant growth in recent years and has become an essential tool for diagnosing and prognosis of tumors. The availability of Whole Slide Images (WSIs) and the implementation of Deep Learning (DL) algorithms have paved the way for the appearance of Artificial Intelligence (AI) systems that support the diagnosis process. These systems require extensive and varied data for their training to be successful. However, creating labeled datasets in histopathology is laborious and time-consuming. We have developed a crowdsourcing-multiple instance labeling/learning protocol that is applied to the creation and use of the CR-AI4SkIN dataset.2 CR-AI4SkIN contains 271 WSIs of 7 Cutaneous Spindle Cell (CSC) neoplasms with expert and non-expert labels at region and WSI levels. It is the first dataset of these types of neoplasms made available. The regions selected by the experts are used to learn an automatic extractor of Regions of Interest (ROIs) from WSIs. To produce the embedding of each WSI, the representations of patches within the ROIs are obtained using a contrastive learning method, and then combined. Finally, they are fed to a Gaussian process-based crowdsourcing classifier, which utilizes the noisy non-expert WSI labels. We validate our crowdsourcing-multiple instance learning method in the CR-AI4SkIN dataset, addressing a binary classification problem (malign vs. benign). The proposed method obtains an F1 score of 0.7911 on the test set, outperforming three widely used aggregation methods for crowdsourcing tasks. Furthermore, our crowdsourcing method also outperforms the supervised model with expert labels on the test set (F1-score = 0.6035). The promising results support the proposed crowdsourcing multiple instance learning annotation protocol. It also validates the automatic extraction of interest regions and the use of contrastive embedding and Gaussian process classification to perform crowdsourcing classification tasks.

Mapping the landscape of artificial intelligence in skin cancer research: a bibliometric analysis

Objective

Artificial intelligence (AI), with its potential to diagnose skin cancer, has the potential to revolutionize future medical and dermatological practices. However, the current knowledge regarding the utilization of AI in skin cancer diagnosis remains somewhat limited, necessitating further research. This study employs visual bibliometric analysis to consolidate and present insights into the evolution and deployment of AI in the context of skin cancer. Through this analysis, we aim to shed light on the research developments, focal areas of interest, and emerging trends within AI and its application to skin cancer diagnosis.

Methods

On July 14, 2023, articles and reviews about the application of AI in skin cancer, spanning the years from 1900 to 2023, were selected from the Web of Science Core Collection. Co-authorship, co-citation, and co-occurrence analyses of countries, institutions, authors, references, and keywords within this field were conducted using a combination of tools, including CiteSpace V (version 6.2. R3), VOSviewer (version 1.6.18), SCImago, Microsoft Excel 2019, and R 4.2.3.

Results

A total of 512 papers matching the search terms and inclusion/exclusion criteria were published between 1991 and 2023. The United States leads in publications with 149, followed by India with 61. Germany holds eight positions among the top 10 institutions, while the United States has two. The most prevalent journals cited were Cancer, the European Journal of Cancer, and Sensors. The most frequently cited keywords include "skin cancer", "classification", "artificial intelligence", and "deep learning".

Conclusions

Research into the application of AI in skin cancer is rapidly expanding, and an increasing number of scholars are dedicating their efforts to this field. With the advancement of AI technology, new opportunities have arisen to enhance the accuracy of skin imaging diagnosis, treatment based on big data, and prognosis prediction. However, at present, the majority of AI research in the field of skin cancer diagnosis is still in the feasibility study stage. It has not yet made significant progress toward practical implementation in clinical settings. To make substantial strides in this field, there is a need to enhance collaboration between countries and institutions. Despite the potential benefits of AI in skin cancer research, numerous challenges remain to be addressed, including developing robust algorithms, resolving data quality issues, and enhancing results interpretability. Consequently, sustained efforts are essential to surmount these obstacles and facilitate the practical application of AI in skin cancer research.

Principles, applications, and future of artificial intelligence in dermatology

This paper provides an overview of artificial-intelligence (AI), as applied to dermatology. We focus our discussion on methodology, AI applications for various skin diseases, limitations, and future opportunities. We review how the current image-based models are being implemented in dermatology across disease subsets, and highlight the challenges facing widespread adoption. Additionally, we discuss how the future of AI in dermatology might evolve and the emerging paradigm of large language, and multi-modal models to emphasize the importance of developing responsible, fair, and equitable models in dermatology.

Learning how to detect: A deep reinforcement learning method for whole-slide melanoma histopathology images

Cutaneous melanoma represents one of the most life-threatening malignancies. Histopathological image analysis serves as a vital tool for early melanoma detection. Deep neural network (DNN) models are frequently employed to aid pathologists in enhancing the efficiency and accuracy of diagnoses. However, due to the paucity of well-annotated, high-resolution, whole-slide histopathology image (WSI) datasets, WSIs are typically fragmented into numerous patches during the model training and testing stages. This process disregards the inherent interconnectedness among patches, potentially impeding the models' performance. Additionally, the presence of excess, non-contributing patches extends processing times and introduces substantial computational burdens. To mitigate these issues, we draw inspiration from the clinical decision-making processes of dermatopathologists to propose an innovative, weakly supervised deep reinforcement learning framework, titled Fast medical decision-making in melanoma histopathology images (FastMDP-RL). This framework expedites model inference by reducing the number of irrelevant patches identified within WSIs. FastMDP-RL integrates two DNN-based agents: the search agent (SeAgent) and the decision agent (DeAgent). The SeAgent initiates actions, steered by the image features observed in the current viewing field at various magnifications. Simultaneously, the DeAgent provides labeling probabilities for each patch. We utilize multi-instance learning (MIL) to construct a teacher-guided model (MILTG), serving a dual purpose: rewarding the SeAgent and guiding the DeAgent. Our evaluations were conducted using two melanoma datasets: the publicly accessible TCIA-CM dataset and the proprietary MELSC dataset. Our experimental findings affirm FastMDP-RL's ability to expedite inference and accurately predict WSIs, even in the absence of pixel-level annotations. Moreover, our research investigates the WSI-based interactive environment, encompassing the design of agents, state and reward functions, and feature extractors suitable for melanoma tissue images. This investigation offers valuable insights and references for researchers engaged in related studies. The code is available at: https://github.com/titizheng/FastMDP-RL.