Semi-Supervised Nests of Melanocytes Segmentation Method Using Convolutional Autoencoders

Utilizing deep learning model for assessing melanocytic density in resection margins of lentigo maligna

BACKGROUND

Surgical excision with clear histopathological margins is the preferred treatment to prevent progression of lentigo maligna (LM) to invasive melanoma. However, the assessment of resection margins on sun-damaged skin is challenging. We developed a deep learning model for detection of melanocytes in resection margins of LM.

METHODS

In total, 353 whole slide images (WSIs) were included. 295 WSIs were used for training and 58 for validation and testing. The algorithm was trained with 3,973 manual pixel-wise annotations. The AI analyses were compared to those of three blinded dermatopathologists and two pathology residents, who performed their evaluations without AI and AI-assisted. Immunohistochemistry (SOX10) served as the reference standard. We used a dichotomized cutoff for low and high risk of recurrence (≤ 25 melanocytes in an area of 0.5 mm for low risk and > 25 for high risk).

RESULTS

The AI model achieved an area under the receiver operating characteristic curve (AUC) of 0.84 in discriminating margins with low and high recurrence risk. In comparison, the AUC for dermatopathologists ranged from 0.72 to 0.90 and for the residents in pathology, 0.68 to 0.80. Additionally, with aid of the AI model the performance of two pathologists significantly improved.

CONCLUSIONS

The deep learning showed notable accuracy in detecting resection margins of LM with a high versus low risk of recurrence. Furthermore, the use of AI improved the performance of 2/5 pathologists. This automated tool could aid pathologists in the assessment or pre-screening of LM margins.

Exploring Radiomics Features Based on H&E Images as Potential Biomarkers for Evaluating Muscle Atrophy: A Preliminary Study

Radiomics features have been widely used as novel biomarkers in the diagnosis of various diseases, but whether radiomics features derived from hematoxylin and eosin (H&E) images can evaluate muscle atrophy has not been studied. Therefore, this study aims to establish a new biomarker based on H&E images using radiomics methods to quantitatively analyze H&E images, which is crucial for improving the accuracy of muscle atrophy assessment. Firstly, a weightless muscle atrophy model was established by laying macaques in bed, and H&E images of the shank muscle fibers of the control and bed rest (BR) macaques were collected. Muscle fibers were accurately segmented by designing a semi-supervised segmentation framework based on contrastive learning. Then, 77 radiomics features were extracted from the segmented muscle fibers, and a stable subset of features was selected through the LASSO method. Finally, the correlation between radiomics features and muscle atrophy was analyzed using a support vector machine (SVM) classifier. The semi-supervised segmentation results show that the proposed method had an average Spearman's and intra-class correlation coefficient (ICC) of 88% and 86% compared to manually extracted features, respectively. Radiomics analysis showed that the AUC of the muscle atrophy evaluation model based on H&E images was 96.87%. For individual features, GLSZM_SZE outperformed other features in terms of AUC (91.5%) and ACC (84.4%). In summary, the feature extraction based on the semi-supervised segmentation method is feasible and reliable for subsequent radiomics research. Texture features have greater advantages in evaluating muscle atrophy compared to other features. This study provides important biomarkers for accurate diagnosis of muscle atrophy.

Deep learning in computational dermatopathology of melanoma: A technical systematic literature review

Deep learning (DL) has become one of the major approaches in computational dermatopathology, evidenced by a significant increase in this topic in the current literature. We aim to provide a structured and comprehensive overview of peer-reviewed publications on DL applied to dermatopathology focused on melanoma. In comparison to well-published DL methods on non-medical images (e.g., classification on ImageNet), this field of application comprises a specific set of challenges, such as staining artifacts, large gigapixel images, and various magnification levels. Thus, we are particularly interested in the pathology-specific technical state-of-the-art. We also aim to summarize the best performances achieved thus far with respect to accuracy, along with an overview of self-reported limitations. Accordingly, we conducted a systematic literature review of peer-reviewed journal and conference articles published between 2012 and 2022 in the databases ACM Digital Library, Embase, IEEE Xplore, PubMed, and Scopus, expanded by forward and backward searches to identify 495 potentially eligible studies. After screening for relevance and quality, a total of 54 studies were included. We qualitatively summarized and analyzed these studies from technical, problem-oriented, and task-oriented perspectives. Our findings suggest that the technical aspects of DL for histopathology in melanoma can be further improved. The DL methodology was adopted later in this field, and still lacks the wider adoption of DL methods already shown to be effective for other applications. We also discuss upcoming trends toward ImageNet-based feature extraction and larger models. While DL has achieved human-competitive accuracy in routine pathological tasks, its performance on advanced tasks is still inferior to wet-lab testing (for example). Finally, we discuss the challenges impeding the translation of DL methods to clinical practice and provide insight into future research directions.

Deep Learning for Skin Melanocytic Tumors in Whole-Slide Images: A Systematic Review

The rise of Artificial Intelligence (AI) has shown promising performance as a support tool in clinical pathology workflows. In addition to the well-known interobserver variability between dermatopathologists, melanomas present a significant challenge in their histological interpretation. This study aims to analyze all previously published studies on whole-slide images of melanocytic tumors that rely on deep learning techniques for automatic image analysis. Embase, Pubmed, Web of Science, and Virtual Health Library were used to search for relevant studies for the systematic review, in accordance with the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) checklist. Articles from 2015 to July 2022 were included, with an emphasis placed on the used artificial intelligence methods. Twenty-eight studies that fulfilled the inclusion criteria were grouped into four groups based on their clinical objectives, including pathologists versus deep learning models (n = 10), diagnostic prediction (n = 7); prognosis (n = 5), and histological features (n = 6). These were then analyzed to draw conclusions on the general parameters and conditions of AI in pathology, as well as the necessary factors for better performance in real scenarios.

Diagnostic and Prognostic Deep Learning Applications for Histological Assessment of Cutaneous Melanoma

Melanoma is among the most devastating human malignancies. Accurate diagnosis and prognosis are essential to offer optimal treatment. Histopathology is the gold standard for establishing melanoma diagnosis and prognostic features. However, discrepancies often exist between pathologists, and analysis is costly and time-consuming. Deep-learning algorithms are deployed to improve melanoma diagnosis and prognostication from histological images of melanoma. In recent years, the development of these machine-learning tools has accelerated, and machine learning is poised to become a clinical tool to aid melanoma histology. Nevertheless, a review of the advances in machine learning in melanoma histology was lacking. We performed a comprehensive literature search to provide a complete overview of the recent advances in machine learning in the assessment of melanoma based on hematoxylin eosin digital pathology images. In our work, we review 37 recent publications, compare the methods and performance of the reviewed studies, and highlight the variety of promising machine-learning applications in melanoma histology.

Automatic Malignant and Benign Skin Cancer Classification Using a Hybrid Deep Learning Approach

Skin cancer is one of the major types of cancer with an increasing incidence in recent decades. The source of skin cancer arises in various dermatologic disorders. Skin cancer is classified into various types based on texture, color, morphological features, and structure. The conventional approach for skin cancer identification needs time and money for the predicted results. Currently, medical science is utilizing various tools based on digital technology for the classification of skin cancer. The machine learning-based classification approach is the robust and dominant approach for automatic methods of classifying skin cancer. The various existing and proposed methods of deep neural network, support vector machine (SVM), neural network (NN), random forest (RF), and K-nearest neighbor are used for malignant and benign skin cancer identification. In this study, a method was proposed based on the stacking of classifiers with three folds towards the classification of melanoma and benign skin cancers. The system was trained with 1000 skin images with the categories of melanoma and benign. The training and testing were performed using 70 and 30 percent of the overall data set, respectively. The primary feature extraction was conducted using the Resnet50, Xception, and VGG16 methods. The accuracy, F1 scores, AUC, and sensitivity metrics were used for the overall performance evaluation. In the proposed Stacked CV method, the system was trained in three levels by deep learning, SVM, RF, NN, KNN, and logistic regression methods. The proposed method for Xception techniques of feature extraction achieved 90.9% accuracy and was stronger compared to ResNet50 and VGG 16 methods. The improvement and optimization of the proposed method with a large training dataset could provide a reliable and robust skin cancer classification system.

A review: The detection of cancer cells in histopathology based on machine vision

Machine vision is being employed in defect detection, size measurement, pattern recognition, image fusion, target tracking and 3D reconstruction. Traditional cancer detection methods are dominated by manual detection, which wastes time and manpower, and heavily relies on the pathologists' skill and work experience. Therefore, these manual detection approaches are not convenient for the inheritance of domain knowledge, and are not suitable for the rapid development of medical care in the future. The emergence of machine vision can iteratively update and learn the domain knowledge of cancer cell pathology detection to achieve automated, high-precision, and consistent detection. Consequently, this paper reviews the use of machine vision to detect cancer cells in histopathology images, as well as the benefits and drawbacks of various detection approaches. First, we review the application of image preprocessing and image segmentation in histopathology for the detection of cancer cells, and compare the benefits and drawbacks of different algorithms. Secondly, for the characteristics of histopathological cancer cell images, the research progress of shape, color and texture features and other methods is mainly reviewed. Furthermore, for the classification methods of histopathological cancer cell images, the benefits and drawbacks of traditional machine vision approaches and deep learning methods are compared and analyzed. Finally, the above research is discussed and forecasted, with the expected future development tendency serving as a guide for future research.

A Convolutional Autoencoder Topology for Classification in High-Dimensional Noisy Image Datasets

Deep convolutional neural networks have shown remarkable performance in the image classification domain. However, Deep Learning models are vulnerable to noise and redundant information encapsulated into the high-dimensional raw input images, leading to unstable and unreliable predictions. Autoencoders constitute an unsupervised dimensionality reduction technique, proven to filter out noise and redundant information and create robust and stable feature representations. In this work, in order to resolve the problem of DL models' vulnerability, we propose a convolutional autoencoder topological model for compressing and filtering out noise and redundant information from initial high dimensionality input images and then feeding this compressed output into convolutional neural networks. Our results reveal the efficiency of the proposed approach, leading to a significant performance improvement compared to Deep Learning models trained with the initial raw images.

Tertiary lymphoid structures (TLS) identification and density assessment on H&E-stained digital slides of lung cancer

Tertiary lymphoid structures (TLS) are ectopic aggregates of lymphoid cells in inflamed, infected, or tumoral tissues that are easily recognized on an H&E histology slide as discrete entities, distinct from lymphocytes. TLS are associated with improved cancer prognosis but there is no standardised method available to quantify their presence. Previous studies have used immunohistochemistry to determine the presence of specific cells as a marker of the TLS. This has now been proven to be an underestimate of the true number of TLS. Thus, we propose a methodology for the automated identification and quantification of TLS, based on H&E slides. We subsequently determined the mathematical criteria defining a TLS. TLS regions were identified through a deep convolutional neural network and segmentation of lymphocytes was performed through an ellipsoidal model. This methodology had a 92.87% specificity at 95% sensitivity, 88.79% specificity at 98% sensitivity and 84.32% specificity at 99% sensitivity level based on 144 TLS annotated H&E slides implying that the automated approach was able to reproduce the histopathologists’ assessment with great accuracy. We showed that the minimum number of lymphocytes within TLS is 45 and the minimum TLS area is 6,245μm². Furthermore, we have shown that the density of the lymphocytes is more than 3 times those outside of the TLS. The mean density and standard deviation of lymphocytes within a TLS area are 0.0128/μm² and 0.0026/μm² respectively compared to 0.004/μm² and 0.001/μm² in non-TLS regions. The proposed methodology shows great potential for automated identification and quantification of the TLS density on digital H&E slides.

Towards Accurate Diagnosis of Skin Lesions Using Feedforward Back Propagation Neural Networks

In the automatic detection framework, there have been many attempts to develop models for real-time melanoma detection. To effectively discriminate benign and malign skin lesions, this work investigates sixty different architectures of the Feedforward Back Propagation Network (FFBPN), based on shape asymmetry for an optimal structural design that includes both the hidden neuron number and the input data selection. The reason for the choice of shape asymmetry was based on the 5–10% disagreement between dermatologists regarding the efficacy of asymmetry in the diagnosis of malignant melanoma. Asymmetry is quantified based on lesion shape (contour), moment of inertia of the lesion shape and histograms. The FFBPN has a high architecture flexibility, which indicates it as a favorable tool to avoid the over-parameterization of the ANN and, equally, to discard those redundant input datasets that usually result in poor test performance. The FFBPN was tested on four public image datasets containing melanoma, dysplastic nevus and nevus images. Experimental results on multiple benchmark data sets demonstrate that asymmetry A2 is a meaningful feature for skin lesion classification, and FFBPN with 16 neurons in the hidden layer can model the data without compromising prediction accuracy.

Multiplexed Immunohistochemistry and Digital Pathology as the Foundation for Next-Generation Pathology in Melanoma: Methodological Comparison and Future Clinical Applications

The state-of-the-art for melanoma treatment has recently witnessed an enormous revolution, evolving from a chemotherapeutic, "one-drug-for-all" approach, to a tailored molecular- and immunological-based approach with the potential to make personalized therapy a reality. Nevertheless, methods still have to improve a lot before these can reliably characterize all the tumoral features that make each patient unique. While the clinical introduction of next-generation sequencing has made it possible to match mutational profiles to specific targeted therapies, improving response rates to immunotherapy will similarly require a deep understanding of the immune microenvironment and the specific contribution of each component in a patient-specific way. Recent advancements in artificial intelligence and single-cell profiling of resected tumor samples are paving the way for this challenging task. In this review, we provide an overview of the state-of-the-art in artificial intelligence and multiplexed immunohistochemistry in pathology, and how these bear the potential to improve diagnostics and therapy matching in melanoma. A major asset of in-situ single-cell profiling methods is that these preserve the spatial distribution of the cells in the tissue, allowing researchers to not only determine the cellular composition of the tumoral microenvironment, but also study tissue sociology, making inferences about specific cell-cell interactions and visualizing distinctive cellular architectures - all features that have an impact on anti-tumoral response rates. Despite the many advantages, the introduction of these approaches requires the digitization of tissue slides and the development of standardized analysis pipelines which pose substantial challenges that need to be addressed before these can enter clinical routine.

Automatic Detection Method for Cancer Cell Nucleus Image Based on Deep-Learning Analysis and Color Layer Signature Analysis Algorithm

Exploring strategies to treat cancer has always been an aim of medical researchers. One of the available strategies is to use targeted therapy drugs to make the chromosomes in cancer cells unstable such that cell death can be induced, and the elimination of highly proliferative cancer cells can be achieved. Studies have reported that the mitotic defects and micronuclei in cancer cells can be used as biomarkers to evaluate the instability of the chromosomes. Researchers use these two biomarkers to assess the effects of drugs on eliminating cancer cells. However, manual work is required to count the number of cells exhibiting mitotic defects and micronuclei either directly from the viewing window of a microscope or from an image, which is tedious and creates errors. Therefore, this study aims to detect cells with mitotic defects and micronuclei by applying an approach that can automatically count the targets. This approach integrates the application of a convolutional neural network for normal cell identification and the proposed color layer signature analysis (CLSA) to spot cells with mitotic defects and micronuclei. This approach provides a method for researchers to detect colon cancer cells in an accurate and time-efficient manner, thereby decreasing errors and the processing time. The following sections will illustrate the methodology and workflow design of this study, as well as explain the practicality of the experimental comparisons and the results that were used to validate the practicality of this algorithm.