Automated detection of malignant features in confocal microscopy on superficial spreading melanoma versus nevi

Line-field confocal optical coherence tomography coupled with artificial intelligence algorithms to identify quantitative biomarkers of facial skin ageing

Quantitative biomarkers of facial skin ageing were studied from one hundred healthy Caucasian female volunteers, aged 20-70 years, using in vivo 3D Line-field Confocal Optical Coherence Tomography (LC-OCT) imaging coupled with Artificial Intelligence (AI)-based quantification algorithms. Layer metrics, i.e. stratum corneum thickness (SC), viable epidermal thickness and Dermal-Epidermal Junction (DEJ) undulation, as well as cellular metrics were measured for the temple, cheekbone and mandible. For all three investigated facial areas, minimal age-related variations were observed in the thickness of the SC and viable epidermis layers. A flatter and more homogeneous epidermis (decrease in the standard deviation of the number of layers means), a less dense cellular network with fewer cells per layer (decrease in cell surface density), and larger and more heterogeneous nuclei within each layer (increase in nuclei volume and their standard deviation) were found with significant variations with age. The higher atypia scores further reflected the heterogeneity of nuclei throughout the viable epidermis. The 3D visualisation of fine structures in the skin at the micrometric resolution and the 1200 µm × 500 µm field of view achieved with LC-OCT imaging enabled to compute relevant quantitative biomarkers for a better understanding of skin biology and the ageing process in vivo.

Age-dependent changes in epidermal architecture explored using an automated image analysis algorithm on in vivo reflectance confocal microscopy images

BACKGROUND

Reflectance confocal microscopy (RCM) allows for real-time in vivo visualization of the epidermis at the cellular level noninvasively. Parameters relating to tissue architecture can be extracted from RCM images, however, analysis of such images requires manual identification of cells to derive these parameters, which can be time-consuming and subject to human error, highlighting the need for an automated cell identification method.

METHODS

First, the region-of-interest (ROI) containing cells needs to be identified, followed by the identification of individual cells within the ROI. To perform this task, we use successive applications of Sato and Gabor filters. The final step is post-processing improvement of cell detection and removal of size outliers. The proposed algorithm is evaluated on manually annotated real data. It is then applied to 5345 images to study the evolution of epidermal architecture in children and adults. The images were acquired on the volar forearm of healthy children (3 months to 10 years) and women (25-80 years), and on the volar forearm and cheek of women (40-80 years). Following the identification of cell locations, parameters such as cell area, cell perimeter, and cell density are calculated, as well as the probability distribution of the number of nearest neighbors per cell. The thicknesses of the Stratum Corneum and supra-papillary epidermis are also calculated using a hybrid deep-learning method.

RESULTS

Epidermal keratinocytes are significantly larger (area and perimeter) in the granular layer than in the spinous layer and they get progressively larger with a child's age. Skin continues to mature dynamically during adulthood, as keratinocyte size continues to increase with age on both the cheeks and volar forearm, but the topology and cell aspect ratio remain unchanged across different epidermal layers, body sites, and age. Stratum Corneum and supra-papillary epidermis thicknesses increase with age, at a faster rate in children than in adults.

CONCLUSIONS

The proposed methodology can be applied to large datasets to automate image analysis and the calculation of parameters relevant to skin physiology. These data validate the dynamic nature of skin maturation during childhood and skin aging in adulthood.

Automatic granular and spinous epidermal cell identification and analysis on in vivo reflectance confocal microscopy images using cell morphological features

Significance

Reflectance confocal microscopy (RCM) allows for real-time in vivo visualization of the skin at the cellular level. The study of RCM images provides information on the structural properties of the epidermis. These may change in each layer of the epidermis, depending on the subject's age and the presence of certain dermatological conditions. Studying RCM images requires manual identification of cells to derive these properties, which is time consuming and subject to human error, highlighting the need for an automated cell identification method.

Aim

We aim to design an automated pipeline for the analysis of the structure of the epidermis from RCM images of the Stratum granulosum and Stratum spinosum.

Approach

We identified the region of interest containing the epidermal cells and the individual cells in the segmented tissue area using tubeness filters to highlight membranes. We used prior biological knowledge on cell size to process the resulting detected cells, removing cells that were too small and reapplying the used filters locally on detected regions that were too big to be considered a single cell. The proposed full image analysis pipeline (FIAP) was compared with machine learning-based approaches (cell cutter, different U-Net configurations, and loss functions).

Results

All methods were evaluated both on simulated data (four images) and on manually annotated RCM data (seven images). Accuracy was measured using recall and precision metrics. Both accuracy metrics were higher in the proposed FIAP for both real ( precision = 0.720 ± 0.068 , recall = 0.850 ± 0.11 ) and synthetic images ( precision = 0.835 ± 0.067 , recall = 0.925 ± 0.012 ). The tested machine learning methods failed to identify and segment keratinocytes on RCM images with a satisfactory accuracy.

Conclusions

We showed that automatic cell segmentation can be achieved using a pipeline based on membrane detection, with an accuracy that matches expert manual cell identification. To our knowledge, this is the first method based on membrane detection to study healthy skin using RCM images evaluated against manually identified cell positions.

Developments and Clinical Applications of Noninvasive Optical Technologies for Skin Cancer Diagnosis

Skin cancer has shown a sharp increase in prevalence over the past few decades and currently accounts for one-third of all cancers diagnosed. The most lethal form of skin cancer is melanoma, which develops in 4% of individuals. The rising prevalence and increased number of fatalities of skin cancer put a significant burden on healthcare resources and the economy. However, early detection and treatment greatly improve survival rates for patients with skin cancer. Since the rising rates of both the incidence and mortality have been particularly noticeable with melanoma, significant resources have been allocated to research aimed at earlier diagnosis and a deeper knowledge of the disease. Dermoscopy, reflectance confocal microscopy, optical coherence tomography, multiphoton-excited fluorescence imaging, and dermatofluorescence are only a few of the optical modalities reviewed here that have been employed to enhance noninvasive diagnosis of skin cancer in recent years. This review article discusses the methodology behind newly emerging noninvasive optical diagnostic technologies, their clinical applications, and advantages and disadvantages of these techniques, as well as the potential for their further advancement in the future.

Artificial intelligence in melanoma: A systematic review

BACKGROUND

Melanoma accounts for the majority of skin cancer deaths. Artificial intelligence has been applied in many types of cancers, and in melanoma in recent years. However, no systematic review summarized the application of artificial intelligence in melanoma.

AIMS

This study aims to systematically review previously published articles to explore the application of artificial intelligence in melanoma.

MATERIALS & METHODS

PubMed database was used to search the eligible publications on August 1, 2020. The query term was "artificial intelligence" and "melanoma."

RESULTS

A total of 51 articles were included in this review. Artificial intelligence technique is mainly used in the evaluation of dermoscopic images, other image segmentation and processing, and artificial intelligence diagnosis system.

DISCUSSION

Artificial intelligence is also applied in metastasis prediction, drug response prediction, and prognosis of melanoma. Besides, patients' perspectives of artificial intelligence and collaboration of human and artificial intelligence in melanoma also attracted attention. The query term might not include all articles, and we could not examine the algorithms that were built without publication.

CONCLUSION

The performance of artificial intelligence in melanoma is satisfactory and the future for potential applications is enormous.

Automating reflectance confocal microscopy image analysis for dermatological research: a review

SIGNIFICANCE

Reflectance confocal microscopy (RCM) is a noninvasive, in vivo technology that offers near histopathological resolution at the cellular level. It is useful in the study of phenomena for which obtaining a biopsy is impractical or would cause unnecessary tissue damage and trauma to the patient.

AIM

This review covers the use of RCM in the study of skin and the use of machine learning to automate information extraction. It has two goals: (1) an overview of information provided by RCM on skin structure and how it changes over time in response to stimuli and in disease and (2) an overview of machine learning approaches developed to automate the extraction of key morphological features from RCM images.

APPROACH

A PubMed search was conducted with additional literature obtained from references lists.

RESULTS

The application of RCM as an in vivo tool in dermatological research and the biologically relevant information derived from it are presented. Algorithms for image classification to epidermal layers, delineation of the dermal-epidermal junction, classification of skin lesions, and demarcation of individual cells within an image, all important factors in the makeup of the skin barrier, were reviewed. Application of image analysis methods in RCM is hindered by low image quality due to noise and/or poor contrast. Use of supervised machine learning is limited by time-consuming manual labeling of RCM images.

CONCLUSIONS

RCM has great potential in the study of skin structures. The use of artificial intelligence could enable an easier, more reproducible, precise, and rigorous study of RCM images for the understanding of skin structures, skin barrier, and skin inflammation and lesions. Although several attempts have been made, further work is still needed to provide a definite gold standard and overcome issues related to image quality, limited labeled datasets, and lack of phenotype variability in available databases.

Line-field confocal optical coherence tomography as a tool for three-dimensional in vivo quantification of healthy epidermis: A pilot study

Epidermal three-dimensional (3D) topography/quantification has not been completely characterized yet. The recently developed line-field confocal optical coherence tomography (LC-OCT) provides real-time, high-resolution, in-vivo 3D imaging of the skin. This pilot study aimed at quantifying epidermal metrics (epidermal thicknesses, dermal-epidermal junction [DEJ] undulation and keratinocyte number/shape/size) using 3D LC-OCT. For each study participant (8 female, skin-type-II, younger/older volunteers), seven body sites were imaged with LC-OCT. Epidermal metrics were calculated by segmentations and measurements assisted by artificial intelligence (AI) when appropriate. Thicknesses of epidermis/SC, DEJ undulation and keratinocyte nuclei volume varied across body sites. Evidence of keratinocyte maturation was observed in vivo: keratinocyte nuclei being small/spherical near the DEJ and flatter/elliptical near the skin surface. Skin microanatomy can be quantified by combining LC-OCT and AI. This technology could be highly relevant to understand aging processes and conditions linked to epidermal disorders. Future clinical/research applications are to be expected in this scenario.

Towards data-driven quantification of skin ageing using reflectance confocal microscopy

INTRODUCTION

Evaluation of skin ageing is a non-standardized, subjective process, with typical measures relying coarse, qualitatively defined features. Reflectance confocal microscopy depth stacks contain indicators of both chrono-ageing and photo-ageing. We hypothesize that an ageing scale could be constructed using machine learning and image analysis, creating a data-driven quantification of skin ageing without human assessment.

METHODS

En-face sections of reflectance confocal microscopy depth stacks from the dorsal and volar forearm of 74 participants (36/18/20 training/testing/validation) were represented using a histogram of visual features learned using unsupervised clustering of small image patches. A logistic regression classifier was trained on these histograms to differentiate between stacks from 20- to 30-year-old and 50- to 70-year-old volunteers. The probabilistic output of the logistic regression was used as the fine-grained ageing score for that stack in the testing set ranging from 0 to 1. Evaluation was performed in two ways: on the test set, the AUC was collected for the binary classification problem as well as by statistical comparison of the scores for age and body site groups. Final validation was performed by assessing the accuracy of the ageing score measurement on 20 depth stacks not used for training or evaluating the classifier.

RESULTS

The classifier effectively differentiated stacks from age groups with a test set AUC of 0.908. Mean scores were significantly different when comparing age groups (mean 0.70 vs. 0.44; t = -6.62, p = 0.0000) and also when comparing stacks from dorsal and volar body sites (mean 0.64 vs. 0.53; t = 3.12, p = 0.0062). On the final validation set, 17 out of 20 depth stacks were correctly labelled.

DISCUSSION

Despite being limited to only coarse training information in the form of example stacks from two age groups, the trained classifier was still able to effectively discriminate between younger skin and older skin. Curiously, despite being only trained with chronological age, there was still evidence for measurable differences in age scores due to sun exposure-with marked differences in scores on sun-exposed dorsal sites of some volunteers compared with less sun-exposed volar sites. These results suggest that fine-grained data-driven quantification of skin ageing is achievable.

Seeking a Widely Adoptable Practical Standard to Estimate Signal-to-Noise Ratio in Magnetic Resonance Imaging for Multiple-Coil Reconstructions

BACKGROUND

Signal-to-noise ratio (SNR) is used to evaluate the performance of magnetic resonance (MR) imaging systems. Accurate and consistent estimations are needed to routinely use SNR to assess coils and image reconstruction techniques.

PURPOSE

To identify a reliable and practical method for SNR estimation in multiple-coil reconstructions.

STUDY TYPE

Technical evaluation and comparison.

SUBJECTS/PHANTOM

A uniform phantom and four healthy volunteers: 35, 38, 39 y/o males, 25 y/o female.

FIELD STRENGTH/SEQUENCE

Two-dimensional multislice gradient-echo pulse sequence at 3 T and 7 T.

ASSESSMENT

Reference-standard SNR was calculated from 100 multiple replicas. Six SNR methods were compared against it: difference image (DI), analytic array combination (AC), pseudo-multiple-replica (PMR), generalized pseudo-replica (GPR), smoothed image subtraction (SIS), and DI with temporal instability correction (TIC). The assessment was repeated for different multiple-coil reconstructions.

STATISTICAL TESTS

SNR methods were evaluated in terms of relative deviation (RD) and normalized mutual information (NMI) with respect to the reference-standard, using a linear regression (0.05 significance level) to assess how different factors affect accuracy.

RESULTS

Average RD (phantom) for DI, AC, PMR, GPR, SIS, and TIC was 7.9%, 6%, 6.7%, 10.1%, 40%, and 14.6%, respectively. RD increased with acceleration. SNR maps with AC were the most similar to the reference standard (NMI = 0.358). Considering all brain regions of interest, average RD for all SNR methods varied 96% among volunteers but remained approximately 10% for AC, PMR, and GPR, whereas it was more than 30% for DI, SIS, and TIC. RD was mainly affected by image reconstruction (beta = 12) for AC and SNR entropy for SIS (beta = 19).

DATA CONCLUSION

AC provided accurate and robust SNR estimation. PMR and GPR are more generally applicable than AC. DI and TIC should be used only at low acceleration factors, when an additional noise-only scan cannot be acquired. SIS is a single-acquisition alternative to DI for generalized autocalibrating partial parallel acquisition (GRAPPA) reconstructions.

EVIDENCE LEVEL

1 TECHNICAL EFFICACY: Stage 1.

Non-invasive optical methods for melanoma diagnosis

Cutaneous melanoma is one of the most common malignancies with increased incidence in the past few decades, making it a significant public health problem. The early diagnosis of melanoma is a major factor in improving patient's survival. The traditional pathway to melanoma diagnosis starts with a visual diagnosis, followed by subsequent biopsy and histopathologic evaluation. Recently, multiple innovative optical technology-based methods, including dermoscopy, reflectance confocal microscopy, optical coherence tomography, multiphoton excited fluorescence imaging and stepwise two-photon excited fluorescence (dermatofluoroscopy), have been developed to increase the diagnostic accuracy for the non-invasive melanoma diagnosis. Some of them have already been applied to real-life clinical settings, others require more research and development. These technologies show promise in facilitating the diagnosis of melanoma since they are non-invasive, sensitive, objective and easy to apply. Diagnostic accuracy, detection time, portability and the cost-effectiveness of the device are all aspects that need to be improved. This article reviews the method of these emerging optical non-invasive diagnostic technologies, their clinical application, their benefits and limitations, as well as their possible future development.

Clinical Applications of In Vivo and Ex Vivo Confocal Microscopy

Confocal laser scanning microscopy (CLSM) has been introduced in clinical settings as a tool enabling a quasi-histologic view of a given tissue, without performing a biopsy. It has been applied to many fields of medicine mainly to the skin and to the analysis of skin cancers for both in vivo and ex vivo CLSM. In vivo CLSM involves reflectance mode, which is based on refractive index of cell structures serving as endogenous chromophores, reaching a depth of exploration of 200 μm. It has been proven to increase the diagnostic accuracy of skin cancers, both melanoma and non-melanoma. While histopathologic examination is the gold standard for diagnosis, in vivo CLSM alone and in addition to dermoscopy, contributes to the reduction of the number of excised lesions to exclude a melanoma, and to improve margin recognition in lentigo maligna, enabling tissue sparing for excisions. Ex vivo CLSM can be performed in reflectance and fluorescent mode. Fluorescence confocal microscopy is applied for “real-time” pathological examination of freshly excised specimens for diagnostic purposes and for the evaluation of margin clearance after excision in Mohs surgery. Further prospective interventional studies using CLSM might contribute to increase the knowledge about its application, reproducing real-life settings.

A Clinical Perspective on the Automated Analysis of Reflectance Confocal Microscopy in Dermatology

BACKGROUND AND OBJECTIVES

Non-invasive optical imaging has the potential to provide a diagnosis without the need for biopsy. One such technology is reflectance confocal microscopy (RCM), which uses low power, near-infrared laser light to enable real-time in vivo visualization of superficial human skin from the epidermis down to the papillary dermis. Although RCM has great potential as a diagnostic tool, there is a need for the development of reliable image analysis programs, as acquired grayscale images can be difficult and time-consuming to visually assess. The purpose of this review is to provide a clinical perspective on the current state of artificial intelligence (AI) for the analysis and diagnostic utility of RCM imaging.

STUDY DESIGN/MATERIALS AND METHODS

A systematic PubMed search was conducted with additional relevant literature obtained from reference lists.

RESULTS

Algorithms used for skin stratification, classification of pigmented lesions, and the quantification of photoaging were reviewed. Image segmentation, statistical methods, and machine learning techniques are among the most common methods used to analyze RCM image stacks. The poor visual contrast within RCM images and difficulty navigating image stacks were mediated by machine learning algorithms, which allowed the identification of specific skin layers.

CONCLUSIONS

AI analysis of RCM images has the potential to increase the clinical utility of this emerging technology. A number of different techniques have been utilized but further refinements are necessary to allow consistent accurate assessments for diagnosis. The automated detection of skin cancers requires more development, but future applications are truly boundless, and it is compelling to envision the role that AI will have in the practice of dermatology. Lasers Surg. Med. © 2020 Wiley Periodicals LLC.

Reflectance confocal microscopy terminology glossary for melanocytic skin lesions: A systematic review

BACKGROUND

There is lack of uniformity in the reflectance confocal microscopy (RCM) terminology for melanocytic lesions.

OBJECTIVE

To review published RCM terms for melanocytic lesions and identify redundant, synonymous terms.

METHODS

A systematic review of original research articles adhering to Preferred Reporting Items for Systematic Reviews and Meta-analyses (PRISMA) guidelines was conducted until August 15, 2018. Two investigators gathered all published RCM terms used to describe melanoma and melanocytic nevi. Synonymous terms were grouped based on similarity in definition and in histopathologic correlation.

RESULTS

Out of 156 full-text screened articles, 59 studies met the inclusion criteria. We identified 209 terms; 191 (91.4%) corresponding to high-magnification/cellular-level terms and 18 (8.6%) corresponding to low-magnification/architectural patterns terms. The overall average use frequency of RCM terms was 3.1 times (range, 1-31). By grouping of individual RCM terms based on likely synonymous definitions and by eliminating terms lacking clear definition, the total number of RCM terms could be potentially reduced from 209 to 40 terms (80.8% reduction).

LIMITATIONS

Non-English and non-peer-reviewed articles were excluded.

CONCLUSIONS

This systematic review of published RCM terms identified significant terminology redundancy. It provides the basis for subsequent terminology consensus on melanocytic neoplasms.

Classification of Lentigo Maligna at Patient-Level by Means of Reflectance Confocal Microscopy Data

Reflectance confocal microscopy is an appropriate tool for the diagnosis of lentigo maligna. Compared with dermoscopy, this device can provide abundant information as a mosaic and/or a stack of images. In this particular context, the number of images per patient varied between 2 and 833 images and the objective, ultimately, is to be able to discern between benign and malignant classes. First, this paper evaluated classification at the image level, with the help of handcrafted methods derived from the literature and transfer learning methods. The transfer learning feature extraction methods outperformed the handcrafted feature extraction methods from literature, with a F 1 score value of 0.82. Secondly, this work proposed patient-level supervised methods based on image decisions and a comparison of these with multi-instance learning methods. This study achieved comparable results to those of the dermatologists, with an auc score of 0.87 for supervised patient diagnosis and an auc score of 0.88 for multi-instance learning patient diagnosis. According to these results, computer-aided diagnosis methods presented in this paper could be easily used in a clinical context to save time or confirm a diagnosis and can be oriented to detect images of interest. Also, this methodology can be used to serve future works based on multimodality.

Reflectance confocal microscopy of skin in vivo: From bench to bedside

Following more than two decades of effort, reflectance confocal microscopy (RCM) imaging of skin was granted codes for reimbursement by the US Centers for Medicare and Medicaid Services. Dermatologists in the USA have started billing and receiving reimbursement for the imaging procedure and for the reading and interpretation of images. RCM imaging combined with dermoscopic examination is guiding the triage of lesions into those that appear benign, which are being spared from biopsy, against those that appear suspicious, which are then biopsied. Thus far, a few thousand patients have been spared from biopsy of benign lesions. The journey of RCM imaging from bench to bedside is certainly a success story, but still much more work lies ahead toward wider dissemination, acceptance, and adoption. We present a brief review of RCM imaging and highlight key challenges and opportunities. The success of RCM imaging paves the way for other emerging optical technologies, as well-and our bet for the future is on multimodal approaches. Lasers Surg. Med. 49:7-19, 2017. © 2016 Wiley Periodicals, Inc.

Unsupervised delineation of stratum corneum using reflectance confocal microscopy and spectral clustering

BACKGROUND

Measuring the thickness of the stratum corneum (SC) in vivo is often required in pharmacological, dermatological, and cosmetological studies. Reflectance confocal microscopy (RCM) offers a non-invasive imaging-based approach. However, RCM-based measurements currently rely on purely visual analysis of images, which is time-consuming and suffers from inter-user subjectivity.

METHODS

We developed an unsupervised segmentation algorithm that can automatically delineate the SC layer in stacks of RCM images of human skin. We represent the unique textural appearance of SC layer using complex wavelet transform and distinguish it from deeper granular layers of skin using spectral clustering. Moreover, through localized processing in a matrix of small areas (called 'tiles'), we obtain lateral variation of SC thickness over the entire field of view.

RESULTS

On a set of 15 RCM stacks of normal human skin, our method estimated SC thickness with a mean error of 5.4 ± 5.1 μm compared to the 'ground truth' segmentation obtained from a clinical expert.

CONCLUSION

Our algorithm provides a non-invasive RCM imaging-based solution which is automated, rapid, objective, and repeatable.

Backscattering of linearly polarized light from turbid tissue-like scattering medium with rough surface

In the framework of further development of a unified computational tool for the needs of biomedical optics, we introduce an electric field Monte Carlo (MC) model for simulation of backscattering of coherent linearly polarized light from a turbid tissue-like scattering medium with a rough surface. We consider the laser speckle patterns formation and the role of surface roughness in the depolarization of linearly polarized light backscattered from the medium. The mutual phase shifts due to the photons’ pathlength difference within the medium and due to reflection/refraction on the rough surface of the medium are taken into account. The validation of the model includes the creation of the phantoms of various roughness and optical properties, measurements of co- and cross-polarized components of the backscattered/reflected light, its analysis and extensive computer modeling accelerated by parallel computing on the NVIDIA graphics processing units using compute unified device architecture (CUDA). The analysis of the spatial intensity distribution is based on second-order statistics that shows a strong correlation with the surface roughness, both with the results of modeling and experiment. The results of modeling show a good agreement with the results of experimental measurements on phantoms mimicking human skin. The developed MC approach can be used for the direct simulation of light scattered by the turbid scattering medium with various roughness of the surface.

Automated Segmentation of Skin Strata in Reflectance Confocal Microscopy Depth Stacks

Reflectance confocal microscopy (RCM) is a powerful tool for in-vivo examination of a variety of skin diseases. However, current use of RCM depends on qualitative examination by a human expert to look for specific features in the different strata of the skin. Developing approaches to quantify features in RCM imagery requires an automated understanding of what anatomical strata is present in a given en-face section. This work presents an automated approach using a bag of features approach to represent en-face sections and a logistic regression classifier to classify sections into one of four classes (stratum corneum, viable epidermis, dermal-epidermal junction and papillary dermis). This approach was developed and tested using a dataset of 308 depth stacks from 54 volunteers in two age groups (20–30 and 50–70 years of age). The classification accuracy on the test set was 85.6%. The mean absolute error in determining the interface depth for each of the stratum corneum/viable epidermis, viable epidermis/dermal-epidermal junction and dermal-epidermal junction/papillary dermis interfaces were 3.1 μm, 6.0 μm and 5.5 μm respectively. The probabilities predicted by the classifier in the test set showed that the classifier learned an effective model of the anatomy of human skin.

Methods of Melanoma Detection

Detection and removal of melanoma, before it has metastasized, dramatically improves prognosis and survival. The purpose of this chapter is to (1) summarize current methods of melanoma detection and (2) review state-of-the-art detection methods and technologies that have the potential to reduce melanoma mortality. Current strategies for the detection of melanoma range from population-based educational campaigns and screening to the use of algorithm-driven imaging technologies and performance of assays that identify markers of transformation. This chapter will begin by describing state-of-the-art methods for educating and increasing awareness of at-risk individuals and for performing comprehensive screening examinations. Standard and advanced photographic methods designed to improve reliability and reproducibility of the clinical examination will also be reviewed. Devices that magnify and/or enhance malignant features of individual melanocytic lesions (and algorithms that are available to interpret the results obtained from these devices) will be compared and contrasted. In vivo confocal microscopy and other cellular-level in vivo technologies will be compared to traditional tissue biopsy, and the role of a noninvasive "optical biopsy" in the clinical setting will be discussed. Finally, cellular and molecular methods that have been applied to the diagnosis of melanoma, such as comparative genomic hybridization (CGH), fluorescent in situ hybridization (FISH), and quantitative reverse transcriptase polymerase chain reaction (qRT-PCR), will be discussed.

Automated delineation of dermal-epidermal junction in reflectance confocal microscopy image stacks of human skin

Reflectance confocal microscopy (RCM) images skin noninvasively, with optical sectioning and nuclear-level resolution comparable with that of pathology. On the basis of the assessment of the dermal-epidermal junction (DEJ) and morphologic features in its vicinity, skin cancer can be diagnosed in vivo with high sensitivity and specificity. However, the current visual, qualitative approach for reading images leads to subjective variability in diagnosis. We hypothesize that machine learning-based algorithms may enable a more quantitative, objective approach. Testing and validation were performed with two algorithms that can automatically delineate the DEJ in RCM stacks of normal human skin. The test set was composed of 15 fair- and 15 dark-skin stacks (30 subjects) with expert labelings. In dark skin, in which the contrast is high owing to melanin, the algorithm produced an average error of 7.9±6.4 μm. In fair skin, the algorithm delineated the DEJ as a transition zone, with average error of 8.3±5.8 μm for the epidermis-to-transition zone boundary and 7.6±5.6 μm for the transition zone-to-dermis. Our results suggest that automated algorithms may quantitatively guide the delineation of the DEJ, to assist in objective reading of RCM images. Further development of such algorithms may guide assessment of abnormal morphological features at the DEJ.

Current methods of non-invasive diagnostics of skin melanoma

Malignant skin neoplasms including melanoma are now occupying the leading positions in the list of cancer pathologies in Russia, a number of European countries and United States. The percentage of melanoma is less than 4% of the total number of skin neoplasms but melanomas account for up to 80% of lethal cases caused by malignant skin neoplasms. Due to the aggressive nature of melanomas including the proliferative reaction to any invasions including diagnostic ones, there is an urgent need in a reliable non-invasive method to differentiate melanomas from other skin neoplasms and diagnose it at an early stage. This review article discloses descriptions, potential, advantages and shortcomings of non-invasive melanoma diagnostics methods: dermatoscopy, SIAscopy and confocal microscopy. Based on the results of the review, the authors made a conclusion that the development of an algorithm of diagnostics taking into account individual clinical and medical history data, results of non-invasive visualization methods and laboratory test results (proteome analysis, tumor markers) would help to overcome the shortcomings of each individual method, diagnose melanomas at early stages and, respectively, personalize the approach to management for each patient.

Modern non-invasive diagnostic techniques in the detection of early cutaneous melanoma

Over the past few years melanoma has grown into a disease of socio-economic importance due to the increasing incidence and persistently high mortality rates. Melanoma is a malignant tumor with a high tendency to metastasize. Therefore, an extremely important part of the therapeutic process is to identify the disease at an early stage: in situ or stage I. Many tools for early diagnosis of melanoma are available today, including dermoscopy, videodermoscopy and in vivo reflectance confocal microscopy. Other methods such as high frequency ultrasound, optical coherence tomography and electrical impedance spectroscopy may serve as additional diagnostic aids. Modern imaging techniques also allow the monitoring of melanocytic skin lesions over months or years to detect the moment of malignant transformation. This review summarizes the current knowledge about modern diagnostic techniques, which may aid early diagnosis of melanoma.

Computer aided measurement of melanoma depth of invasion in microscopic images

This paper presents a novel computer aided technique for measurement of melanoma depth of invasion. Melanoma is the deadliest form of skin cancer with worldwide increasing incidences. For a conclusive diagnosis of melanoma, skin biopsies should be examined under a microscope. Visual inspection of microscopic samples is often subjective, time-consuming, cumbersome and prone to human errors. This fact demonstrates the necessity of developing an automated method which assists pathologists in evaluating histopathological samples more accurately in the busy clinical environment. To the best of our knowledge, this is the first time that a computer-assisted diagnosis algorithm has been applied in measurement of melanoma invasion depth. The proposed method uses a clustering algorithm for granular layer extraction and a pre-trained SVM classifier for detection of malignant melanocytes. The experimental results with average error of 3.9μm demonstrate that the proposed method is reliable and effective.

Segmentation of diffuse reflectance hyperspectral datasets with noise for detection of Melanoma

We present a segmentation algorithm that allows optical properties to be extracted from diffuse reflectance hyperspectral datasets with a speedup of three orders of magnitude when compared to current methods. Such data could be used for the detection of melanoma. The algorithm first performs dimensionality reduction using principal component analysis, and then the image is segmented using k-means clustering. The mean spectrum from each cluster is then calculated and can be used to extract chemical information. By reducing the number of spectra to be analyzed, extraction of physiological information can be achieved three orders of magnitude faster than methods requiring the analysis of every spectrum in the hyperspectral dataset. The effect of noise on the ability of the algorithm to accurately segment images was tested using digital phantoms, for which the noise level was under the control of the investigators. The analysis showed a linear relationship between the level of noise and the smallest difference in scattering that the algorithm was able to accurately detect and segment. This finding can be used to determine the maximum amount of noise in the imaging system that will still allow detection of the difference in optical properties between non-melanoma and melanoma.

In vivo confocal reflectance microscopy in melanoma

Reflectance confocal microscopy (RCM) enables the noninvasive in vivo imaging of the skin with a horizontal axis and a cellular-level resolution allowing the study of the skin from superficial layers to papillary dermis. It has arisen an important tool in the study of tumors and specially an important role in the characterization of melanoma. Melanocytic lesions present a large number of characteristic findings visible in upper parts of the tumors, such as in the case of melanoma: pagetoid roundish or dendritic cells in superficial epidermis, atypical nests at the dermoepidermal junction, nonedged papillae and atypical nucleated cells in papillary dermis. Several studies have demonstrated that RCM may improve the accuracy in the differentiation of benign and malignant melanocytic lesions as an adjuvant technique to dermoscopy, and three main algorithms have been developed to apply in equivocal lesions. The advantage of in vivo observation in real time of the tumor at the bedside is opening the clinical applications of RCM in the evaluation of melanocytic lesions, and in particular in the study of facial maculae and lentigo maligna melanoma, amelanotic melanoma, and management of subclinical margins, recurrences, or monitoring noninvasive treatment of tumors.

Out-of-plane Stokes imaging polarimeter for early skin cancer diagnosis

Optimal treatment of skin cancer before it metastasizes critically depends on early diagnosis and treatment. Imaging spectroscopy and polarized remittance have been utilized in the past for diagnostic purposes, but valuable information can be also obtained from the analysis of skin roughness. For this purpose, we have developed an out-of-plane hemispherical Stokes imaging polarimeter designed to monitor potential skin neoplasia based on a roughness assessment of the epidermis. The system was utilized to study the rough surface scattering for wax samples and human skin. The scattering by rough skin-simulating phantoms showed behavior that is reasonably described by a facet scattering model. Clinical tests were conducted on patients grouped as follows: benign nevi, melanocytic nevus, melanoma, and normal skin. Images were captured and analyzed, and polarization properties are presented in terms of the principal angle of the polarization ellipse and the degree of polarization. In the former case, there is separation between different groups of patients for some incidence azimuth angles. In the latter, separation between different skin samples for various incidence azimuth angles is observed.

Validation Study of Automated Dermal/Epidermal Junction Localization Algorithm in Reflectance Confocal Microscopy Images of Skin

Reflectance confocal microscopy (RCM) has seen increasing clinical application for noninvasive diagnosis of skin cancer. Identifying the location of the dermal-epidermal junction (DEJ) in the image stacks is key for effective clinical imaging. For example, one clinical imaging procedure acquires a dense stack of 0.5×0.5mm FOV images and then, after manual determination of DEJ depth, collects a 5×5mm mosaic at that depth for diagnosis. However, especially in lightly pigmented skin, RCM images have low contrast at the DEJ which makes repeatable, objective visual identification challenging. We have previously published proof of concept for an automated algorithm for DEJ detection in both highly- and lightly-pigmented skin types based on sequential feature segmentation and classification. In lightly-pigmented skin the change of skin texture with depth was detected by the algorithm and used to locate the DEJ. Here we report on further validation of our algorithm on a more extensive collection of 24 image stacks (15 fair skin, 9 dark skin). We compare algorithm performance against classification by three clinical experts. We also evaluate inter-expert consistency among the experts. The average correlation across experts was 0.81 for lightly pigmented skin, indicating the difficulty of the problem. The algorithm achieved epidermis/dermis misclassification rates smaller than 10% (based on 25×25 mm tiles) and average distance from the expert labeled boundaries of ~6.4 μm for fair skin and ~5.3 μm for dark skin, well within average cell size and less than 2x the instrument resolution in the optical axis.

Semi-automated Algorithm for Localization of Dermal/ Epidermal Junction in Reflectance Confocal Microscopy Images of Human Skin

The examination of the dermis/epidermis junction (DEJ) is clinically important for skin cancer diagnosis. Reflectance confocal microscopy (RCM) is an emerging tool for detection of skin cancers in vivo. However, visual localization of the DEJ in RCM images, with high accuracy and repeatability, is challenging, especially in fair skin, due to low contrast, heterogeneous structure and high inter- and intra-subject variability. We recently proposed a semi-automated algorithm to localize the DEJ in z-stacks of RCM images of fair skin, based on feature segmentation and classification. Here we extend the algorithm to dark skin. The extended algorithm first decides the skin type and then applies the appropriate DEJ localization method. In dark skin, strong backscatter from the pigment melanin causes the basal cells above the DEJ to appear with high contrast. To locate those high contrast regions, the algorithm operates on small tiles (regions) and finds the peaks of the smoothed average intensity depth profile of each tile. However, for some tiles, due to heterogeneity, multiple peaks in the depth profile exist and the strongest peak might not be the basal layer peak. To select the correct peak, basal cells are represented with a vector of texture features. The peak with most similar features to this feature vector is selected. The results show that the algorithm detected the skin types correctly for all 17 stacks tested (8 fair, 9 dark). The DEJ detection algorithm achieved an average distance from the ground truth DEJ surface of around 4.7μm for dark skin and around 7-14μm for fair skin.

Automated identification of epidermal keratinocytes in reflectance confocal microscopy

Keratinocytes in skin epidermis, which have bright cytoplasmic contrast and dark nuclear contrast in reflectance confocal microscopy (RCM), were modeled with a simple error function reflectance profile: erf( ). Forty-two example keratinocytes were identified as a training set which characterized the nuclear size a = 8.6±2.8 μm and reflectance gradient b = 3.6±2.1 μm at the nuclear∕cytoplasmic boundary. These mean a and b parameters were used to create a rotationally symmetric erf( ) mask that approximated the mean keratinocyte image. A computer vision algorithm used an erf( ) mask to scan RCM images, identifying the coordinates of keratinocytes. Applying the mask to the confocal data identified the positions of keratinocytes in the epidermis. This simple model may be used to noninvasively evaluate keratinocyte populations as a quantitative morphometric diagnostic in skin cancer detection and evaluation of dermatological cosmetics.