Pilot study of semiautomated localization of the dermal/epidermal junction in reflectance confocal microscopy images of skin

The skin through reflectance confocal microscopy - Historical background, technical principles, and its correlation with histopathology

Since its first introduction into medical practice, reflectance confocal microscopy (RCM) has been a valuable non-invasive diagnostic tool for the assessment of benign and malignant neoplasms of the skin. It has also been used as an adjunct for diagnosing equivocal cutaneous neoplasms that lack characteristic clinical or dermoscopic features. The use of RCM has led to a decreased number of biopsies of benign lesions. Multiple published studies show a strong correlation between RCM and histopathology thereby creating a bridge between clinical aspects, dermoscopy, and histopathology. Dermatopathologists may potentially play an important role in the interpretation of confocal images, by their ability to correlate histopathologic findings. RCM has also been shown to be an important adjunct to delineating tumoral margins during surgery, as well as for monitoring the non-surgical treatment of skin cancers. Advanced technology with smaller probes, such as the VivaScope 3000, has allowed access to lesions in previously inaccessible anatomic locations. This review explains the technical principles of RCM and describes the most common RCM features of normal skin with their corresponding histological correlation.

Artificial Intelligence-Based Approaches to Reflectance Confocal Microscopy Image Analysis in Dermatology

Reflectance confocal microscopy (RCM) is a non-invasive imaging method designed to identify various skin diseases. Confocal based diagnosis may be subjective due to the learning curve of the method, the scarcity of training programs available for RCM, and the lack of clearly defined diagnostic criteria for all skin conditions. Given that in vivo RCM is becoming more widely used in dermatology, numerous deep learning technologies have been developed in recent years to provide a more objective approach to RCM image analysis. Machine learning-based algorithms are used in RCM image quality assessment to reduce the number of artifacts the operator has to view, shorten evaluation times, and decrease the number of patient visits to the clinic. However, the current visual method for identifying the dermal-epidermal junction (DEJ) in RCM images is subjective, and there is a lot of variation. The delineation of DEJ on RCM images could be automated through artificial intelligence, saving time and assisting novice RCM users in studying the key DEJ morphological structure. The purpose of this paper is to supply a current summary of machine learning and artificial intelligence’s impact on the quality control of RCM images, key morphological structures identification, and detection of different skin lesion types on static RCM images.

Skin strata delineation in reflectance confocal microscopy images using recurrent convolutional networks with attention

Reflectance confocal microscopy (RCM) is an effective non-invasive tool for cancer diagnosis. However, acquiring and reading RCM images requires extensive training and experience, and novice clinicians exhibit high discordance in diagnostic accuracy. Quantitative tools to standardize image acquisition could reduce both required training and diagnostic variability. To perform diagnostic analysis, clinicians collect a set of RCM mosaics (RCM images concatenated in a raster fashion to extend the field view) at 4-5 specific layers in skin, all localized in the junction between the epidermal and dermal layers (dermal-epidermal junction, DEJ), necessitating locating that junction before mosaic acquisition. In this study, we automate DEJ localization using deep recurrent convolutional neural networks to delineate skin strata in stacks of RCM images collected at consecutive depths. Success will guide to automated and quantitative mosaic acquisition thus reducing inter operator variability and bring standardization in imaging. Testing our model against an expert labeled dataset of 504 RCM stacks, we achieved [Formula: see text] classification accuracy and nine-fold reduction in the number of anatomically impossible errors compared to the previous state-of-the-art.

Visualizing the in-vivo application of zinc in sensitive skin using reflectance confocal microscopy

Findings obtained on objective assessments to evaluate sensitive skin do not correlate well with the symptomatology. We utilized reflectance confocal microscopy (RCM) to compare transepidermal application of zinc in sensitive and non-sensitive skin. Thirty-six subjects participated in this study. They were divided into groups based on lactic acid sting test (LAST):'stinger' and 'non-stinger'; transepidermal water loss (TEWL) measurements; and sensitivity self-assessments: 'sensitive' and 'non-sensitive'. RCM images were taken to visualize transepidermal application of topically-applied zinc. The intensity of zinc reflectance at different depths was measured by ImageJ software. Based on LAST scores, the 'stinger' group showed significantly higher reflectance of zinc at 8 µm (stratum corneum) [face (P < 0.001), forearm (P = 0.004)], and at 80-104 µm (dermo-epidermal junction layer) on the face. High-TEWL group showed increased zinc reflectance at 8-24 µm (tight junction layer, P < 0.001). There were no significant differences amongst subjects self-reporting 'sensitive' and 'non-sensitive' skin. RCM demonstrates that in sensitive skin, there is deeper and higher reflectance of zinc at multiple depths. Structural differences are also visualized. We suggest that RCM is a useful tool for evaluating skin barrier integrity.

In vivo dual-mode full-field optical coherence tomography for differentiation of types of melanocytic nevi

SIGNIFICANCE

Melanocytic nevi represent the most common dermal melanocytic lesions in humans. Nevus is typically diagnosed clinically with the naked eye or with dermoscopy. However, it is essential to identify the type of nevus by invasive biopsy for histopathological examination. The use of noninvasive imaging tools can be used to evaluate the types of nevi to reduce unnecessary excisions of benign entities.

AIM

To evaluate the feasibility of using en face and cross-sectional full-field optical coherence tomography (FF-OCT) in differentiation of melanocytic nevi that can facilitate the reduction of unnecessary excisions of benign entities.

APPROACH

Dual-mode Mirau-type FF-OCT for cross-sectional imaging (B-scan) and en face imaging were used to distinguish the types of nevi.

RESULTS

Although the B-scan reveals the distribution of melanosomes, users can set a specific depth of the en face image to explore the morphology of surrounding skin cells instantly. According to the locations of nevus nests, the different types of nevi, including junction nevus and compound nevus, can be identified using this dual-mode FF-OCT system.

CONCLUSIONS

Combining B-scan and en face imaging in vivo FF-OCT enables the examination and navigation of skin tissues in real time and in three dimensions.

A Clinical Perspective on the Automated Analysis of Reflectance Confocal Microscopy in Dermatology

BACKGROUND AND OBJECTIVES

Non-invasive optical imaging has the potential to provide a diagnosis without the need for biopsy. One such technology is reflectance confocal microscopy (RCM), which uses low power, near-infrared laser light to enable real-time in vivo visualization of superficial human skin from the epidermis down to the papillary dermis. Although RCM has great potential as a diagnostic tool, there is a need for the development of reliable image analysis programs, as acquired grayscale images can be difficult and time-consuming to visually assess. The purpose of this review is to provide a clinical perspective on the current state of artificial intelligence (AI) for the analysis and diagnostic utility of RCM imaging.

STUDY DESIGN/MATERIALS AND METHODS

A systematic PubMed search was conducted with additional relevant literature obtained from reference lists.

RESULTS

Algorithms used for skin stratification, classification of pigmented lesions, and the quantification of photoaging were reviewed. Image segmentation, statistical methods, and machine learning techniques are among the most common methods used to analyze RCM image stacks. The poor visual contrast within RCM images and difficulty navigating image stacks were mediated by machine learning algorithms, which allowed the identification of specific skin layers.

CONCLUSIONS

AI analysis of RCM images has the potential to increase the clinical utility of this emerging technology. A number of different techniques have been utilized but further refinements are necessary to allow consistent accurate assessments for diagnosis. The automated detection of skin cancers requires more development, but future applications are truly boundless, and it is compelling to envision the role that AI will have in the practice of dermatology. Lasers Surg. Med. © 2020 Wiley Periodicals LLC.

Dermal epidermal junction detection for full-field optical coherence tomography data of human skin by deep learning

Full-field optical coherence tomography (FF-OCT) has been developed to obtain three-dimensional (3D) OCT data of human skin for early diagnosis of skin cancer. Detection of dermal epidermal junction (DEJ), where melanomas and basal cell carcinomas originate, is an essential step for skin cancer diagnosis. However, most existing DEJ detection methods consider each cross-sectional frame of the 3D OCT data independently, leaving the relationship between neighboring frames unexplored. In this paper, we exploit the continuity of 3D OCT data to enhance DEJ detection. In particular, we propose a method for noise reduction of the training data and a multi-directional convolutional neural network to predict the probability of epidermal pixels in the 3D OCT data, which is more stable than one-directional convolutional neural network for DEJ detection. Our crosscheck refinement method also exploits the domain knowledge to generate a smooth DEJ surface. The average mean error of the entire DEJ detection system is approximately 6 μm.

A Marked Poisson Process Driven Latent Shape Model for 3D Segmentation of Reflectance Confocal Microscopy Image Stacks of Human Skin

Segmenting objects of interest from 3D data sets is a common problem encountered in biological data. Small field of view and intrinsic biological variability combined with optically subtle changes of intensity, resolution, and low contrast in images make the task of segmentation difficult, especially for microscopy of unstained living or freshly excised thick tissues. Incorporating shape information in addition to the appearance of the object of interest can often help improve segmentation performance. However, the shapes of objects in tissue can be highly variable and design of a flexible shape model that encompasses these variations is challenging. To address such complex segmentation problems, we propose a unified probabilistic framework that can incorporate the uncertainty associated with complex shapes, variable appearance, and unknown locations. The driving application that inspired the development of this framework is a biologically important segmentation problem: the task of automatically detecting and segmenting the dermal-epidermal junction (DEJ) in 3D reflectance confocal microscopy (RCM) images of human skin. RCM imaging allows noninvasive observation of cellular, nuclear, and morphological detail. The DEJ is an important morphological feature as it is where disorder, disease, and cancer usually start. Detecting the DEJ is challenging, because it is a 2D surface in a 3D volume which has strong but highly variable number of irregularly spaced and variably shaped "peaks and valleys." In addition, RCM imaging resolution, contrast, and intensity vary with depth. Thus, a prior model needs to incorporate the intrinsic structure while allowing variability in essentially all its parameters. We propose a model which can incorporate objects of interest with complex shapes and variable appearance in an unsupervised setting by utilizing domain knowledge to build appropriate priors of the model. Our novel strategy to model this structure combines a spatial Poisson process with shape priors and performs inference using Gibbs sampling. Experimental results show that the proposed unsupervised model is able to automatically detect the DEJ with physiologically relevant accuracy in the range 10- 20 μm .

Reflectance confocal microscopy of skin in vivo: From bench to bedside

Following more than two decades of effort, reflectance confocal microscopy (RCM) imaging of skin was granted codes for reimbursement by the US Centers for Medicare and Medicaid Services. Dermatologists in the USA have started billing and receiving reimbursement for the imaging procedure and for the reading and interpretation of images. RCM imaging combined with dermoscopic examination is guiding the triage of lesions into those that appear benign, which are being spared from biopsy, against those that appear suspicious, which are then biopsied. Thus far, a few thousand patients have been spared from biopsy of benign lesions. The journey of RCM imaging from bench to bedside is certainly a success story, but still much more work lies ahead toward wider dissemination, acceptance, and adoption. We present a brief review of RCM imaging and highlight key challenges and opportunities. The success of RCM imaging paves the way for other emerging optical technologies, as well-and our bet for the future is on multimodal approaches. Lasers Surg. Med. 49:7-19, 2017. © 2016 Wiley Periodicals, Inc.

Unsupervised delineation of stratum corneum using reflectance confocal microscopy and spectral clustering

BACKGROUND

Measuring the thickness of the stratum corneum (SC) in vivo is often required in pharmacological, dermatological, and cosmetological studies. Reflectance confocal microscopy (RCM) offers a non-invasive imaging-based approach. However, RCM-based measurements currently rely on purely visual analysis of images, which is time-consuming and suffers from inter-user subjectivity.

METHODS

We developed an unsupervised segmentation algorithm that can automatically delineate the SC layer in stacks of RCM images of human skin. We represent the unique textural appearance of SC layer using complex wavelet transform and distinguish it from deeper granular layers of skin using spectral clustering. Moreover, through localized processing in a matrix of small areas (called 'tiles'), we obtain lateral variation of SC thickness over the entire field of view.

RESULTS

On a set of 15 RCM stacks of normal human skin, our method estimated SC thickness with a mean error of 5.4 ± 5.1 μm compared to the 'ground truth' segmentation obtained from a clinical expert.

CONCLUSION

Our algorithm provides a non-invasive RCM imaging-based solution which is automated, rapid, objective, and repeatable.

Methods of Melanoma Detection

Detection and removal of melanoma, before it has metastasized, dramatically improves prognosis and survival. The purpose of this chapter is to (1) summarize current methods of melanoma detection and (2) review state-of-the-art detection methods and technologies that have the potential to reduce melanoma mortality. Current strategies for the detection of melanoma range from population-based educational campaigns and screening to the use of algorithm-driven imaging technologies and performance of assays that identify markers of transformation. This chapter will begin by describing state-of-the-art methods for educating and increasing awareness of at-risk individuals and for performing comprehensive screening examinations. Standard and advanced photographic methods designed to improve reliability and reproducibility of the clinical examination will also be reviewed. Devices that magnify and/or enhance malignant features of individual melanocytic lesions (and algorithms that are available to interpret the results obtained from these devices) will be compared and contrasted. In vivo confocal microscopy and other cellular-level in vivo technologies will be compared to traditional tissue biopsy, and the role of a noninvasive "optical biopsy" in the clinical setting will be discussed. Finally, cellular and molecular methods that have been applied to the diagnosis of melanoma, such as comparative genomic hybridization (CGH), fluorescent in situ hybridization (FISH), and quantitative reverse transcriptase polymerase chain reaction (qRT-PCR), will be discussed.

Through the looking glass: Basics and principles of reflectance confocal microscopy

Reflectance confocal microscopy (RCM) offers high-resolution, noninvasive skin imaging and can help avoid obtaining unnecessary biopsy specimens. It can also increase efficiency in the surgical setting by helping to delineate tumor margins. Diagnostic criteria and several RCM algorithms have been published for the differentiation of benign and malignant neoplasms. We provide an overview of the basic principles of RCM, characteristic RCM features of normal skin and cutaneous neoplasms, and the limitations and future directions of RCM.

Automated detection of actinic keratoses in clinical photographs

BACKGROUND

Clinical diagnosis of actinic keratosis is known to have intra- and inter-observer variability, and there is currently no non-invasive and objective measure to diagnose these lesions.

OBJECTIVE

The aim of this pilot study was to determine if automatically detecting and circumscribing actinic keratoses in clinical photographs is feasible.

METHODS

Photographs of the face and dorsal forearms were acquired in 20 volunteers from two groups: the first with at least on actinic keratosis present on the face and each arm, the second with no actinic keratoses. The photographs were automatically analysed using colour space transforms and morphological features to detect erythema. The automated output was compared with a senior consultant dermatologist's assessment of the photographs, including the intra-observer variability. Performance was assessed by the correlation between total lesions detected by automated method and dermatologist, and whether the individual lesions detected were in the same location as the dermatologist identified lesions. Additionally, the ability to limit false positives was assessed by automatic assessment of the photographs from the no actinic keratosis group in comparison to the high actinic keratosis group.

RESULTS

The correlation between the automatic and dermatologist counts was 0.62 on the face and 0.51 on the arms, compared to the dermatologist's intra-observer variation of 0.83 and 0.93 for the same. Sensitivity of automatic detection was 39.5% on the face, 53.1% on the arms. Positive predictive values were 13.9% on the face and 39.8% on the arms. Significantly more lesions (p<0.0001) were detected in the high actinic keratosis group compared to the no actinic keratosis group.

CONCLUSIONS

The proposed method was inferior to assessment by the dermatologist in terms of sensitivity and positive predictive value. However, this pilot study used only a single simple feature and was still able to achieve sensitivity of detection of 53.1% on the arms.This suggests that image analysis is a feasible avenue of investigation for overcoming variability in clinical assessment. Future studies should focus on more sophisticated features to improve sensitivity for actinic keratoses without erythema and limit false positives associated with the anatomical structures on the face.

Automated delineation of dermal-epidermal junction in reflectance confocal microscopy image stacks of human skin

Reflectance confocal microscopy (RCM) images skin noninvasively, with optical sectioning and nuclear-level resolution comparable with that of pathology. On the basis of the assessment of the dermal-epidermal junction (DEJ) and morphologic features in its vicinity, skin cancer can be diagnosed in vivo with high sensitivity and specificity. However, the current visual, qualitative approach for reading images leads to subjective variability in diagnosis. We hypothesize that machine learning-based algorithms may enable a more quantitative, objective approach. Testing and validation were performed with two algorithms that can automatically delineate the DEJ in RCM stacks of normal human skin. The test set was composed of 15 fair- and 15 dark-skin stacks (30 subjects) with expert labelings. In dark skin, in which the contrast is high owing to melanin, the algorithm produced an average error of 7.9±6.4 μm. In fair skin, the algorithm delineated the DEJ as a transition zone, with average error of 8.3±5.8 μm for the epidermis-to-transition zone boundary and 7.6±5.6 μm for the transition zone-to-dermis. Our results suggest that automated algorithms may quantitatively guide the delineation of the DEJ, to assist in objective reading of RCM images. Further development of such algorithms may guide assessment of abnormal morphological features at the DEJ.

Validation Study of Automated Dermal/Epidermal Junction Localization Algorithm in Reflectance Confocal Microscopy Images of Skin

Reflectance confocal microscopy (RCM) has seen increasing clinical application for noninvasive diagnosis of skin cancer. Identifying the location of the dermal-epidermal junction (DEJ) in the image stacks is key for effective clinical imaging. For example, one clinical imaging procedure acquires a dense stack of 0.5×0.5mm FOV images and then, after manual determination of DEJ depth, collects a 5×5mm mosaic at that depth for diagnosis. However, especially in lightly pigmented skin, RCM images have low contrast at the DEJ which makes repeatable, objective visual identification challenging. We have previously published proof of concept for an automated algorithm for DEJ detection in both highly- and lightly-pigmented skin types based on sequential feature segmentation and classification. In lightly-pigmented skin the change of skin texture with depth was detected by the algorithm and used to locate the DEJ. Here we report on further validation of our algorithm on a more extensive collection of 24 image stacks (15 fair skin, 9 dark skin). We compare algorithm performance against classification by three clinical experts. We also evaluate inter-expert consistency among the experts. The average correlation across experts was 0.81 for lightly pigmented skin, indicating the difficulty of the problem. The algorithm achieved epidermis/dermis misclassification rates smaller than 10% (based on 25×25 mm tiles) and average distance from the expert labeled boundaries of ~6.4 μm for fair skin and ~5.3 μm for dark skin, well within average cell size and less than 2x the instrument resolution in the optical axis.

Semi-automated Algorithm for Localization of Dermal/ Epidermal Junction in Reflectance Confocal Microscopy Images of Human Skin

The examination of the dermis/epidermis junction (DEJ) is clinically important for skin cancer diagnosis. Reflectance confocal microscopy (RCM) is an emerging tool for detection of skin cancers in vivo. However, visual localization of the DEJ in RCM images, with high accuracy and repeatability, is challenging, especially in fair skin, due to low contrast, heterogeneous structure and high inter- and intra-subject variability. We recently proposed a semi-automated algorithm to localize the DEJ in z-stacks of RCM images of fair skin, based on feature segmentation and classification. Here we extend the algorithm to dark skin. The extended algorithm first decides the skin type and then applies the appropriate DEJ localization method. In dark skin, strong backscatter from the pigment melanin causes the basal cells above the DEJ to appear with high contrast. To locate those high contrast regions, the algorithm operates on small tiles (regions) and finds the peaks of the smoothed average intensity depth profile of each tile. However, for some tiles, due to heterogeneity, multiple peaks in the depth profile exist and the strongest peak might not be the basal layer peak. To select the correct peak, basal cells are represented with a vector of texture features. The peak with most similar features to this feature vector is selected. The results show that the algorithm detected the skin types correctly for all 17 stacks tested (8 fair, 9 dark). The DEJ detection algorithm achieved an average distance from the ground truth DEJ surface of around 4.7μm for dark skin and around 7-14μm for fair skin.