Skin image illumination modeling and chromophore identification for melanoma diagnosis

Deep learning-based optical approach for skin analysis of melanin and hemoglobin distribution

SIGNIFICANCE

Melanin and hemoglobin have been measured as important diagnostic indicators of facial skin conditions for aesthetic and diagnostic purposes. Commercial clinical equipment provides reliable analysis results, but it has several drawbacks: exclusive to the acquisition system, expensive, and computationally intensive.

AIM

We propose an approach to alleviate those drawbacks using a deep learning model trained to solve the forward problem of light-tissue interactions. The model is structurally extensible for various light sources and cameras and maintains the input image resolution for medical applications.

APPROACH

A facial image is divided into multiple patches and decomposed into melanin, hemoglobin, shading, and specular maps. The outputs are reconstructed into a facial image by solving the forward problem over skin areas. As learning progresses, the difference between the reconstructed image and input image is reduced, resulting in the melanin and hemoglobin maps becoming closer to their distribution of the input image.

RESULTS

The proposed approach was evaluated on 30 subjects using the professional clinical system, VISIA VAESTRO. The correlation coefficients for melanin and hemoglobin were found to be 0.932 and 0.857, respectively. Additionally, this approach was applied to simulated images with varying amounts of melanin and hemoglobin.

CONCLUSION

The proposed approach showed high correlation with the clinical system for analyzing melanin and hemoglobin distribution, indicating its potential for accurate diagnosis. Further calibration studies using clinical equipment can enhance its diagnostic ability. The structurally extensible model makes it a promising tool for various image acquisition conditions.

Three-dimensional representation of triple spectral line imaging data as an option for noncontact skin diagnostics

SIGNIFICANCE

Skin malformations in dermatology are mostly evaluated subjectively, based on a doctor's experience and visual perception; an option for objective quantitative skin assessment is camera-based spectrally selective diagnostics. Multispectral imaging is a technique capable to provide information about concentrations of the absorbing chromophores and their distribution over the malformation in a noncontact way. Conversion of spectral images into distribution maps of chromophores can be performed by means of the modified Beer-Lambert law. However, such distribution maps represent only single specific cases, therefore, some extensive method for data comparison is needed.

AIM

This study aims to develop a more informative approach for identification and characterization of skin malformations using three-dimensional (3D) representation of triple spectral line imaging data.

APPROACH

The 3D-representation method is experimentally tested on eight different skin pathology types, including both benign and malignant pathologies; an imaging device ensuring uniform three laser line (448, 532, and 659 nm) illumination is used. Three spectral line images are extracted from a single snapshot RGB image data, with subsequent calculation of attenuation coefficients for each working wavelength at every image pixel and represented as 3D graphs. Skin chromophore content variations in malformations are represented in a similar way.

RESULTS

Clinical measurement results for 99 skin pathologies, including basal cell carcinomas, melanoma, dermal nevi, combined nevi, junctional nevi, blue nevi, seborrheic keratosis, and hemangiomas. They are presented as 3D spectral attenuation maps exhibiting specific individual features for each group of pathologies. Along with intensity attenuation maps, 3D maps for content variations of three main skin chromophores (melanin, oxyhemoglobin, and deoxyhemoglobin), calculated in frame of a model based on modified Beer-Lambert law, are also presented. Advantages and disadvantages of the proposed data representation method are discussed.

CONCLUSIONS

The described 3D-representation method of triple spectral line imaging data shows promising potential for objective quantitative noncontact diagnosis of skin pathologies.

Visualizing veins from color images under varying illuminations for medical applications

SIGNIFICANCE

Effective vein visualization is critically important for several clinical procedures, such as venous blood sampling and intravenous injection. Existing technologies using infrared device or ultrasound rely on professional equipment and are not suitable for daily medical care. A regression-based vein visualization method is proposed.

AIM

We visualize veins from conventional RGB images to provide assistance in venipuncture procedures as well as clinical diagnosis of some venous insufficiency.

APPROACH

The RGB images taken by digital cameras are first transformed to spectral reflectance images using Wiener estimation. Multiple regression analysis is then applied to derive the relationship between spectral reflectance and the concentrations of pigments. Monte Carlo simulation is adopted to get prior information. Finally, vein patterns are visualized from the spatial distribution of pigments. To minimize the effect of illumination on skin color, light correction and shading removal operations are performed in advance.

RESULTS

Experimental results from inner forearms of 60 subjects show the effectiveness of the regression-based method. Subjective and objective evaluations demonstrate that the clarity and completeness of vein patterns can be improved by light correction and shading removal.

CONCLUSIONS

Vein patterns can be successfully visualized from RGB images without any professional equipment. The proposed method can assist in venipuncture procedures. It also shows promising potential to be used in clinical diagnosis and treatment of some venous insufficiency.

Melanoma detection using adversarial training and deep transfer learning

Skin lesion datasets consist predominantly of normal samples with only a small percentage of abnormal ones, giving rise to the class imbalance problem. Also, skin lesion images are largely similar in overall appearance owing to the low inter-class variability. In this paper, we propose a two-stage framework for automatic classification of skin lesion images using adversarial training and transfer learning toward melanoma detection. In the first stage, we leverage the inter-class variation of the data distribution for the task of conditional image synthesis by learning the inter-class mapping and synthesizing under-represented class samples from the over-represented ones using unpaired image-to-image translation. In the second stage, we train a deep convolutional neural network for skin lesion classification using the original training set combined with the newly synthesized under-represented class samples. The training of this classifier is carried out by minimizing the focal loss function, which assists the model in learning from hard examples, while down-weighting the easy ones. Experiments conducted on a dermatology image benchmark demonstrate the superiority of our proposed approach over several standard baseline methods, achieving significant performance improvements. Interestingly, we show through feature visualization and analysis that our method leads to context based lesion assessment that can reach an expert dermatologist level.

A review of melanin sensor devices

Knowing how readily the skin produces melanin is invaluable in reducing photochemical and phototherapy overtreatment in dermatology and also in reducing the risk of actinic skin damage and skin cancer from excessive radiant light exposure. The commonly used Fitzpatrick skin type (FST) classification scale is often used to subjectively assess ultraviolet light sensitivity and susceptibility to sunburn following significant sunlight exposure. However, the FST scale falls short in the assessment of nonwhite skin types. Alternatively, commercially available melanin sensor devices, called melanometers, can be used to objectively quantify useful skin parameters such as the epidermal melanin concentration (EMC). This study reviews commercially available melanometers and their use in quantifying epidermal melanin concentration (EMC) and the individual maximum safe radiant exposure (IMSRE) for an individual in clinical, workplace and community settings.

Computational texture features of dermoscopic images and their link to the descriptive terminology: A survey

Computer-extracted texture features are relevant to diagnose cutaneous lesions such as melanomas. Our goal is to set a relationship between a well-established descriptive terminology, which describes the attributes of dermoscopic structures based on their aspect rather than their underlying causes, and the computational methods to extract texture-based features. By tackling this problem, we can ascertain what indicators used by dermatologists are reflected in the extracted texture features. We first review the state-of-the-art models for texture extraction in dermoscopic images. By comparing the methods' performance and goals, we conclude that (I) a single color space does not seem to give performances as good as using several ones, thus the latter is reasonable (II) the optimal number of extracted features seems to vary depending on the method's goal, and extracting a large number of features can lead to a loss of models robustness (III) methods such as GLCM, Sobel or Law energy filters are mainly used to capture local properties to detect specific dermoscopic structures (IV) methods that extract local and global features, like Gabor wavelets or SPT, tend to be used to analyze the presence of certain patterns of dermoscopic structures, e.g. globular, reticular, etc.

Accuracy of Computer-Aided Diagnosis of Melanoma: A Meta-analysis

IMPORTANCE

The recent advances in the field of machine learning have raised expectations that computer-aided diagnosis will become the standard for the diagnosis of melanoma.

OBJECTIVE

To critically review the current literature and compare the diagnostic accuracy of computer-aided diagnosis with that of human experts.

DATA SOURCES

The MEDLINE, arXiv, and PubMed Central databases were searched to identify eligible studies published between January 1, 2002, and December 31, 2018.

STUDY SELECTION

Studies that reported on the accuracy of automated systems for melanoma were selected. Search terms included melanoma, diagnosis, detection, computer aided, and artificial intelligence.

DATA EXTRACTION AND SYNTHESIS

Evaluation of the risk of bias was performed using the QUADAS-2 tool, and quality assessment was based on predefined criteria. Data were analyzed from February 1 to March 10, 2019.

MAIN OUTCOMES AND MEASURES

Summary estimates of sensitivity and specificity and summary receiver operating characteristic curves were the primary outcomes.

RESULTS

The literature search yielded 1694 potentially eligible studies, of which 132 were included and 70 offered sufficient information for a quantitative analysis. Most studies came from the field of computer science. Prospective clinical studies were rare. Combining the results for automated systems gave a melanoma sensitivity of 0.74 (95% CI, 0.66-0.80) and a specificity of 0.84 (95% CI, 0.79-0.88). Sensitivity was lower in studies that used independent test sets than in those that did not (0.51; 95% CI, 0.34-0.69 vs 0.82; 95% CI, 0.77-0.86; P < .001); however, the specificity was similar (0.83; 95% CI, 0.71-0.91 vs 0.85; 95% CI, 0.80-0.88; P = .67). In comparison with dermatologists' diagnosis, computer-aided diagnosis showed similar sensitivities and a 10 percentage points lower specificity, but the difference was not statistically significant. Studies were heterogeneous and substantial risk of bias was found in all but 4 of the 70 studies included in the quantitative analysis.

CONCLUSIONS AND RELEVANCE

Although the accuracy of computer-aided diagnosis for melanoma detection is comparable to that of experts, the real-world applicability of these systems is unknown and potentially limited owing to overfitting and the risk of bias of the studies at hand.

Deep-learning-based, computer-aided classifier developed with a small dataset of clinical images surpasses board-certified dermatologists in skin tumour diagnosis

BACKGROUND

Application of deep-learning technology to skin cancer classification can potentially improve the sensitivity and specificity of skin cancer screening, but the number of training images required for such a system is thought to be extremely large.

OBJECTIVES

To determine whether deep-learning technology could be used to develop an efficient skin cancer classification system with a relatively small dataset of clinical images.

METHODS

A deep convolutional neural network (DCNN) was trained using a dataset of 4867 clinical images obtained from 1842 patients diagnosed with skin tumours at the University of Tsukuba Hospital from 2003 to 2016. The images consisted of 14 diagnoses, including both malignant and benign conditions. Its performance was tested against 13 board-certified dermatologists and nine dermatology trainees.

RESULTS

The overall classification accuracy of the trained DCNN was 76·5%. The DCNN achieved 96·3% sensitivity (correctly classified malignant as malignant) and 89·5% specificity (correctly classified benign as benign). Although the accuracy of malignant or benign classification by the board-certified dermatologists was statistically higher than that of the dermatology trainees (85·3% ± 3·7% and 74·4% ± 6·8%, P < 0·01), the DCNN achieved even greater accuracy, as high as 92·4% ± 2·1% (P < 0·001).

CONCLUSIONS

We have developed an efficient skin tumour classifier using a DCNN trained on a relatively small dataset. The DCNN classified images of skin tumours more accurately than board-certified dermatologists. Collectively, the current system may have capabilities for screening purposes in general medical practice, particularly because it requires only a single clinical image for classification.

Multispectral imaging for highly accurate analysis of tumour-infiltrating lymphocytes in primary melanoma

AIMS

The quality and quantity of the infiltration of immune cells into tumour tissues have substantial impacts on patients' clinical outcomes, and are associated with response to immunotherapy. Therefore, the precise analysis of tumour-infiltrating lymphocytes (TILs) is becoming an important additional pathological biomarker. Analysis of TILs is usually performed semiquantitatively by pathologists on haematoxylin and eosin-stained or immunostained tissue sections. However, automated quantification outperforms semiquantitative approaches, and is becoming the standard. Owing to the presence of melanin pigment, this approach is seriously hampered in melanoma, because the spectrum of melanin lies close to that of commonly used immunohistochemical stains. Aim of this study is to overcome the technical issues due to the presence of melanin for an automated and accurate quantification of TILs in melanoma.

METHODS AND RESULTS

Here, we successfully applied a novel multispectral imaging (MSI) technique to enumerate T cells in human primary melanomas. This microscopy technique combines imaging with spectroscopy to obtain both quantitative expression data and the tissue distributions of different cellular markers. We demonstrate that MSI allows complete and accurate analysis of TILs, successfully avoiding the blurring of images by melanin pigments, in whole tissue slide primary melanoma lesions, which could otherwise not be accurately detected by conventional digital image methodologies.

CONCLUSIONS

Our study highlights the potential of MSI for accurate assessment of immune cell infiltrates, including those in notoriously difficult tissues, such as pigmented melanomas. Quantification of tumour infiltration by different immune cell types is crucial in the search for new biomarkers to predict patient responses to immunotherapies. Our findings show that this innovative microscopy technique is an important extension of the armamentarium of pathologists.

Screening neonatal jaundice based on the sclera color of the eye using digital photography

A new screening technique for neonatal jaundice is proposed exploiting the yellow discoloration in the sclera. It involves taking digital photographs of newborn infants' eyes (n = 110) and processing the pixel colour values of the sclera to predict the total serum bilirubin (TSB) levels. This technique has linear and rank correlation coefficients of 0.75 and 0.72 (both p<0.01) with the measured TSB. The mean difference ( ± SD) is 0.00 ± 41.60 µmol/l. The receiver operating characteristic curve shows that this technique can identify subjects with TSB above 205 µmol/l with sensitivity of 1.00 and specificity of 0.50, showing its potential as a screening device.