An improved Internet-based melanoma screening system with dermatologist-like tumor area extraction algorithm

Deep learning algorithms for melanoma detection using dermoscopic images: A systematic review and meta-analysis

BACKGROUND

Melanoma is a serious risk to human health and early identification is vital for treatment success. Deep learning (DL) has the potential to detect cancer using imaging technologies and many studies provide evidence that DL algorithms can achieve high accuracy in melanoma diagnostics.

OBJECTIVES

To critically assess different DL performances in diagnosing melanoma using dermatoscopic images and discuss the relationship between dermatologists and DL.

METHODS

Ovid-Medline, Embase, IEEE Xplore, and the Cochrane Library were systematically searched from inception until 7th December 2021. Studies that reported diagnostic DL model performances in detecting melanoma using dermatoscopic images were included if they had specific outcomes and histopathologic confirmation. Binary diagnostic accuracy data and contingency tables were extracted to analyze outcomes of interest, which included sensitivity (SEN), specificity (SPE), and area under the curve (AUC). Subgroup analyses were performed according to human-machine comparison and cooperation. The study was registered in PROSPERO, CRD42022367824.

RESULTS

2309 records were initially retrieved, of which 37 studies met our inclusion criteria, and 27 provided sufficient data for meta-analytical synthesis. The pooled SEN was 82 % (range 77-86), SPE was 87 % (range 84-90), with an AUC of 0.92 (range 0.89-0.94). Human-machine comparison had pooled AUCs of 0.87 (0.84-0.90) and 0.83 (0.79-0.86) for DL and dermatologists, respectively. Pooled AUCs were 0.90 (0.87-0.93), 0.80 (0.76-0.83), and 0.88 (0.85-0.91) for DL, and junior and senior dermatologists, respectively. Analyses of human-machine cooperation were 0.88 (0.85-0.91) for DL, 0.76 (0.72-0.79) for unassisted, and 0.87 (0.84-0.90) for DL-assisted dermatologists.

CONCLUSIONS

Evidence suggests that DL algorithms are as accurate as senior dermatologists in melanoma diagnostics. Therefore, DL could be used to support dermatologists in diagnostic decision-making. Although, further high-quality, large-scale multicenter studies are required to address the specific challenges associated with medical AI-based diagnostics.

DMpDP: a Diagnostic Multiple-patient DermoFeature Profile store-and-forward teledermoscopy system

Telehealth demand is rapidly growing along with the necessity of providing wide-scale services covering multiple patients at the same time. In this work, the development of a store-and-forward (SAF) teledermoscopy system was considered. The dermoFeatures profile (DP) was proposed to decrease the size of the original dermoscopy image using its most significant features in the form of a newly generated diagonal alignment to generate a small-sized image DP, which is based on the extraction of a weighted intensity-difference frequency (WIDF) features along with morphological features (MOFs). These DPs were assembled to establish a Diagnostic Multiple-patient DermoFeature Profile (DMpDP). Different arrangements are proposed, namely the horizontally aligned, the diagonal-based, and the sequential-based DMpDPs to support the SAF systems. The DMpDPs are then embedded in a recorded patient-information signal (RPS) using a weight factor β to boost the transmitted patient-information signal. The effect of the different transform domains, β values, and number of DPs within the DMpDP were investigated in terms of the diagnostic classification accuracy at the receiver based on the extracted DPs, along with the recorded signal quality evaluation metrics of the recovered RPS. The sequential-based DMpDP achieved the highest classification accuracy, under - 5 dB additive white Gaussian noise, with a realized signal-to-noise ratio of 98.79% during the transmission of 248 DPs using β = 100, and spectral subtraction filtering.

A survey on deep learning for skin lesion segmentation

Skin cancer is a major public health problem that could benefit from computer-aided diagnosis to reduce the burden of this common disease. Skin lesion segmentation from images is an important step toward achieving this goal. However, the presence of natural and artificial artifacts (e.g., hair and air bubbles), intrinsic factors (e.g., lesion shape and contrast), and variations in image acquisition conditions make skin lesion segmentation a challenging task. Recently, various researchers have explored the applicability of deep learning models to skin lesion segmentation. In this survey, we cross-examine 177 research papers that deal with deep learning-based segmentation of skin lesions. We analyze these works along several dimensions, including input data (datasets, preprocessing, and synthetic data generation), model design (architecture, modules, and losses), and evaluation aspects (data annotation requirements and segmentation performance). We discuss these dimensions both from the viewpoint of select seminal works, and from a systematic viewpoint, examining how those choices have influenced current trends, and how their limitations should be addressed. To facilitate comparisons, we summarize all examined works in a comprehensive table as well as an interactive table available online³.

Artificial intelligence in melanoma: A systematic review

BACKGROUND

Melanoma accounts for the majority of skin cancer deaths. Artificial intelligence has been applied in many types of cancers, and in melanoma in recent years. However, no systematic review summarized the application of artificial intelligence in melanoma.

AIMS

This study aims to systematically review previously published articles to explore the application of artificial intelligence in melanoma.

MATERIALS & METHODS

PubMed database was used to search the eligible publications on August 1, 2020. The query term was "artificial intelligence" and "melanoma."

RESULTS

A total of 51 articles were included in this review. Artificial intelligence technique is mainly used in the evaluation of dermoscopic images, other image segmentation and processing, and artificial intelligence diagnosis system.

DISCUSSION

Artificial intelligence is also applied in metastasis prediction, drug response prediction, and prognosis of melanoma. Besides, patients' perspectives of artificial intelligence and collaboration of human and artificial intelligence in melanoma also attracted attention. The query term might not include all articles, and we could not examine the algorithms that were built without publication.

CONCLUSION

The performance of artificial intelligence in melanoma is satisfactory and the future for potential applications is enormous.

DTP-Net: A convolutional neural network model to predict threshold for localizing the lesions on dermatological macro-images

Highly focused images of skin captured with ordinary cameras, called macro-images, are extensively used in dermatology. Being highly focused views, the macro-images contain only lesions and background regions. Hence, the localization of lesions on the macro-images is a simple thresholding problem. However, algorithms that offer an accurate estimate of threshold and retain consistent performance on different dermatological macro-images are rare. A deep learning model, termed 'Deep Threshold Prediction Network (DTP-Net)', is proposed in this paper to address this issue. For training the model, grayscale versions of the macro-images are fed as input to the model, and the corresponding gray-level threshold values at which the Dice similarity index (DSI) between the segmented and the ground-truth images are maximized are defined as the targets. The DTP-Net exhibited the least value of root mean square error for the predicted threshold, compared with 11 state-of-the-art threshold estimation algorithms (such as Otsu's thresholding, Valley emphasized otsu's thresholding, Isodata thresholding, Histogram slope difference distribution-based thresholding, Minimum error thresholding, Poisson's distribution-based minimum error thresholding, Kapur's maximum entropy thresholding, Entropy-weighted otsu's thresholding, Minimum cross-entropy thresholding, Type-2 fuzzy-based thresholding, and Fuzzy entropy thresholding). The DTP-Net could learn the difference between the lesion and background in the intensity space and accurately predict the threshold that separates the lesion from the background. The proposed DTP-Net can be integrated into the segmentation module in automated tools that detect skin cancer from dermatological macro-images.

An EfficientNet-based modified sigmoid transform for enhancing dermatological macro-images of melanoma and nevi skin lesions

BACKGROUND AND OBJECTIVE

During the initial stages, skin lesions may not have sufficient intensity difference or contrast from the background region on dermatological macro-images. The lack of proper light exposure at the time of capturing the image also reduces the contrast. Low contrast between lesion and background regions adversely impacts segmentation. Enhancement techniques for improving the contrast between lesion and background skin on dermatological macro-images are limited in the literature. An EfficientNet-based modified sigmoid transform for enhancing the contrast on dermatological macro-images is proposed to address this issue.

METHODS

A modified sigmoid transform is applied in the HSV color space. The crossover point in the modified sigmoid transform that divides the macro-image into lesion and background is predicted using a modified EfficientNet regressor to exclude manual intervention and subjectivity. The Modified EfficientNet regressor is constructed by replacing the classifier layer in the conventional EfficientNet with a regression layer. Transfer learning is employed to reduce the training time and size of the dataset required to train the modified EfficientNet regressor. For training the modified EfficientNet regressor, a set of value components extracted from the HSV color space representation of the macro-images in the training dataset is fed as input. The corresponding set of ideal crossover points at which the values of Dice similarity coefficient (DSC) between the ground-truth images and the segmented output images obtained from Otsu's thresholding are maximum, is defined as the target.

RESULTS

On images enhanced with the proposed framework, the DSC of segmented results obtained by Otsu's thresholding increased from 0.68 ± 0.34 to 0.81 ± 0.17.

CONCLUSIONS

The proposed algorithm could consistently improve the contrast between lesion and background on a comprehensive set of test images, justifying its applications in automated analysis of dermatological macro-images.

Skin Lesion Segmentation with Improved Convolutional Neural Network

Recently, the incidence of skin cancer has increased considerably and is seriously threatening human health. Automatic detection of this disease, where early detection is critical to human life, is quite challenging. Factors such as undesirable residues (hair, ruler markers), indistinct boundaries, variable contrast, shape differences, and color differences in the skin lesion images make automatic analysis quite difficult. To overcome these challenges, a highly effective segmentation method based on a fully convolutional network (FCN) is presented in this paper. The proposed improved FCN (iFCN) architecture is used for the segmentation of full-resolution skin lesion images without any pre- or post-processing. It is to support the residual structure of the FCN architecture with spatial information. This situation, which creates a more advanced residual system, enables more precise detection of details on the edges of the lesion, and an analysis independent of skin color can be performed. It offers two contributions: determining the center of the lesion and clarifying the edge details despite the undesirable effects. Two publicly available datasets, the IEEE International Symposium on Biomedical Imaging (ISBI) 2017 Challenge and PH2 datasets, are used to evaluate the performance of the iFCN method. The mean Jaccard index is 78.34%, the mean Dice score is 88.64%, and the mean accuracy value is 95.30% for the proposed method for the ISBI 2017 test dataset. Furthermore, the mean Jaccard index is 87.1%, the mean Dice score is 93.02%, and the mean accuracy value is 96.92% for the proposed method for the PH2 test dataset.

Using deep learning for dermatologist-level detection of suspicious pigmented skin lesions from wide-field images

A reported 96,480 people were diagnosed with melanoma in the United States in 2019, leading to 7230 reported deaths. Early-stage identification of suspicious pigmented lesions (SPLs) in primary care settings can lead to improved melanoma prognosis and a possible 20-fold reduction in treatment cost. Despite this clinical and economic value, efficient tools for SPL detection are mostly absent. To bridge this gap, we developed an SPL analysis system for wide-field images using deep convolutional neural networks (DCNNs) and applied it to a 38,283 dermatological dataset collected from 133 patients and publicly available images. These images were obtained from a variety of consumer-grade cameras (15,244 nondermoscopy) and classified by three board-certified dermatologists. Our system achieved more than 90.3% sensitivity (95% confidence interval, 90 to 90.6) and 89.9% specificity (89.6 to 90.2%) in distinguishing SPLs from nonsuspicious lesions, skin, and complex backgrounds, avoiding the need for cumbersome individual lesion imaging. We also present a new method to extract intrapatient lesion saliency (ugly duckling criteria) on the basis of DCNN features from detected lesions. This saliency ranking was validated against three board-certified dermatologists using a set of 135 individual wide-field images from 68 dermatological patients not included in the DCNN training set, exhibiting 82.96% (67.88 to 88.26%) agreement with at least one of the top three lesions in the dermatological consensus ranking. This method could allow for rapid and accurate assessments of pigmented lesion suspiciousness within a primary care visit and could enable improved patient triaging, utilization of resources, and earlier treatment of melanoma.

Deep Learning Approaches Towards Skin Lesion Segmentation and Classification from Dermoscopic Images - A Review

BACKGROUND

Automated intelligent systems for unbiased diagnosis are primary requirement for the pigment lesion analysis. It has gained the attention of researchers in the last few decades. These systems involve multiple phases such as pre-processing, feature extraction, segmentation, classification and post processing. It is crucial to accurately localize and segment the skin lesion. It is observed that recent enhancements in machine learning algorithms and dermoscopic techniques reduced the misclassification rate therefore, the focus towards computer aided systems increased exponentially in recent years. Computer aided diagnostic systems are reliable source for dermatologists to analyze the type of cancer, but it is widely acknowledged that even higher accuracy is needed for computer aided diagnostic systems to be adopted practically in the diagnostic process of life threatening diseases.

INTRODUCTION

Skin cancer is one of the most threatening cancers. It occurs by the abnormal multiplication of cells. The core three types of skin cells are: Squamous, Basal and Melanocytes. There are two wide classes of skin cancer; Melanocytic and non-Melanocytic. It is difficult to differentiate between benign and malignant melanoma, therefore dermatologists sometimes misclassify the benign and malignant melanoma. Melanoma is estimated as 19th most frequent cancer, it is riskier than the Basel and Squamous carcinoma because it rapidly spreads throughout the body. Hence, to lower the death risk, it is critical to diagnose the correct type of cancer in early rudimentary phases. It can occur on any part of body, but it has higher probability to occur on chest, back and legs.

METHODS

The paper presents a review of segmentation and classification techniques for skin lesion detection. Dermoscopy and its features are discussed briefly. After that Image pre-processing techniques are described. A thorough review of segmentation and classification phases of skin lesion detection using deep learning techniques is presented Literature is discussed and a comparative analysis of discussed methods is presented.

CONCLUSION

In this paper, we have presented the survey of more than 100 papers and comparative analysis of state of the art techniques, model and methodologies. Malignant melanoma is one of the most threating and deadliest cancers. Since the last few decades, researchers are putting extra attention and effort in accurate diagnosis of melanoma. The main challenges of dermoscopic skin lesion images are: low contrasts, multiple lesions, irregular and fuzzy borders, blood vessels, regression, hairs, bubbles, variegated coloring and other kinds of distortions. The lack of large training dataset makes these problems even more challenging. Due to recent advancement in the paradigm of deep learning, and specially the outstanding performance in medical imaging, it has become important to review the deep learning algorithms performance in skin lesion segmentation. Here, we have discussed the results of different techniques on the basis of different evaluation parameters such as Jaccard coefficient, sensitivity, specificity and accuracy. And the paper listed down the major achievements in this domain with the detailed discussion of the techniques. In future, it is expected to improve results by utilizing the capabilities of deep learning frameworks with other pre and post processing techniques so reliable and accurate diagnostic systems can be built.

Quantifying acceptable artefact ranges for dermatologic classification algorithms

Background

Many classifiers have been developed that can distinguish different types of skin lesions (e.g., benign nevi, melanoma) with varying degrees of success.^1–5 However, even successfully trained classifiers may perform poorly on images that include artefacts. While problems created by hair and ink markings have been published, quantitative measurements of blur, colour and lighting variations on classification accuracy has not yet been reported to our knowledge.

Objectives

We created a system that measures the impact of various artefacts on machine learning accuracy. Our objectives were to (1) quantitatively identify the most egregious artefacts and (2) demonstrate how to assess a classification algorithm's accuracy when input images include artefacts.

Methods

We injected artefacts into dermatologic images using techniques that could be controlled with a single variable. This allows us to quantitatively evaluate the impact on the accuracy. We trained two convolutional neural networks on two different binary classification tasks and measured the impact on dermoscopy images over a range of parameter values. The area under the curve and specificity‐at‐a‐given‐sensitivity values were measured for each artefact induced at each parameter.

Results

General blur had the strongest negative effect on the melanoma versus other task. Conversely, shifting the hue towards blue had a more pronounced effect on the suspicious versus follow task.

Conclusions

Classifiers should either mitigate artefacts or detect them. Images should be excluded from diagnosis/recommendation when artefacts are present in amounts outside the machine perceived quality range. Failure to do so will reduce accuracy and impede approval from regulatory agencies.

Computer-aided classification of suspicious pigmented lesions using wide-field images

BACKGROUND AND OBJECTIVE

Early identification of melanoma is conducted through whole-body visual examinations to detect suspicious pigmented lesions, a situation that fluctuates in accuracy depending on the experience and time of the examiner. Computer-aided diagnosis tools for skin lesions are typically trained using pre-selected single-lesion images, taken under controlled conditions, which limits their use in wide-field scenes. Here, we propose a computer-aided classifier system with such input conditions to aid in the rapid identification of suspicious pigmented lesions at the primary care level.

METHODS

133 patients with a multitude of skin lesions were recruited for this study. All lesions were examined by a board-certified dermatologist and classified into "suspicious" and "non-suspicious". A new clinical database was acquired and created by taking Wide-Field images of all major body parts with a consumer-grade camera under natural illumination condition and with a consistent source of image variability. 3-8 images were acquired per patient on different sites of the body, and a total of 1759 pigmented lesions were extracted. A machine learning classifier was optimized and build into a computer aided classification system to binary classify each lesion using a suspiciousness score.

RESULTS

In a testing set, our computer-aided classification system achieved a sensitivity of 100% for suspicious pigmented lesions that were later confirmed by dermoscopy examination ("SPL_A") and 83.2% for suspicious pigmented lesions that were not confirmed after examination ("SPL_B"). Sensitivity for non-suspicious lesions was 72.1%, and accuracy was 75.9%. With these results we defined a suspiciousness score that is aligned with common macro-screening (naked eye) practices.

CONCLUSIONS

This work demonstrates that wide-field photography combined with computer-aided classification systems can distinguish suspicious from non-suspicious pigmented lesions, and might be effective to assess the severity of a suspicious pigmented lesions. We believe this approach could be useful to support skin screenings at a population-level.

Fast fully automatic skin lesions segmentation probabilistic with Parzen window

Cutaneous melanoma accounts for over 90% of all melanoma, causing up to 55,500 annual deaths. However, it is a potentially curable type of cancer. Since melanoma is potentially curable, the disease's mortality rate is directly linked to late detection. This work proposes an approach that presents the balance between time and efficiency. This paper proposes the method of fast and automatic segmentation of skin lesions using probabilistic characteristics with the Parzen window (SPPW). The results obtained by the method were based on PH2 and ISIC datasets. The SPPW approach reached the following averages between the two datasets Specificity of 98.55%, Accuracy of 95.48%, Dice of 91.12%, Sensitivity of 88.45%, Mattheus of 87.86%, and Jaccard Index of 84.90%. The highlights of the proposed method are its short average segmentation time per image, and its metrics values, which are often higher than the ones obtained by other methods. Therefore, the SPPW method of segmentation is a quick, viable, and easily accessible option to aid in the diagnosis of diseased skin.

Diagnostic techniques for improved segmentation, feature extraction, and classification of malignant melanoma

A typical diagnosis of malignant melanoma involves three major steps: segmentation of a lesion from the input color image, feature extraction from the separated lesion, and classification to distinguish malignant from benign melanomas based on features obtained. We suggest new methods for segmentation, feature extraction, and classification compared. We replaced edge-imfill method with U-Otsu method for segmentation, the previous features with new features for the criteria ABCD (asymmetry, border irregularity, color variegation, diameter) criteria, and the median thresholding with weighted receiver operating characteristic thresholding for classification. We used 88 melanoma images and expert's segmentation. All the three steps in the suggested method were compared with the steps in the previous method, with respect to sensitivity, specificity, and accuracy of the 88 samples. For segmentation, the previous and the suggested segmentations were also compared assuming the skin cancer expert's segmentation as a ground truth. All three steps resulted in remarkable improvement in the suggested method.

Towards clinical application of image mining: a systematic review on artificial intelligence and radiomics

PURPOSE

The aim of this systematic review was to analyse literature on artificial intelligence (AI) and radiomics, including all medical imaging modalities, for oncological and non-oncological applications, in order to assess how far the image mining research stands from routine medical application. To do this, we applied a trial phases classification inspired from the drug development process.

METHODS

Among the articles we considered for inclusion from PubMed were multimodality AI and radiomics investigations, with a validation analysis aimed at relevant clinical objectives. Quality assessment of selected papers was performed according to the QUADAS-2 criteria. We developed the phases classification criteria for image mining studies.

RESULTS

Overall 34,626 articles were retrieved, 300 were selected applying the inclusion/exclusion criteria, and 171 high-quality papers (QUADAS-2 ≥ 7) were identified and analysed. In 27/171 (16%), 141/171 (82%), and 3/171 (2%) studies the development of an AI-based algorithm, radiomics model, and a combined radiomics/AI approach, respectively, was described. A total of 26/27(96%) and 1/27 (4%) AI studies were classified as phase II and III, respectively. Consequently, 13/141 (9%), 10/141 (7%), 111/141 (79%), and 7/141 (5%) radiomics studies were classified as phase 0, I, II, and III, respectively. All three radiomics/AI studies were categorised as phase II trials.

CONCLUSIONS

The results of the studies are promising but still not mature enough for image mining tools to be implemented in the clinical setting and be widely used. The transfer learning from the well-known drug development process, with some specific adaptations to the image mining discipline could represent the most effective way for radiomics and AI algorithms to become the standard of care tools.

Segmentation of skin lesions in dermoscopy images using fuzzy classification of pixels and histogram thresholding

BACKGROUND AND OBJECTIVE

To ensure proper functioning of a Computer Aided Diagnosis (CAD) system for melanoma detection in dermoscopy images, it is important to accurately detect the border of the lesion. This paper proposes a method developed by the authors to address this problem.

METHODS

The algorithm for segmentation of skin lesions in dermoscopy images is based on fuzzy classification of pixels and subsequent histogram thresholding.

RESULTS

This method participated in the 2016 and 2017 ISBI (International Symposium on Biomedical Imaging) Challenges, hosted by the ISIC (International Skin Imaging Collaboration). It was tested against two public databases containing 379 and 600 images respectively, and compared using the same defined metrics (Accuracy, Dice Coefficient, Jaccard Index, Sensitivity and Specificity) with the rest of participating state-of-the-art work, obtaining good results: (0.934, 0.869, 0.791, 0.870 and 0.978) and (0.884, 0.760, 0.665, 0.869 and 0.923) respectively, ranking 9th and 15th out of a total of 21 and 28 participants respectively using the Jaccard Index (which was the indicator used as a basis for ranking) and the 1st in the 2017 Challenge using the Sensitivity.

CONCLUSION

The method has been proven to be robust and reliable. It's main contribution is the very design of the algorithm, highly innovative, which could also be used to deal with other segmentation problems of a similar nature.

Computer-assisted diagnosis techniques (dermoscopy and spectroscopy-based) for diagnosing skin cancer in adults

BACKGROUND

Early accurate detection of all skin cancer types is essential to guide appropriate management and to improve morbidity and survival. Melanoma and cutaneous squamous cell carcinoma (cSCC) are high-risk skin cancers which have the potential to metastasise and ultimately lead to death, whereas basal cell carcinoma (BCC) is usually localised with potential to infiltrate and damage surrounding tissue. Anxiety around missing early curable cases needs to be balanced against inappropriate referral and unnecessary excision of benign lesions. Computer-assisted diagnosis (CAD) systems use artificial intelligence to analyse lesion data and arrive at a diagnosis of skin cancer. When used in unreferred settings ('primary care'), CAD may assist general practitioners (GPs) or other clinicians to more appropriately triage high-risk lesions to secondary care. Used alongside clinical and dermoscopic suspicion of malignancy, CAD may reduce unnecessary excisions without missing melanoma cases.

OBJECTIVES

To determine the accuracy of CAD systems for diagnosing cutaneous invasive melanoma and atypical intraepidermal melanocytic variants, BCC or cSCC in adults, and to compare its accuracy with that of dermoscopy.

SEARCH METHODS

We undertook a comprehensive search of the following databases from inception up to August 2016: Cochrane Central Register of Controlled Trials (CENTRAL); MEDLINE; Embase; CINAHL; CPCI; Zetoc; Science Citation Index; US National Institutes of Health Ongoing Trials Register; NIHR Clinical Research Network Portfolio Database; and the World Health Organization International Clinical Trials Registry Platform. We studied reference lists and published systematic review articles.

SELECTION CRITERIA

Studies of any design that evaluated CAD alone, or in comparison with dermoscopy, in adults with lesions suspicious for melanoma or BCC or cSCC, and compared with a reference standard of either histological confirmation or clinical follow-up.

DATA COLLECTION AND ANALYSIS

Two review authors independently extracted all data using a standardised data extraction and quality assessment form (based on QUADAS-2). We contacted authors of included studies where information related to the target condition or diagnostic threshold were missing. We estimated summary sensitivities and specificities separately by type of CAD system, using the bivariate hierarchical model. We compared CAD with dermoscopy using (a) all available CAD data (indirect comparisons), and (b) studies providing paired data for both tests (direct comparisons). We tested the contribution of human decision-making to the accuracy of CAD diagnoses in a sensitivity analysis by removing studies that gave CAD results to clinicians to guide diagnostic decision-making.

MAIN RESULTS

We included 42 studies, 24 evaluating digital dermoscopy-based CAD systems (Derm-CAD) in 23 study cohorts with 9602 lesions (1220 melanomas, at least 83 BCCs, 9 cSCCs), providing 32 datasets for Derm-CAD and seven for dermoscopy. Eighteen studies evaluated spectroscopy-based CAD (Spectro-CAD) in 16 study cohorts with 6336 lesions (934 melanomas, 163 BCC, 49 cSCCs), providing 32 datasets for Spectro-CAD and six for dermoscopy. These consisted of 15 studies using multispectral imaging (MSI), two studies using electrical impedance spectroscopy (EIS) and one study using diffuse-reflectance spectroscopy. Studies were incompletely reported and at unclear to high risk of bias across all domains. Included studies inadequately address the review question, due to an abundance of low-quality studies, poor reporting, and recruitment of highly selected groups of participants.Across all CAD systems, we found considerable variation in the hardware and software technologies used, the types of classification algorithm employed, methods used to train the algorithms, and which lesion morphological features were extracted and analysed across all CAD systems, and even between studies evaluating CAD systems. Meta-analysis found CAD systems had high sensitivity for correct identification of cutaneous invasive melanoma and atypical intraepidermal melanocytic variants in highly selected populations, but with low and very variable specificity, particularly for Spectro-CAD systems. Pooled data from 22 studies estimated the sensitivity of Derm-CAD for the detection of melanoma as 90.1% (95% confidence interval (CI) 84.0% to 94.0%) and specificity as 74.3% (95% CI 63.6% to 82.7%). Pooled data from eight studies estimated the sensitivity of multispectral imaging CAD (MSI-CAD) as 92.9% (95% CI 83.7% to 97.1%) and specificity as 43.6% (95% CI 24.8% to 64.5%). When applied to a hypothetical population of 1000 lesions at the mean observed melanoma prevalence of 20%, Derm-CAD would miss 20 melanomas and would lead to 206 false-positive results for melanoma. MSI-CAD would miss 14 melanomas and would lead to 451 false diagnoses for melanoma. Preliminary findings suggest CAD systems are at least as sensitive as assessment of dermoscopic images for the diagnosis of invasive melanoma and atypical intraepidermal melanocytic variants. We are unable to make summary statements about the use of CAD in unreferred populations, or its accuracy in detecting keratinocyte cancers, or its use in any setting as a diagnostic aid, because of the paucity of studies.

AUTHORS' CONCLUSIONS

In highly selected patient populations all CAD types demonstrate high sensitivity, and could prove useful as a back-up for specialist diagnosis to assist in minimising the risk of missing melanomas. However, the evidence base is currently too poor to understand whether CAD system outputs translate to different clinical decision-making in practice. Insufficient data are available on the use of CAD in community settings, or for the detection of keratinocyte cancers. The evidence base for individual systems is too limited to draw conclusions on which might be preferred for practice. Prospective comparative studies are required that evaluate the use of already evaluated CAD systems as diagnostic aids, by comparison to face-to-face dermoscopy, and in participant populations that are representative of those in which the test would be used in practice.

Medical Image Classification Based on Deep Features Extracted by Deep Model and Statistic Feature Fusion with Multilayer Perceptron‬

Medical image classification is a key technique of Computer-Aided Diagnosis (CAD) systems. Traditional methods rely mainly on the shape, color, and/or texture features as well as their combinations, most of which are problem-specific and have shown to be complementary in medical images, which leads to a system that lacks the ability to make representations of high-level problem domain concepts and that has poor model generalization ability. Recent deep learning methods provide an effective way to construct an end-to-end model that can compute final classification labels with the raw pixels of medical images. However, due to the high resolution of the medical images and the small dataset size, deep learning models suffer from high computational costs and limitations in the model layers and channels. To solve these problems, in this paper, we propose a deep learning model that integrates Coding Network with Multilayer Perceptron (CNMP), which combines high-level features that are extracted from a deep convolutional neural network and some selected traditional features. The construction of the proposed model includes the following steps. First, we train a deep convolutional neural network as a coding network in a supervised manner, and the result is that it can code the raw pixels of medical images into feature vectors that represent high-level concepts for classification. Second, we extract a set of selected traditional features based on background knowledge of medical images. Finally, we design an efficient model that is based on neural networks to fuse the different feature groups obtained in the first and second step. We evaluate the proposed approach on two benchmark medical image datasets: HIS2828 and ISIC2017. We achieve an overall classification accuracy of 90.1% and 90.2%, respectively, which are higher than the current successful methods.

Statistical Detection of Colors in Dermoscopic Images With a Texton-Based Estimation of Probabilities

Color has great diagnostic significance in dermatoscopy. Several diagnosis methods are based on the colors detected within a lesion. Malignant lesions frequently show more than three colors, whereas in benign lesions, three or fewer colors are usually observed. Black, red, white, and blue-gray are found more frequently in melanomas than in benign nevi. In this paper, a method to automatically identify the colors of a lesion is presented. A color label identification problem is proposed and solved by maximizing the posterior probability of a pixel to belong to a label, given its color value and the neighborhood color values. The main contribution of this paper is the estimation of the different terms involved in the computation of this probability. Two evaluations are performed on a database of 200 dermoscopic images. The first one evaluates if all the colors detected in a lesion are indeed present in it. The second analyzes if each pixel within a lesion is assigned the correct color label. The results show that the proposed method performs correctly and outperforms other methods, with an average F-measure of 0.89, an accuracy of 0.90, and a Spearman correlation of 0.831.

Supervised Saliency Map Driven Segmentation of Lesions in Dermoscopic Images

Lesion segmentation is the first step in most automatic melanoma recognition systems. Deficiencies and difficulties in dermoscopic images such as color inconstancy, hair occlusion, dark corners, and color charts make lesion segmentation an intricate task. In order to detect the lesion in the presence of these problems, we propose a supervised saliency detection method tailored for dermoscopic images based on the discriminative regional feature integration (DRFI). A DRFI method incorporates multilevel segmentation, regional contrast, property, background descriptors, and a random forest regressor to create saliency scores for each region in the image. In our improved saliency detection method, mDRFI, we have added some new features to regional property descriptors. Also, in order to achieve more robust regional background descriptors, a thresholding algorithm is proposed to obtain a new pseudo-background region. Findings reveal that mDRFI is superior to DRFI in detecting the lesion as the salient object in dermoscopic images. The proposed overall lesion segmentation framework uses detected saliency map to construct an initial mask of the lesion through thresholding and postprocessing operations. The initial mask is then evolving in a level set framework to fit better on the lesion's boundaries. The results of evaluation tests on three public datasets show that our proposed segmentation method outperforms the other conventional state-of-the-art segmentation algorithms and its performance is comparable with most recent approaches that are based on deep convolutional neural networks.

Melanoma detection algorithm based on feature fusion

A Computer Aided-Diagnosis (CAD) System for melanoma diagnosis usually makes use of different types of features to characterize the lesions. The features are often combined into a single vector that can belong to a high dimensional space (early fusion). However, it is not clear if this is the optimal strategy and works on other fields have shown that early fusion has some limitations. In this work, we address this issue and investigate which is the best approach to combine different features comparing early and late fusion. Experiments carried on the datasets PH2 (single source) and EDRA (multi source) show that late fusion performs better, leading to classification scores of Sensitivity = 98% and Specificity = 90% (PH(2)) and Sensitivity = 83% and Specificity = 76% (EDRA).

Skin Lesion Analysis towards Melanoma Detection Using Deep Learning Network

Skin lesions are a severe disease globally. Early detection of melanoma in dermoscopy images significantly increases the survival rate. However, the accurate recognition of melanoma is extremely challenging due to the following reasons: low contrast between lesions and skin, visual similarity between melanoma and non-melanoma lesions, etc. Hence, reliable automatic detection of skin tumors is very useful to increase the accuracy and efficiency of pathologists. In this paper, we proposed two deep learning methods to address three main tasks emerging in the area of skin lesion image processing, i.e., lesion segmentation (task 1), lesion dermoscopic feature extraction (task 2) and lesion classification (task 3). A deep learning framework consisting of two fully convolutional residual networks (FCRN) is proposed to simultaneously produce the segmentation result and the coarse classification result. A lesion index calculation unit (LICU) is developed to refine the coarse classification results by calculating the distance heat-map. A straight-forward CNN is proposed for the dermoscopic feature extraction task. The proposed deep learning frameworks were evaluated on the ISIC 2017 dataset. Experimental results show the promising accuracies of our frameworks, i.e., 0.753 for task 1, 0.848 for task 2 and 0.912 for task 3 were achieved.

Texture based skin lesion abruptness quantification to detect malignancy

BACKGROUND

Abruptness of pigment patterns at the periphery of a skin lesion is one of the most important dermoscopic features for detection of malignancy. In current clinical setting, abrupt cutoff of a skin lesion determined by an examination of a dermatologist. This process is subjective, nonquantitative, and error-prone. We present an improved computational model to quantitatively measure abruptness of a skin lesion over our previous method. To achieve this, we quantitatively analyze the texture features of a region within the lesion boundary. This region is bounded by an interior border line of the lesion boundary which is determined using level set propagation (LSP) method. This method provides a fast border contraction without a need for extensive boolean operations. Then, we build feature vectors of homogeneity, standard deviation of pixel values, and mean of the pixel values of the region between the contracted border and the original border. These vectors are then classified using neural networks (NN) and SVM classifiers.

RESULTS

As lower homogeneity indicates sharp cutoffs, suggesting melanoma, we carried out our experiments on two dermoscopy image datasets, which consist of 800 benign and 200 malignant melanoma cases. LSP method helped produce better results than Kaya et al., 2016 study. By using texture homogeneity at the periphery of a lesion border determined by LSP, as a classification results, we obtained 87% f1-score and 78% specificity; that we obtained better results than in the previous study. We also compared the performances of two different NN classifiers and support vector machine classifier. The best results obtained using combination of RGB color spaces with the fully-connected multi-hidden layer NN.

CONCLUSIONS

Computational results also show that skin lesion abrupt cutoff is a reliable indicator of malignancy. Results show that computational model of texture homogeneity along the periphery of skin lesion borders based on LSP is an effective way of quantitatively measuring abrupt cutoff of a lesion.

Results of the 2016 International Skin Imaging Collaboration International Symposium on Biomedical Imaging challenge: Comparison of the accuracy of computer algorithms to dermatologists for the diagnosis of melanoma from dermoscopic images

BACKGROUND

Computer vision may aid in melanoma detection.

OBJECTIVE

We sought to compare melanoma diagnostic accuracy of computer algorithms to dermatologists using dermoscopic images.

METHODS

We conducted a cross-sectional study using 100 randomly selected dermoscopic images (50 melanomas, 44 nevi, and 6 lentigines) from an international computer vision melanoma challenge dataset (n = 379), along with individual algorithm results from 25 teams. We used 5 methods (nonlearned and machine learning) to combine individual automated predictions into "fusion" algorithms. In a companion study, 8 dermatologists classified the lesions in the 100 images as either benign or malignant.

RESULTS

The average sensitivity and specificity of dermatologists in classification was 82% and 59%. At 82% sensitivity, dermatologist specificity was similar to the top challenge algorithm (59% vs. 62%, P = .68) but lower than the best-performing fusion algorithm (59% vs. 76%, P = .02). Receiver operating characteristic area of the top fusion algorithm was greater than the mean receiver operating characteristic area of dermatologists (0.86 vs. 0.71, P = .001).

LIMITATIONS

The dataset lacked the full spectrum of skin lesions encountered in clinical practice, particularly banal lesions. Readers and algorithms were not provided clinical data (eg, age or lesion history/symptoms). Results obtained using our study design cannot be extrapolated to clinical practice.

CONCLUSION

Deep learning computer vision systems classified melanoma dermoscopy images with accuracy that exceeded some but not all dermatologists.

Incorporating Colour Information for Computer-Aided Diagnosis of Melanoma from Dermoscopy Images: A Retrospective Survey and Critical Analysis

Cutaneous melanoma is the most life-threatening form of skin cancer. Although advanced melanoma is often considered as incurable, if detected and excised early, the prognosis is promising. Today, clinicians use computer vision in an increasing number of applications to aid early detection of melanoma through dermatological image analysis (dermoscopy images, in particular). Colour assessment is essential for the clinical diagnosis of skin cancers. Due to this diagnostic importance, many studies have either focused on or employed colour features as a constituent part of their skin lesion analysis systems. These studies range from using low-level colour features, such as simple statistical measures of colours occurring in the lesion, to availing themselves of high-level semantic features such as the presence of blue-white veil, globules, or colour variegation in the lesion. This paper provides a retrospective survey and critical analysis of contributions in this research direction.

Dermatologist-like feature extraction from skin lesion for improved asymmetry classification in PH2 database

Asymmetry is one of key characteristics for early diagnosis of melanoma according to medical algorithms such as (ABCD, CASH etc.). Besides shape information, cues such as irregular distribution of colors and structures within the lesion area are assessed by dermatologists to determine lesion asymmetry. Motivated by the clinical practices, we have used Kullback-Leibler divergence of color histogram and Structural Similarity metric as a measures of these irregularities. We have presented performance of several classifiers using these features on publicly available PH2 dataset. The obtained result shows better asymmetry classification than available literature. Besides being a new benchmark, the proposed technique can be used for early diagnosis of melanoma by both clinical experts and other automated diagnosis systems.

Computational methods for the image segmentation of pigmented skin lesions: A review

BACKGROUND AND OBJECTIVES

Because skin cancer affects millions of people worldwide, computational methods for the segmentation of pigmented skin lesions in images have been developed in order to assist dermatologists in their diagnosis. This paper aims to present a review of the current methods, and outline a comparative analysis with regards to several of the fundamental steps of image processing, such as image acquisition, pre-processing and segmentation.

METHODS

Techniques that have been proposed to achieve these tasks were identified and reviewed. As to the image segmentation task, the techniques were classified according to their principle.

RESULTS

The techniques employed in each step are explained, and their strengths and weaknesses are identified. In addition, several of the reviewed techniques are applied to macroscopic and dermoscopy images in order to exemplify their results.

CONCLUSIONS

The image segmentation of skin lesions has been addressed successfully in many studies; however, there is a demand for new methodologies in order to improve the efficiency.

Biologically inspired skin lesion segmentation using a geodesic active contour technique

BACKGROUND/PURPOSE

Computer-aided diagnosis of skin cancer requires accurate lesion segmentation, which must overcome noise such as hair, skin color variations, and ambient light variability.

METHODS

A biologically inspired geodesic active contour (GAC) technique is used for lesion segmentation. The algorithm presented here employs automatic contour initialization close to the actual lesion boundary, overcoming the 'sticking' at minimum local energy spots caused by noise artifacts such as hair. The border is significantly smoothed to mimic natural lesions. In addition, features that mimic biological parameters include spectral image subtraction and removal of peninsulas and inlets. Multiple boundary choices borders are created by parameter options used at different steps. These choices can allow future improvement over the basic default border.

RESULTS

The basic GAC algorithm was tested on 100 images (30 melanomas and 70 benign lesions), yielding a median XOR border error of 6.7%, comparable to the median inter-dermatologist XOR border error (7.4%), and lower than the gradient vector flow snake median XOR error of 14.2% on the same image set. On a difficult low-contrast border set of 1238 images, which included 350 non-melanocytic lesions, a median XOR error of 23.9% is obtained.

CONCLUSION

GAC techniques show promise in attaining the goal of automatic skin lesion segmentation.

PH² - a dermoscopic image database for research and benchmarking

The increasing incidence of melanoma has recently promoted the development of computer-aided diagnosis systems for the classification of dermoscopic images. Unfortunately, the performance of such systems cannot be compared since they are evaluated in different sets of images by their authors and there are no public databases available to perform a fair evaluation of multiple systems. In this paper, a dermoscopic image database, called PH², is presented. The PH² database includes the manual segmentation, the clinical diagnosis, and the identification of several dermoscopic structures, performed by expert dermatologists, in a set of 200 dermoscopic images. The PH² database will be made freely available for research and benchmarking purposes.