Computational methods for pigmented skin lesion classification in images: review and future trends

A Narrative Review: Opportunities and Challenges in Artificial Intelligence Skin Image Analyses Using Total Body Photography

Artificial intelligence (AI) algorithms for skin lesion classification have reported accuracy at par with and even outperformance of expert dermatologists in experimental settings. However, the majority of algorithms do not represent real-world clinical approach where skin phenotype and clinical background information are considered. We review the current state of AI for skin lesion classification and present opportunities and challenges when applied to total body photography (TBP). AI in TBP analysis presents opportunities for intrapatient assessment of skin phenotype and holistic risk assessment by incorporating patient-level metadata, although challenges exist for protecting patient privacy in algorithm development and improving explainable AI methods.

Precision in Dermatology: Developing an Optimal Feature Selection Framework for Skin Lesion Classification

Melanoma is widely recognized as one of the most lethal forms of skin cancer, with its incidence showing an upward trend in recent years. Nonetheless, the timely detection of this malignancy substantially enhances the likelihood of patients' long-term survival. Several computer-based methods have recently been proposed, in the pursuit of diagnosing skin lesions at their early stages. Despite achieving some level of success, there still remains a margin of error that the machine learning community considers to be an unresolved research challenge. The primary objective of this study was to maximize the input feature information by combining multiple deep models in the first phase, and then to avoid noisy and redundant information by downsampling the feature set, using a novel evolutionary feature selection technique, in the second phase. By maintaining the integrity of the original feature space, the proposed idea generated highly discriminant feature information. Recent deep models, including Darknet53, DenseNet201, InceptionV3, and InceptionResNetV2, were employed in our study, for the purpose of feature extraction. Additionally, transfer learning was leveraged, to enhance the performance of our approach. In the subsequent phase, the extracted feature information from the chosen pre-existing models was combined, with the aim of preserving maximum information, prior to undergoing the process of feature selection, using a novel entropy-controlled gray wolf optimization (ECGWO) algorithm. The integration of fusion and selection techniques was employed, initially to incorporate the feature vector with a high level of information and, subsequently, to eliminate redundant and irrelevant feature information. The effectiveness of our concept is supported by an assessment conducted on three benchmark dermoscopic datasets: PH2, ISIC-MSK, and ISIC-UDA. In order to validate the proposed methodology, a comprehensive evaluation was conducted, including a rigorous comparison to established techniques in the field.

A survey, review, and future trends of skin lesion segmentation and classification

The Computer-aided Diagnosis or Detection (CAD) approach for skin lesion analysis is an emerging field of research that has the potential to alleviate the burden and cost of skin cancer screening. Researchers have recently indicated increasing interest in developing such CAD systems, with the intention of providing a user-friendly tool to dermatologists to reduce the challenges encountered or associated with manual inspection. This article aims to provide a comprehensive literature survey and review of a total of 594 publications (356 for skin lesion segmentation and 238 for skin lesion classification) published between 2011 and 2022. These articles are analyzed and summarized in a number of different ways to contribute vital information regarding the methods for the development of CAD systems. These ways include: relevant and essential definitions and theories, input data (dataset utilization, preprocessing, augmentations, and fixing imbalance problems), method configuration (techniques, architectures, module frameworks, and losses), training tactics (hyperparameter settings), and evaluation criteria. We intend to investigate a variety of performance-enhancing approaches, including ensemble and post-processing. We also discuss these dimensions to reveal their current trends based on utilization frequencies. In addition, we highlight the primary difficulties associated with evaluating skin lesion segmentation and classification systems using minimal datasets, as well as the potential solutions to these difficulties. Findings, recommendations, and trends are disclosed to inform future research on developing an automated and robust CAD system for skin lesion analysis.

Skin lesion classification using multi-resolution empirical mode decomposition and local binary pattern

Skin cancer is the most common type of cancer in many parts of the world. As skin cancers start as skin lesions, it is important to identify precancerous skin lesions early. In this paper we propose an image based skin lesion identification to classify seven different classes of skin lesions. First, Multi Resolution Empirical Mode Decomposition (MREMD) is used to decompose each skin lesion image into a few Bidimensional intrinsic mode functions (BIMF). MREMD is a simplified bidimensional empirical mode decomposition (BEMD) that employs downsampling and upsampling (interpolation) in the upper and lower envelope formation to speed up the decomposition process. A few BIMFs are extracted from the image using MREMD. The next step is to locate the lesion or the region of interest (ROI) in the image using active contour. Then Local Binary Pattern (LBP) is applied to the ROI of the image and its first BIMF to extract a total of 512 texture features from the lesion area. In the training phase, texture features of seven different classes of skin lesions are used to train an Artificial Neural Network (ANN) classifier. Altogether, 490 images from HAM10000 dataset are used to train the ANN. Then the accuracy of the approach is evaluated using 315 test images that are different from the training images. The test images are taken from the same dataset and each test image contains one type of lesion from the seven types that are classified. From each test image, 512 texture features are extracted from the lesion area and introduced to the classifier to determine its class. The proposed method achieves an overall classification rate of 98.9%.

Dermoscopic image segmentation based on Pyramid Residual Attention Module

We propose a stacked convolutional neural network incorporating a novel and efficient pyramid residual attention (PRA) module for the task of automatic segmentation of dermoscopic images. Precise segmentation is a significant and challenging step for computer-aided diagnosis technology in skin lesion diagnosis and treatment. The proposed PRA has the following characteristics: First, we concentrate on three widely used modules in the PRA. The purpose of the pyramid structure is to extract the feature information of the lesion area at different scales, the residual means is aimed to ensure the efficiency of model training, and the attention mechanism is used to screen effective features maps. Thanks to the PRA, our network can still obtain precise boundary information that distinguishes healthy skin from diseased areas for the blurred lesion areas. Secondly, the proposed PRA can increase the segmentation ability of a single module for lesion regions through efficient stacking. The third, we incorporate the idea of encoder-decoder into the architecture of the overall network. Compared with the traditional networks, we divide the segmentation procedure into three levels and construct the pyramid residual attention network (PRAN). The shallow layer mainly processes spatial information, the middle layer refines both spatial and semantic information, and the deep layer intensively learns semantic information. The basic module of PRAN is PRA, which is enough to ensure the efficiency of the three-layer architecture network. We extensively evaluate our method on ISIC2017 and ISIC2018 datasets. The experimental results demonstrate that PRAN can obtain better segmentation performance comparable to state-of-the-art deep learning models under the same experiment environment conditions.

MobileSkin: Classification of Skin Lesion Images Acquired Using Mobile Phone-Attached Hand-Held Dermoscopes

Dermoscopy is the visual examination of the skin under a polarized or non-polarized light source. By using dermoscopic equipment, many lesion patterns that are invisible under visible light can be clearly distinguished. Thus, more accurate decisions can be made regarding the treatment of skin lesions. The use of images collected from a dermoscope has both increased the performance of human examiners and allowed the development of deep learning models. The availability of large-scale dermoscopic datasets has allowed the development of deep learning models that can classify skin lesions with high accuracy. However, most dermoscopic datasets contain images that were collected from digital dermoscopic devices, as these devices are frequently used for clinical examination. However, dermatologists also often use non-digital hand-held (optomechanical) dermoscopes. This study presents a dataset consisting of dermoscopic images taken using a mobile phone-attached hand-held dermoscope. Four deep learning models based on the MobileNetV1, MobileNetV2, NASNetMobile, and Xception architectures have been developed to classify eight different lesion types using this dataset. The number of images in the dataset was increased with different data augmentation methods. The models were initialized with weights that were pre-trained on the ImageNet dataset, and then they were further fine-tuned using the presented dataset. The most successful models on the unseen test data, MobileNetV2 and Xception, had performances of 89.18% and 89.64%. The results were evaluated with the 5-fold cross-validation method and compared. Our method allows for automated examination of dermoscopic images taken with mobile phone-attached hand-held dermoscopes.

SCDNet: A Deep Learning-Based Framework for the Multiclassification of Skin Cancer Using Dermoscopy Images

Skin cancer is a deadly disease, and its early diagnosis enhances the chances of survival. Deep learning algorithms for skin cancer detection have become popular in recent years. A novel framework based on deep learning is proposed in this study for the multiclassification of skin cancer types such as Melanoma, Melanocytic Nevi, Basal Cell Carcinoma and Benign Keratosis. The proposed model is named as SCDNet which combines Vgg16 with convolutional neural networks (CNN) for the classification of different types of skin cancer. Moreover, the accuracy of the proposed method is also compared with the four state-of-the-art pre-trained classifiers in the medical domain named Resnet 50, Inception v3, AlexNet and Vgg19. The performance of the proposed SCDNet classifier, as well as the four state-of-the-art classifiers, is evaluated using the ISIC 2019 dataset. The accuracy rate of the proposed SDCNet is 96.91% for the multiclassification of skin cancer whereas, the accuracy rates for Resnet 50, Alexnet, Vgg19 and Inception-v3 are 95.21%, 93.14%, 94.25% and 92.54%, respectively. The results showed that the proposed SCDNet performed better than the competing classifiers.

Can Dermoscopy and Ultrasonography be Considered a Prognostic Tool in Management of Psoriasis?

Background:

Dermoscopy can reliably predict the diagnosis of plaque psoriasis. Ultrasonography has been increasingly used in dermatology in inflammatory diseases like psoriasis as a tool for evaluation. Hence, this study was done to evaluate the role of dermoscopy and ultrasonography as prognostic aid in plaque psoriasis.

Aims and Objectives:

To study the sonographic and dermoscopic findings of clinically diagnosed psoriatic lesions and the changes in the psoriatic lesions if any, with the treatment. How these findings can be utilized to assess the prognosis in these patients.

Materials and Methods:

The present study comprised 50 patients with clinically diagnosed plaque psoriasis. Lesions were assessed with Dino-Lite digital microscope AM7515MZT, followed by ultrasonography using a 15 MHz probe, and findings were recorded. All the patients included in this study were given appropriate treatment (topical/systemic) for 6 weeks and were followed up twice i.e., at 3 weeks and 6 weeks after initiating treatment.

Results:

Whitish scales were the most common scale color seen in our study seen in 35/50 patients (70%). All the vascular structures were reddish, red dots and globules being the predominant type and with the improvement of the lesions, brown structures increased. A total of 28 (56%) patients had a regular pattern of vessel arrangement. Mean capillary size was 0.097 ± 0.012 mm that reduced to 0.075 ± 0.019 mm at the end of the third week and 0.027 ± 0.032 mm at the end of 6 weeks. In ultrasonographic assessment, mean epidermal thickness reduced from 0.1008 to 0.0764 cm at third week and 0.068 cm at the sixth week, and mean dermal thickness reduced from 0.2692cm to 0.1906cm at the third week and then to 0.1906cm 0.1806cm at the sixth week. In our study, clinical improvement preceded dermoscopic improvement. Newer structures identified in the study are a perifollicular arrangement of capillaries and the presence of lacunar structures in the healing lesions.

Conclusion:

The scale distribution, capillary number, and capillary size in dermoscopic assessment, and epidermal and dermal thickness in ultrasonography showed statistically significant changes with treatment and thus may be taken as the prognostic indicators. Thus, both these noninvasive modalities may be useful in the therapeutic monitoring of plaque psoriasis.

Skin Lesion Classification Based on Surface Fractal Dimensions and Statistical Color Cluster Features Using an Ensemble of Machine Learning Techniques

Simple Summary

This study aimed to investigate the efficacy of implementation of novel skin surface fractal dimension features as an auxiliary diagnostic method for melanoma recognition. We therefore examined the skin lesion classification accuracy of the kNN-CV algorithm and of the proposed Radial basis function neural network model. We found an increased accuracy of classification when the fractal analysis is added to the classical color distribution analysis. Our results indicate that by using a reliable classifier, more opportunities exist to detect timely cancerous skin lesions.

Abstract

(1) Background: An approach for skin cancer recognition and classification by implementation of a novel combination of features and two classifiers, as an auxiliary diagnostic method, is proposed. (2) Methods: The predictions are made by k-nearest neighbor with a 5-fold cross validation algorithm and a neural network model to assist dermatologists in the diagnosis of cancerous skin lesions. As a main contribution, this work proposes a descriptor that combines skin surface fractal dimension and relevant color area features for skin lesion classification purposes. The surface fractal dimension is computed using a 2D generalization of Higuchi’s method. A clustering method allows for the selection of the relevant color distribution in skin lesion images by determining the average percentage of color areas within the nevi and melanoma lesion areas. In a classification stage, the Higuchi fractal dimensions (HFDs) and the color features are classified, separately, using a kNN-CV algorithm. In addition, these features are prototypes for a Radial basis function neural network (RBFNN) classifier. The efficiency of our algorithms was verified by utilizing images belonging to the 7-Point, Med-Node, and PH2 databases; (3) Results: Experimental results show that the accuracy of the proposed RBFNN model in skin cancer classification is 95.42% for 7-Point, 94.71% for Med-Node, and 94.88% for PH2, which are all significantly better than that of the kNN algorithm. (4) Conclusions: 2D Higuchi’s surface fractal features have not been previously used for skin lesion classification purpose. We used fractal features further correlated to color features to create a RBFNN classifier that provides high accuracies of classification.

Intelligent Dermatologist Tool for Classifying Multiple Skin Cancer Subtypes by Incorporating Manifold Radiomics Features Categories

The rates of skin cancer (SC) are rising every year and becoming a critical health issue worldwide. SC's early and accurate diagnosis is the key procedure to reduce these rates and improve survivability. However, the manual diagnosis is exhausting, complicated, expensive, prone to diagnostic error, and highly dependent on the dermatologist's experience and abilities. Thus, there is a vital need to create automated dermatologist tools that are capable of accurately classifying SC subclasses. Recently, artificial intelligence (AI) techniques including machine learning (ML) and deep learning (DL) have verified the success of computer-assisted dermatologist tools in the automatic diagnosis and detection of SC diseases. Previous AI-based dermatologist tools are based on features which are either high-level features based on DL methods or low-level features based on handcrafted operations. Most of them were constructed for binary classification of SC. This study proposes an intelligent dermatologist tool to accurately diagnose multiple skin lesions automatically. This tool incorporates manifold radiomics features categories involving high-level features such as ResNet-50, DenseNet-201, and DarkNet-53 and low-level features including discrete wavelet transform (DWT) and local binary pattern (LBP). The results of the proposed intelligent tool prove that merging manifold features of different categories has a high influence on the classification accuracy. Moreover, these results are superior to those obtained by other related AI-based dermatologist tools. Therefore, the proposed intelligent tool can be used by dermatologists to help them in the accurate diagnosis of the SC subcategory. It can also overcome manual diagnosis limitations, reduce the rates of infection, and enhance survival rates.

Deepening into the suitability of using pre-trained models of ImageNet against a lightweight convolutional neural network in medical imaging: an experimental study

Transfer learning (TL) has been widely utilized to address the lack of training data for deep learning models. Specifically, one of the most popular uses of TL has been for the pre-trained models of the ImageNet dataset. Nevertheless, although these pre-trained models have shown an effective performance in several domains of application, those models may not offer significant benefits in all instances when dealing with medical imaging scenarios. Such models were designed to classify a thousand classes of natural images. There are fundamental differences between these models and those dealing with medical imaging tasks regarding learned features. Most medical imaging applications range from two to ten different classes, where we suspect that it would not be necessary to employ deeper learning models. This paper investigates such a hypothesis and develops an experimental study to examine the corresponding conclusions about this issue. The lightweight convolutional neural network (CNN) model and the pre-trained models have been evaluated using three different medical imaging datasets. We have trained the lightweight CNN model and the pre-trained models with two scenarios which are with a small number of images once and a large number of images once again. Surprisingly, it has been found that the lightweight model trained from scratch achieved a more competitive performance when compared to the pre-trained model. More importantly, the lightweight CNN model can be successfully trained and tested using basic computational tools and provide high-quality results, specifically when using medical imaging datasets.

Skin Lesion Segmentation with Improved Convolutional Neural Network

Recently, the incidence of skin cancer has increased considerably and is seriously threatening human health. Automatic detection of this disease, where early detection is critical to human life, is quite challenging. Factors such as undesirable residues (hair, ruler markers), indistinct boundaries, variable contrast, shape differences, and color differences in the skin lesion images make automatic analysis quite difficult. To overcome these challenges, a highly effective segmentation method based on a fully convolutional network (FCN) is presented in this paper. The proposed improved FCN (iFCN) architecture is used for the segmentation of full-resolution skin lesion images without any pre- or post-processing. It is to support the residual structure of the FCN architecture with spatial information. This situation, which creates a more advanced residual system, enables more precise detection of details on the edges of the lesion, and an analysis independent of skin color can be performed. It offers two contributions: determining the center of the lesion and clarifying the edge details despite the undesirable effects. Two publicly available datasets, the IEEE International Symposium on Biomedical Imaging (ISBI) 2017 Challenge and PH2 datasets, are used to evaluate the performance of the iFCN method. The mean Jaccard index is 78.34%, the mean Dice score is 88.64%, and the mean accuracy value is 95.30% for the proposed method for the ISBI 2017 test dataset. Furthermore, the mean Jaccard index is 87.1%, the mean Dice score is 93.02%, and the mean accuracy value is 96.92% for the proposed method for the PH2 test dataset.

Machine Learning and Deep Learning Methods for Skin Lesion Classification and Diagnosis: A Systematic Review

Computer-aided systems for skin lesion diagnosis is a growing area of research. Recently, researchers have shown an increasing interest in developing computer-aided diagnosis systems. This paper aims to review, synthesize and evaluate the quality of evidence for the diagnostic accuracy of computer-aided systems. This study discusses the papers published in the last five years in ScienceDirect, IEEE, and SpringerLink databases. It includes 53 articles using traditional machine learning methods and 49 articles using deep learning methods. The studies are compared based on their contributions, the methods used and the achieved results. The work identified the main challenges of evaluating skin lesion segmentation and classification methods such as small datasets, ad hoc image selection and racial bias.

Melanoma Detection Using Spatial and Spectral Analysis on Superpixel Graphs

Melanoma is the most fatal type of skin cancer. Detection of melanoma from dermoscopic images in an early stage is critical for improving survival rates. Numerous image processing methods have been devised to discriminate between melanoma and benign skin lesions. Previous studies show that the detection performance depends significantly on the skin lesion image representations and features. In this work, we propose a melanoma detection approach that combines graph-theoretic representations with conventional dermoscopic image features to enhance the detection performance. Instead of using individual pixels of skin lesion images as nodes for complex graph representations, superpixels are generated from the skin lesion images and are then used as graph nodes in a superpixel graph. An edge of such a graph connects two adjacent superpixels where the edge weight is a function of the distance between feature descriptors of these superpixels. A graph signal can be defined by assigning to each graph node the output of some single-valued function of the associated superpixel descriptor. Features are extracted from weighted and unweighted graph models in the vertex domain at both local and global scales and in the spectral domain using the graph Fourier transform (GFT). Other features based on color, geometry and texture are extracted from the skin lesion images. Several conventional and ensemble classifiers have been trained and tested on different combinations from those features using two datasets of dermoscopic images from the International Skin Imaging Collaboration (ISIC) archive. The proposed system achieved an AUC of [Formula: see text], an accuracy of [Formula: see text], a specificity of [Formula: see text] and a sensitivity of [Formula: see text].

The effects of skin lesion segmentation on the performance of dermatoscopic image classification

BACKGROUND AND OBJECTIVE

Malignant melanoma (MM) is one of the deadliest types of skin cancer. Analysing dermatoscopic images plays an important role in the early detection of MM and other pigmented skin lesions. Among different computer-based methods, deep learning-based approaches and in particular convolutional neural networks have shown excellent classification and segmentation performances for dermatoscopic skin lesion images. These models can be trained end-to-end without requiring any hand-crafted features. However, the effect of using lesion segmentation information on classification performance has remained an open question.

METHODS

In this study, we explicitly investigated the impact of using skin lesion segmentation masks on the performance of dermatoscopic image classification. To do this, first, we developed a baseline classifier as the reference model without using any segmentation masks. Then, we used either manually or automatically created segmentation masks in both training and test phases in different scenarios and investigated the classification performances. The different scenarios included approaches that exploited the segmentation masks either for cropping of skin lesion images or removing the surrounding background or using the segmentation masks as an additional input channel for model training.

RESULTS

Evaluated on the ISIC 2017 challenge dataset which contained two binary classification tasks (i.e. MM vs. all and seborrheic keratosis (SK) vs. all) and based on the derived area under the receiver operating characteristic curve scores, we observed four main outcomes. Our results show that 1) using segmentation masks did not significantly improve the MM classification performance in any scenario, 2) in one of the scenarios (using segmentation masks for dilated cropping), SK classification performance was significantly improved, 3) removing all background information by the segmentation masks significantly degraded the overall classification performance, and 4) in case of using the appropriate scenario (using segmentation for dilated cropping), there is no significant difference of using manually or automatically created segmentation masks.

CONCLUSIONS

We systematically explored the effects of using image segmentation on the performance of dermatoscopic skin lesion classification.

Deep learning based classification of facial dermatological disorders

Common properties of dermatological diseases are mostly lesions with abnormal pattern and skin color (usually redness). Therefore, dermatology is one of the most appropriate areas in medicine for automated diagnosis from images using pattern recognition techniques to provide accurate, objective, early diagnosis and interventions. Also, automated techniques provide diagnosis without depending on location and time. In addition, the number of patients in dermatology departments and costs of dermatologist visits can be reduced. Therefore, in this work, an automated method is proposed to classify dermatological diseases from color digital photographs. Efficiency of the proposed approach is provided by 2 stages. In the 1st stage, lesions are detected and extracted by using a variational level set technique after noise reduction and intensity normalization steps. In the 2nd stage, lesions are classified using a pre-trained DenseNet201 architecture with an efficient loss function. In this study, five common facial dermatological diseases are handled since they also cause anxiety, depression and even suicide death. The main contributions provided by this work can be identified as follows: (i) A comprehensive survey about the state-of-the-art works on classifications of dermatological diseases using deep learning; (ii) A new fully automated lesion detection and segmentation based on level sets; (iii) A new adaptive, hybrid and non-symmetric loss function; (iv) Using a pre-trained DenseNet201 structure with the new loss function to classify skin lesions; (v) Comparative evaluations of ten convolutional networks for skin lesion classification. Experimental results indicate that the proposed approach can classify lesions with high performance (95.24% accuracy).

Transfer learning using a multi-scale and multi-network ensemble for skin lesion classification

BACKGROUND AND OBJECTIVE

Skin cancer is among the most common cancer types in the white population and consequently computer aided methods for skin lesion classification based on dermoscopic images are of great interest. A promising approach for this uses transfer learning to adapt pre-trained convolutional neural networks (CNNs) for skin lesion diagnosis. Since pre-training commonly occurs with natural images of a fixed image resolution and these training images are usually significantly smaller than dermoscopic images, downsampling or cropping of skin lesion images is required. This however may result in a loss of useful medical information, while the ideal resizing or cropping factor of dermoscopic images for the fine-tuning process remains unknown.

METHODS

We investigate the effect of image size for skin lesion classification based on pre-trained CNNs and transfer learning. Dermoscopic images from the International Skin Imaging Collaboration (ISIC) skin lesion classification challenge datasets are either resized to or cropped at six different sizes ranging from 224 × 224 to 450 × 450. The resulting classification performance of three well established CNNs, namely EfficientNetB0, EfficientNetB1 and SeReNeXt-50 is explored. We also propose and evaluate a multi-scale multi-CNN (MSM-CNN) fusion approach based on a three-level ensemble strategy that utilises the three network architectures trained on cropped dermoscopic images of various scales.

RESULTS

Our results show that image cropping is a better strategy compared to image resizing delivering superior classification performance at all explored image scales. Moreover, fusing the results of all three fine-tuned networks using cropped images at all six scales in the proposed MSM-CNN approach boosts the classification performance compared to a single network or a single image scale. On the ISIC 2018 skin lesion classification challenge test set, our MSM-CNN algorithm yields a balanced multi-class accuracy of 86.2% making it the currently second ranked algorithm on the live leaderboard.

CONCLUSIONS

We confirm that the image size has an effect on skin lesion classification performance when employing transfer learning of CNNs. We also show that image cropping results in better performance compared to image resizing. Finally, a straightforward ensembling approach that fuses the results from images cropped at six scales and three fine-tuned CNNs is shown to lead to the best classification performance.

Automatic skin lesion classification based on mid-level feature learning

Dermoscopic images are widely used for melanoma detection. Many existing works based on traditional classification methods and deep learning models have been proposed for automatic skin lesion analysis. The traditional classification methods use hand-crafted features as input. However, due to the strong visual similarity between different classes of skin lesions and complex skin conditions, the hand-crafted features are not discriminative enough and fail in many cases. Recently, deep convolutional neural networks (CNN) have gained popularity since they can automatically learn optimal features during the training phase. Different from existing works, a novel mid-level feature learning method for skin lesion classification task is proposed in this paper. In this method, skin lesion segmentation is first performed to detect the regions of interest (ROI) of skin lesion images. Next, pretrained neural networks including ResNet and DenseNet are used as the feature extractors for the ROI images. Instead of using the extracted features directly as input of classifiers, the proposed method obtains the mid-level feature representations by utilizing the relationships among different image samples based on distance metric learning. The learned feature representation is a soft discriminative descriptor, having more tolerance to the hard samples and hence is more robust to the large intra-class difference and inter-class similarity. Experimental results demonstrate advantages of the proposed mid-level features, and the proposed method obtains state-of-the-art performance compared with the existing CNN based methods.

Diagnosis support of sickle cell anemia by classifying red blood cell shape in peripheral blood images

Red blood cell (RBC) deformation is the consequence of several diseases, including sickle cell anemia, which causes recurring episodes of pain and severe pronounced anemia. Monitoring patients with these diseases involves the observation of peripheral blood samples under a microscope, a time-consuming procedure. Moreover, a specialist is required to perform this technique, and owing to the subjective nature of the observation of isolated RBCs, the error rate is high. In this paper, we propose an automated method for differentially enumerating RBCs that uses peripheral blood smear image analysis. In this method, the objects of interest in the image are segmented using a Chan-Vese active contour model. An analysis is then performed to classify the RBCs, also called erythrocytes, as normal or elongated or having other deformations, using the basic shape analysis descriptors: circular shape factor (CSF) and elliptical shape factor (ESF). To analyze cells that become partially occluded in a cluster during sample preparation, an elliptical adjustment is performed to allow the analysis of erythrocytes with discoidal and elongated shapes. The images of patient blood samples used in the study were acquired by a clinical laboratory specialist in the Special Hematology Department of the "Dr. Juan Bruno Zayas" General Hospital in Santiago de Cuba. A comparison of the results obtained by the proposed method in our experiments with those obtained by some state-of-the-art methods showed that the proposed method is superior for the diagnosis of sickle cell anemia. This superiority is achieved for evidenced by the obtained F-measure value (0.97 for normal cells and 0.95 for elongated ones) and several overall multiclass performance measures. The results achieved by the proposed method are suitable for the purpose of clinical treatment and diagnostic support of sickle cell anemia. We present a new method to obtain erythrocyte shape classification using peripheral blood smear sample images. The aim of the method is to segment the cells, to separate clusters and classify cells (circulars, elongated and others). We compared our method with state-of the-art. Results showed that our method with is superior for the diagnosis support of sickle cell anemia.

Multilevel and Multiscale Feature Aggregation in Deep Networks for Facial Constitution Classification

Constitution classification is the basis and core content of TCM constitution research. In order to improve the accuracy of constitution classification, this paper proposes a multilevel and multiscale features aggregation method within the convolutional neural network, which consists of four steps. First, it uses the pretrained VGG16 as the basic network and then refines the network structure through supervised feature learning so as to capture local image features. Second, it extracts the image features of different layers from the fine-tuned VGG16 model, which are then dimensionally reduced by principal component analysis (PCA). Third, it uses another pretrained NASNetMobile network for supervised feature learning, where the previous layer features of the global average pooling layer are outputted. Similarly, these features are dimensionally reduced by PCA and then are fused with the features of different layers in VGG16 after the PCA. Finally, all features are aggregated with the fully connected layers of the fine-tuned VGG16, and then the constitution classification is performed. The conducted experiments show that using the multilevel and multiscale feature aggregation is very effective in the constitution classification, and the accuracy on the test dataset reaches 69.61%.

The impact of patient clinical information on automated skin cancer detection

Skin cancer is one of the most common types of cancer worldwide. Over the past few years, different approaches have been proposed to deal with automated skin cancer detection. Nonetheless, most of them are based only on dermoscopic images and do not take into account the patient clinical information, an important clue towards clinical diagnosis. In this work, we present an approach to fill this gap. First, we introduce a new dataset composed of clinical images, collected using smartphones, and clinical data related to the patient. Next, we propose a straightforward method that includes an aggregation mechanism in well-known deep learning models to combine features from images and clinical data. Last, we carry out experiments to compare the models' performance with and without using this mechanism. The results present an improvement of approximately 7% in balanced accuracy when the aggregation method is applied. Overall, the impact of clinical data on models' performance is significant and shows the importance of including these features on automated skin cancer detection.

Integration of morphological preprocessing and fractal based feature extraction with recursive feature elimination for skin lesion types classification

BACKGROUND AND OBJECTIVE

Skin cancer is the commonest form of cancer in the worldwide population. Non-invasive and non-contact imaging modalities are being used for the screening of melanoma and other cutaneous malignancies to endorse early detection and prevention of the disease. Traditionally it has been a problem for medical personnel to differentiate melanoma, dysplastic nevi and basal cell carcinoma (BCC) diseases from one another due to the confusing appearance and similarity in the characteristics of the pigmented lesions. The paper reports an integrated method developed for identifying these skin diseases from the dermoscopic images.

METHODS

The proposed integrated computer-aided method has been employed for the identification of each of these diseases using recursive feature elimination (RFE) based layered structured multiclass image classification technique. Prior to the classification, different quantitative features have been extracted by analyzing the shape, the border irregularity, the texture and the color of the skin lesions, using different image processing tools. Primarily, a combination of gray level co-occurrence matrix (GLCM) and a proposed fractal-based regional texture analysis (FRTA) algorithm has been used for the quantification of textural information. The performance of the framework has been evaluated using a layered structure classification model using support vector machine (SVM) classifier with radial basis function (RBF).

RESULTS

The performance of the morphological skin lesion segmentation algorithm has been evaluated by estimating the pixel level sensitivity (Sen) of 0.9172, 0.9788 specificity (Spec), 0.9521 accuracy (ACU), along with the image similarity measuring indices as Jaccard similarity index (JSI) of 0.8562 and Dice similarity coefficient (DSC) of 0.9142 with respect to the corresponding ground truth (GT) images. The quantitative features extracted from the proposed feature extraction algorithms have been employed for the proposed multi-class skin disease identification. The proposed layered structure identifies all the three classes of skin diseases with a highly acceptable classification accuracy of 98.99%, 97.54% and 99.65% for melanoma, dysplastic nevi and BCC respectively.

CONCLUSION

To overcome the difficulties of proper diagnosis of diseases based on visual evaluation, the proposed integrated system plays an important role by quantifying the effective features and identifying the diseases with higher degree of accuracy. This combined approach of quantitative and qualitative analysis not only increases the diagnostic accuracy, but also provides some important information not obtainable from qualitative assessment alone.

Digital hair segmentation using hybrid convolutional and recurrent neural networks architecture

BACKGROUND AND OBJECTIVE

Skin melanoma is one of the major health problems in many countries. Dermatologists usually diagnose melanoma by visual inspection of moles. Digital hair removal can provide a non-invasive way to remove hair and hair-like regions as a pre-processing step for skin lesion images. Hair removal has two main steps: hair segmentation and hair gaps inpainting. However, hair segmentation is a challenging task which requires manual tuning of thresholding parameters. Hard-coded threshold leads to over-segmentation (false positives) which in return changes the textural integrity of lesions and or under-segmentation (false negatives) which leaves hair traces and artefacts which affect subsequent diagnosis. Additionally, dermal hair exhibits different characteristics: thin; overlapping; faded; occluded and overlaid on textured lesions.

METHODS

In this presented paper, we proposed a deep learning approach based on a hybrid network of convolutional and recurrent layers for hair segmentation using weakly labelled data. We utilised the deep encoded features for accurate detection and delineation of hair in skin images. The encoded features are then fed into recurrent neural network layers to encode the spatial dependencies between disjointed patches. Experiments are conducted on a publicly available dataset, called "Towards Melanoma Detection: Challenge". We chose two metrics to evaluate the produced segmentation masks. The first metric is the Jaccard Index which penalises false positives and false negatives. The second metric is the tumour disturb pattern which assesses the overall effect over the lesion texture due to unnecessary inpainting as a result of over segmentation. The qualitative and quantitative evaluations are employed to compare the proposed technique with state-of-the-art methods.

RESULTS

The proposed approach showed superior segmentation accuracy as demonstrated by a Jaccard Index of 77.8% in comparison to a 66.5% reported by the state-of-the-art method. We also achieved tumour disturb pattern as low as 14% compared to 23% for the state-of-the-art method.

CONCLUSION

The hybrid architecture for segmentation was able to accurately delineate and segment the hair from the background including lesions and the skin using weakly labelled ground truth for training.

Groundwater Potential Mapping Using an Integrated Ensemble of Three Bivariate Statistical Models with Random Forest and Logistic Model Tree Models

In the future, groundwater will be the major source of water for agriculture, drinking and food production as a result of global climate change. With increasing population growth, demand for groundwater has increased. Therefore, sustainable groundwater storage management has become a major challenge. This study introduces a new ensemble data mining approach with bivariate statistical models, using FR (frequency ratio), CF (certainty factor), EBF (evidential belief function), RF (random forest) and LMT (logistic model tree) to prepare a groundwater potential map (GPM) for the Booshehr plain. In the first step, 339 wells were chosen and randomly split into two groups with groundwater yields above 11 m3/h. A total of 238 wells (70%) were used for model training, and 101 wells (30%) were used for model validation. Then, 15 effective factors, including topographic and hydrologic factors, were selected for the modeling. The accuracy of the groundwater potential maps was determined using the ROC (receiver operating characteristic) curve and the AUC (area under the curve). The results show that the AUC obtained using the CF-RF, EBF-RF, FR-RF, CF-LMT, EBF-LMT and FR-LMT methods were 0.927, 0.924, 0.917, 0.906, 0.885 and 0.83, respectively. Therefore, it can be inferred that the ensemble of bivariate statistic and data mining models can improve the effectiveness of the methods in developing a groundwater potential map.

A New Approach to Border Irregularity Assessment with Application in Skin Pathology

The border irregularity assessment of tissue structures is an important step in medical diagnostics (e.g., in dermatoscopy, pathology, and cardiology). The diagnostic criteria based on the degree of uniformity and symmetry of border irregularities are particularly vital in dermatopathology, to distinguish between benign and malignant skin lesions. We propose a new method for the segmentation of individual border projections and measuring their morphometry. It is based mainly on analyzing the curvature of the object’s border to identify endpoints of projection bases, and on analyzing object’s skeleton in the graph representation to identify bases of projections and their location along the object’s main axis. The proposed segmentation method has been tested on 25 skin whole slide images of common melanocytic lesions. In total, 825 out of 992 (83%) manually segmented retes (projections of epidermis) were detected correctly and the Jaccard similarity coefficient for the task of detecting retes was 0.798. Experimental results verified the effectiveness of the proposed approach. Our method is particularly well suited for assessing the border irregularity of human epidermis and thus could help develop computer-aided diagnostic algorithms for skin cancer detection.

Skin Cancer Classification Using Convolutional Neural Networks: Systematic Review

Background

State-of-the-art classifiers based on convolutional neural networks (CNNs) were shown to classify images of skin cancer on par with dermatologists and could enable lifesaving and fast diagnoses, even outside the hospital via installation of apps on mobile devices. To our knowledge, at present there is no review of the current work in this research area.

Objective

This study presents the first systematic review of the state-of-the-art research on classifying skin lesions with CNNs. We limit our review to skin lesion classifiers. In particular, methods that apply a CNN only for segmentation or for the classification of dermoscopic patterns are not considered here. Furthermore, this study discusses why the comparability of the presented procedures is very difficult and which challenges must be addressed in the future.

Methods

We searched the Google Scholar, PubMed, Medline, ScienceDirect, and Web of Science databases for systematic reviews and original research articles published in English. Only papers that reported sufficient scientific proceedings are included in this review.

Results

We found 13 papers that classified skin lesions using CNNs. In principle, classification methods can be differentiated according to three principles. Approaches that use a CNN already trained by means of another large dataset and then optimize its parameters to the classification of skin lesions are the most common ones used and they display the best performance with the currently available limited datasets.

Conclusions

CNNs display a high performance as state-of-the-art skin lesion classifiers. Unfortunately, it is difficult to compare different classification methods because some approaches use nonpublic datasets for training and/or testing, thereby making reproducibility difficult. Future publications should use publicly available benchmarks and fully disclose methods used for training to allow comparability.

A novel cumulative level difference mean based GLDM and modified ABCD features ranked using eigenvector centrality approach for four skin lesion types classification

BACKGROUND AND OBJECTIVE

Melanoma is one of the major death causes while basal cell carcinoma (BCC) is the utmost incident skin lesion type. At their early stages, medical experts may be confused between both types with benign nevus and pigmented benign keratoses (BKL). This inspired the current study to develop an accurate automated, user-friendly skin lesion identification system.

METHODS

The current work targets a novel discrimination technique of four pre-mentioned skin lesion classes. A novel proposed texture feature, named cumulative level-difference mean (CLDM) based on the gray-level difference method (GLDM) is extracted. The asymmetry, border irregularity, color variation and diameter are summed up as the ABCD rule feature vector is originally used to classify the melanoma from benign lesions. The proposed method improved the ABCD rule to also classify BCC and BKL by using the proposed modified-ABCD feature vector. In the modified set of ABCD features, each border feature, such as compact index, fractal dimension, and edge abruptness is considered a separate feature. Then, the composite feature vector having the pre-mentioned features is ranked using the Eigenvector Centrality (ECFS) feature ranking method. The ranked features are then classified by a cubic support vector machine for different numbers of selected features.

RESULTS

The proposed CLDM texture features combined with the ranked ABCD features achieved outstanding performance to classify the four targeted classes (melanoma, BCC, nevi and BKL). The results report 100% outstanding performance of the sensitivity, accuracy and specificity per each class compared to other features when using the highest seven ranked features.

CONCLUSIONS

The proposed system established that Melanoma, BCC, nevus and BKL are efficiently classified using cubic SVM with the new feature set. In addition, the comparative studies proved the superiority of the cubic SVM to classify the four classes.

Lymphoma images analysis using morphological and non-morphological descriptors for classification

Mantle cell lymphoma, follicular lymphoma and chronic lymphocytic leukemia are the principle subtypes of the non-Hodgkin lymphomas. The diversity of clinical presentations and the histopathological features have made diagnosis of such lymphomas a complex task for specialists. Computer aided diagnosis systems employ computational vision and image processing techniques, which contribute toward the detection, diagnosis and prognosis of digitised images of histological samples. Studies aimed at improving the understanding of morphological and non-morphological features for classification of lymphoma remain a challenge in this area. This work presents a new approach for the classification of information extracted from morphological and non-morphological features of nuclei of lymphoma images. We extracted data of the RGB model of the image and employed contrast limited adaptive histogram equalisation and 2D order-statistics filter methods to enhance the contrast and remove noise. The regions of interest were identified by the global thresholding method. The flood-fill and watershed techniques were used to remove the small false positive regions. The area, extent, perimeter, convex area, solidity, eccentricity, equivalent diameter, minor axis and major axis measurements were computed for the regions detected in the nuclei. In the feature selection stage, we applied the ANOVA, Ansari-Bradley and Wilcoxon rank sum methods. Finally, we employed the polynomial, support vector machine, random forest and decision tree classifiers in order to evaluate the performance of the proposed approach. The non-morphological features achieved higher AUC and AC values for all cases: the obtained rates were between 95% and 100%. These results are relevant, especially when we consider the difficulties of clinical practice in distinguishing the studied groups. The proposed approach is useful as an automated protocol for the diagnosis of lymphoma histological tissue.