Melanoma Is Skin Deep: A 3D Reconstruction Technique for Computerized Dermoscopic Skin Lesion Classification

CR-Conformer: a fusion network for clinical skin lesion classification

Deep convolutional neural network (DCNN) models have been widely used to diagnose skin lesions, and some of them have achieved diagnostic results comparable to or even better than dermatologists. Most publicly available skin lesion datasets used to train DCNN were dermoscopic images. Expensive dermoscopic equipment is rarely available in rural clinics or small hospitals in remote areas. Therefore, it is of great significance to rely on clinical images for computer-aided diagnosis of skin lesions. This paper proposes an improved dual-branch fusion network called CR-Conformer. It integrates a DCNN branch that can effectively extract local features and a Transformer branch that can extract global features to capture more valuable features in clinical skin lesion images. In addition, we improved the DCNN branch to extract enhanced features in four directions through the convolutional rotation operation, further improving the classification performance of clinical skin lesion images. To verify the effectiveness of our proposed method, we conducted comprehensive tests on a private dataset named XJUSL, which contains ten types of clinical skin lesions. The test results indicate that our proposed method reduced the number of parameters by 11.17 M and improved the accuracy of clinical skin lesion image classification by 1.08%. It has the potential to realize automatic diagnosis of skin lesions in mobile devices.

SkinNet-INIO: Multiclass Skin Lesion Localization and Classification Using Fusion-Assisted Deep Neural Networks and Improved Nature-Inspired Optimization Algorithm

Background: Using artificial intelligence (AI) with the concept of a deep learning-based automated computer-aided diagnosis (CAD) system has shown improved performance for skin lesion classification. Although deep convolutional neural networks (DCNNs) have significantly improved many image classification tasks, it is still difficult to accurately classify skin lesions because of a lack of training data, inter-class similarity, intra-class variation, and the inability to concentrate on semantically significant lesion parts. Innovations: To address these issues, we proposed an automated deep learning and best feature selection framework for multiclass skin lesion classification in dermoscopy images. The proposed framework performs a preprocessing step at the initial step for contrast enhancement using a new technique that is based on dark channel haze and top-bottom filtering. Three pre-trained deep learning models are fine-tuned in the next step and trained using the transfer learning concept. In the fine-tuning process, we added and removed a few additional layers to lessen the parameters and later selected the hyperparameters using a genetic algorithm (GA) instead of manual assignment. The purpose of hyperparameter selection using GA is to improve the learning performance. After that, the deeper layer is selected for each network and deep features are extracted. The extracted deep features are fused using a novel serial correlation-based approach. This technique reduces the feature vector length to the serial-based approach, but there is little redundant information. We proposed an improved anti-Lion optimization algorithm for the best feature selection to address this issue. The selected features are finally classified using machine learning algorithms. Main Results: The experimental process was conducted using two publicly available datasets, ISIC2018 and ISIC2019. Employing these datasets, we obtained an accuracy of 96.1 and 99.9%, respectively. Comparison was also conducted with state-of-the-art techniques and shows the proposed framework improved accuracy. Conclusions: The proposed framework successfully enhances the contrast of the cancer region. Moreover, the selection of hyperparameters using the automated techniques improved the learning process of the proposed framework. The proposed fusion and improved version of the selection process maintains the best accuracy and shorten the computational time.

Diagnosing Melanomas in Dermoscopy Images Using Deep Learning

When it comes to skin tumors and cancers, melanoma ranks among the most prevalent and deadly. With the advancement of deep learning and computer vision, it is now possible to quickly and accurately determine whether or not a patient has malignancy. This is significant since a prompt identification greatly decreases the likelihood of a fatal outcome. Artificial intelligence has the potential to improve healthcare in many ways, including melanoma diagnosis. In a nutshell, this research employed an Inception-V3 and InceptionResnet-V2 strategy for melanoma recognition. The feature extraction layers that were previously frozen were fine-tuned after the newly added top layers were trained. This study used data from the HAM10000 dataset, which included an unrepresentative sample of seven different forms of skin cancer. To fix the discrepancy, we utilized data augmentation. The proposed models outperformed the results of the previous investigation with an effectiveness of 0.89 for Inception-V3 and 0.91 for InceptionResnet-V2.

A Novel Vision Transformer Model for Skin Cancer Classification

Skin cancer can be fatal if it is found to be malignant. Modern diagnosis of skin cancer heavily relies on visual inspection through clinical screening, dermoscopy, or histopathological examinations. However, due to similarity among cancer types, it is usually challenging to identify the type of skin cancer, especially at its early stages. Deep learning techniques have been developed over the last few years and have achieved success in helping to improve the accuracy of diagnosis and classification. However, the latest deep learning algorithms still do not provide ideal classification accuracy. To further improve the performance of classification accuracy, this paper presents a novel method of classifying skin cancer in clinical skin images. The method consists of four blocks. First, class rebalancing is applied to the images of seven skin cancer types for better classification performance. Second, an image is preprocessed by being split into patches of the same size and then flattened into a series of tokens. Third, a transformer encoder is used to process the flattened patches. The transformer encoder consists of N identical layers with each layer containing two sublayers. Sublayer one is a multihead self-attention unit, and sublayer two is a fully connected feed-forward network unit. For each of the two sublayers, a normalization operation is applied to its input, and a residual connection of its input and its output is calculated. Finally, a classification block is implemented after the transformer encoder. The block consists of a flattened layer and a dense layer with batch normalization. Transfer learning is implemented to build the whole network, where the ImageNet dataset is used to pretrain the network and the HAM10000 dataset is used to fine-tune the network. Experiments have shown that the method has achieved a classification accuracy of 94.1%, outperforming the current state-of-the-art model IRv2 with soft attention on the same training and testing datasets. On the Edinburgh DERMOFIT dataset also, the method has better performance compared with baseline models.

A Hybrid Stacked Restricted Boltzmann Machine with Sobel Directional Patterns for Melanoma Prediction in Colored Skin Images

Melanoma, a kind of skin cancer that is very risky, is distinguished by uncontrolled cell multiplication. Melanoma detection is of the utmost significance in clinical practice because of the atypical border structure and the numerous types of tissue it can involve. The identification of melanoma is still a challenging process for color images, despite the fact that numerous approaches have been proposed in the research that has been done. In this research, we present a comprehensive system for the efficient and precise classification of skin lesions. The framework includes preprocessing, segmentation, feature extraction, and classification modules. Preprocessing with DullRazor eliminates skin-imaging hair artifacts. Next, Fully Connected Neural Network (FCNN) semantic segmentation extracts precise and obvious Regions of Interest (ROIs). We then extract relevant skin image features from ROIs using an enhanced Sobel Directional Pattern (SDP). For skin image analysis, Sobel Directional Pattern outperforms ABCD. Finally, a stacked Restricted Boltzmann Machine (RBM) classifies skin ROIs. Stacked RBMs accurately classify skin melanoma. The experiments have been conducted on five datasets: Pedro Hispano Hospital (PH2), International Skin Imaging Collaboration (ISIC 2016), ISIC 2017, Dermnet, and DermIS, and achieved an accuracy of 99.8%, 96.5%, 95.5%, 87.9%, and 97.6%, respectively. The results show that a stack of Restricted Boltzmann Machines is superior for categorizing skin cancer types using the proposed innovative SDP.

A survey, review, and future trends of skin lesion segmentation and classification

The Computer-aided Diagnosis or Detection (CAD) approach for skin lesion analysis is an emerging field of research that has the potential to alleviate the burden and cost of skin cancer screening. Researchers have recently indicated increasing interest in developing such CAD systems, with the intention of providing a user-friendly tool to dermatologists to reduce the challenges encountered or associated with manual inspection. This article aims to provide a comprehensive literature survey and review of a total of 594 publications (356 for skin lesion segmentation and 238 for skin lesion classification) published between 2011 and 2022. These articles are analyzed and summarized in a number of different ways to contribute vital information regarding the methods for the development of CAD systems. These ways include: relevant and essential definitions and theories, input data (dataset utilization, preprocessing, augmentations, and fixing imbalance problems), method configuration (techniques, architectures, module frameworks, and losses), training tactics (hyperparameter settings), and evaluation criteria. We intend to investigate a variety of performance-enhancing approaches, including ensemble and post-processing. We also discuss these dimensions to reveal their current trends based on utilization frequencies. In addition, we highlight the primary difficulties associated with evaluating skin lesion segmentation and classification systems using minimal datasets, as well as the potential solutions to these difficulties. Findings, recommendations, and trends are disclosed to inform future research on developing an automated and robust CAD system for skin lesion analysis.

A Survey on Computer-Aided Intelligent Methods to Identify and Classify Skin Cancer

Melanoma is one of the skin cancer types that is more dangerous to human society. It easily spreads to other parts of the human body. An early diagnosis is necessary for a higher survival rate. Computer-aided diagnosis (CAD) is suitable for providing precise findings before the critical stage. The computer-aided diagnostic process includes preprocessing, segmentation, feature extraction, and classification. This study discusses the advantages and disadvantages of various computer-aided algorithms. It also discusses the current approaches, problems, and various types of datasets for skin images. Information about possible future works is also highlighted in this paper. The inferences derived from this survey will be useful for researchers carrying out research in skin cancer image analysis.

Melanoma Detection Using Deep Learning-Based Classifications

One of the most prevalent cancers worldwide is skin cancer, and it is becoming more common as the population ages. As a general rule, the earlier skin cancer can be diagnosed, the better. As a result of the success of deep learning (DL) algorithms in other industries, there has been a substantial increase in automated diagnosis systems in healthcare. This work proposes DL as a method for extracting a lesion zone with precision. First, the image is enhanced using Enhanced Super-Resolution Generative Adversarial Networks (ESRGAN) to improve the image's quality. Then, segmentation is used to segment Regions of Interest (ROI) from the full image. We employed data augmentation to rectify the data disparity. The image is then analyzed with a convolutional neural network (CNN) and a modified version of Resnet-50 to classify skin lesions. This analysis utilized an unequal sample of seven kinds of skin cancer from the HAM10000 dataset. With an accuracy of 0.86, a precision of 0.84, a recall of 0.86, and an F-score of 0.86, the proposed CNN-based Model outperformed the earlier study's results by a significant margin. The study culminates with an improved automated method for diagnosing skin cancer that benefits medical professionals and patients.

Discrimination of cancerous from benign pigmented skin lesions based on multispectral autofluorescence lifetime imaging dermoscopy and machine learning

SIGNIFICANCE

Accurate early diagnosis of malignant skin lesions is critical in providing adequate and timely treatment; unfortunately, initial clinical evaluation of similar-looking benign and malignant skin lesions can result in missed diagnosis of malignant lesions and unnecessary biopsy of benign ones.

AIM

To develop and validate a label-free and objective image-guided strategy for the clinical evaluation of suspicious pigmented skin lesions based on multispectral autofluorescence lifetime imaging (maFLIM) dermoscopy.

APPROACH

We tested the hypothesis that maFLIM-derived autofluorescence global features can be used in machine-learning (ML) models to discriminate malignant from benign pigmented skin lesions. Clinical widefield maFLIM dermoscopy imaging of 41 benign and 19 malignant pigmented skin lesions from 30 patients were acquired prior to tissue biopsy sampling. Three different pools of global image-level maFLIM features were extracted: multispectral intensity, time-domain biexponential, and frequency-domain phasor features. The classification potential of each feature pool to discriminate benign versus malignant pigmented skin lesions was evaluated by training quadratic discriminant analysis (QDA) classification models and applying a leave-one-patient-out cross-validation strategy.

RESULTS

Classification performance estimates obtained after unbiased feature selection were as follows: 68% sensitivity and 80% specificity with the phasor feature pool, 84% sensitivity, and 71% specificity with the biexponential feature pool, and 84% sensitivity and 32% specificity with the intensity feature pool. Ensemble combinations of QDA models trained with phasor and biexponential features yielded sensitivity of 84% and specificity of 90%, outperforming all other models considered.

CONCLUSIONS

Simple classification ML models based on time-resolved (biexponential and phasor) autofluorescence global features extracted from maFLIM dermoscopy images have the potential to provide objective discrimination of malignant from benign pigmented lesions. ML-assisted maFLIM dermoscopy could potentially assist with the clinical evaluation of suspicious lesions and the identification of those patients benefiting the most from biopsy examination.

Single Model Deep Learning on Imbalanced Small Datasets for Skin Lesion Classification

Deep convolutional neural network (DCNN) models have been widely explored for skin disease diagnosis and some of them have achieved the diagnostic outcomes comparable or even superior to those of dermatologists. However, broad implementation of DCNN in skin disease detection is hindered by small size and data imbalance of the publically accessible skin lesion datasets. This paper proposes a novel single-model based strategy for classification of skin lesions on small and imbalanced datasets. First, various DCNNs are trained on different small and imbalanced datasets to verify that the models with moderate complexity outperform the larger models. Second, regularization DropOut and DropBlock are added to reduce overfitting and a Modified RandAugment augmentation strategy is proposed to deal with the defects of sample underrepresentation in the small dataset. Finally, a novel Multi-Weighted New Loss (MWNL) function and an end-to-end cumulative learning strategy (CLS) are introduced to overcome the challenge of uneven sample size and classification difficulty and to reduce the impact of abnormal samples on training. By combining Modified RandAugment, MWNL and CLS, our single DCNN model method achieved the classification accuracy comparable or superior to those of multiple ensembling models on different dermoscopic image datasets. Our study shows that this method is able to achieve a high classification performance at a low cost of computational resources and inference time, potentially suitable to implement in mobile devices for automated screening of skin lesions and many other malignancies in low resource settings.

Skin lesion classification using features of 3D border lines

Machine learning algorithms are progressively assuming important roles as computational tools to support clinical diagnosis, namely in the classification of pigmented skin lesions using RGB images. Most current classification methods rely on common 2D image features derived from shape, colour or texture, which does not always guarantee the best results. This work presents a contribution to this field, by exploiting the lesions' border line characteristics using a new dimension - depth, which has not been thoroughly investigated so far. A selected group of features is extracted from the depth information of 3D images, which are then used for classification using a quadratic Support Vector Machine. Despite class imbalance often present in medical image datasets, the proposed algorithm achieves a top geometric mean of 94.87%, comprising 100.00% sensitivity and 90.00% specificity, using only depth information for the detection of Melanomas. Such results show that potential gains can be achieved by extracting information from this often overlooked dimension, which provides more balanced results in terms of sensitivity and specificity than other settings.

Melanoma classification using light-Fields with morlet scattering transform and CNN: Surface depth as a valuable tool to increase detection rate

Medical image classification through learning-based approaches has been increasingly used, namely in the discrimination of melanoma. However, for skin lesion classification in general, such methods commonly rely on dermoscopic or other 2D-macro RGB images. This work proposes to exploit beyond conventional 2D image characteristics, by considering a third dimension (depth) that characterises the skin surface rugosity, which can be obtained from light-field images, such as those available in the SKINL2 dataset. To achieve this goal, a processing pipeline was deployed using a morlet scattering transform and a CNN model, allowing to perform a comparison between using 2D information, only 3D information, or both. Results show that discrimination between Melanoma and Nevus reaches an accuracy of 84.00, 74.00 or 94.00% when using only 2D, only 3D, or both, respectively. An increase of 14.29pp in sensitivity and 8.33pp in specificity is achieved when expanding beyond conventional 2D information by also using depth. When discriminating between Melanoma and all other types of lesions (a further imbalanced setting), an increase of 28.57pp in sensitivity and decrease of 1.19pp in specificity is achieved for the same test conditions. Overall the results of this work demonstrate significant improvements over conventional approaches.

Machine Learning and Deep Learning Methods for Skin Lesion Classification and Diagnosis: A Systematic Review

Computer-aided systems for skin lesion diagnosis is a growing area of research. Recently, researchers have shown an increasing interest in developing computer-aided diagnosis systems. This paper aims to review, synthesize and evaluate the quality of evidence for the diagnostic accuracy of computer-aided systems. This study discusses the papers published in the last five years in ScienceDirect, IEEE, and SpringerLink databases. It includes 53 articles using traditional machine learning methods and 49 articles using deep learning methods. The studies are compared based on their contributions, the methods used and the achieved results. The work identified the main challenges of evaluating skin lesion segmentation and classification methods such as small datasets, ad hoc image selection and racial bias.

Realistic hair simulator for skin lesion images: A novel benchemarking tool

Automated skin lesion analysis is one of the trending fields that has gained attention among the dermatologists and health care practitioners. Skin lesion restoration is an essential pre-processing step for lesion enhancements for accurate automated analysis and diagnosis by both dermatologists and computer-aided diagnosis tools. Hair occlusion is one of the most popular artifacts in dermatoscopic images. It can negatively impact the skin lesions diagnosis by both dermatologists and automated computer diagnostic tools. Digital hair removal is a non-invasive method for image enhancement for decrease the hair-occlusion artifact in previously captured images. Several hair removal methods were proposed for skin delineation and removal without standardized benchmarking techniques. Manual annotation is one of the main challenges that hinder the validation of these proposed methods on a large number of images or against benchmarking datasets for comparison purposes. In the presented work, we propose a photo-realistic hair simulator based on context-aware image synthesis using image-to-image translation techniques via conditional adversarial generative networks for generation of different hair occlusions in skin images, along with ground-truth mask for hair location. Hair-occluded image is synthesized using the latent structure of any input hair-free image by deep encoding the input image into a latent vector of features. The locations of required hair are highlighted using white pixels on the input image. Then, these deep encoded features are used to reconstruct the synthetic highly realistic hair-occluded image. Besides, we explored using three loss functions including L₁-norm, L₂-norm and structural similarity index (SSIM) to maximize the image synthesis visual quality. For the evaluation of the generated samples, the t-SNE feature mapping and Bland-Altman test are used as visualization tools for the experimental results. The results show the superior performance of our proposed method compared to previous methods for hair synthesis with plausible colours and preserving the integrity of the lesion texture. The proposed method can be used to generate benchmarking datasets for comparing the performance of digital hair removal methods. The code is available online at: https://github.com/attiamohammed/realhair.

Computational models of melanoma

Genes, proteins, or cells influence each other and consequently create patterns, which can be increasingly better observed by experimental biology and medicine. Thereby, descriptive methods of statistics and bioinformatics sharpen and structure our perception. However, additionally considering the interconnectivity between biological elements promises a deeper and more coherent understanding of melanoma. For instance, integrative network-based tools and well-grounded inductive in silico research reveal disease mechanisms, stratify patients, and support treatment individualization. This review gives an overview of different modeling techniques beyond statistics, shows how different strategies align with the respective medical biology, and identifies possible areas of new computational melanoma research.

Simplification of neural networks for skin lesion image segmentation using color channel pruning

Automatic analysis of skin abnormality is an effective way for medical experts to facilitate diagnosis procedures and improve their capabilities. Efficient and accurate methods for analysis of the skin abnormalities such as convolutional neural networks (CNNs) are typically complex. Hence, the implementation of such complex structures in portable medical instruments is not feasible due to power and resource limitations. CNNs can extract features from the skin abnormality images automatically. To reduce the burden of the network for feature extraction, which can lead to the network simplicity, proper input color channels could be selected. In this paper, a pruning framework is proposed to simplify these complex structures through the selection of most informative color channels and simplification of the network. Moreover, hardware requirements of different network structures are identified to analyze the complexity of different networks. Experimental results are conducted for segmentation of images from two publicly available datasets of both dermoscopy and non-dermoscopy images. Simulation results show that using the proposed color channel selection method, simple and efficient neural network structures can be applied for segmentation of skin abnormalities.

Microscopic melanoma detection and classification: A framework of pixel-based fusion and multilevel features reduction

The numbers of diagnosed patients by melanoma are drastic and contribute more deaths annually among young peoples. An approximately 192,310 new cases of skin cancer are diagnosed in 2019, which shows the importance of automated systems for the diagnosis process. Accordingly, this article presents an automated method for skin lesions detection and recognition using pixel-based seed segmented images fusion and multilevel features reduction. The proposed method involves four key steps: (a) mean-based function is implemented and fed input to top-hat and bottom-hat filters which later fused for contrast stretching, (b) seed region growing and graph-cut method-based lesion segmentation and fused both segmented lesions through pixel-based fusion, (c) multilevel features such as histogram oriented gradient (HOG), speeded up robust features (SURF), and color are extracted and simple concatenation is performed, and (d) finally variance precise entropy-based features reduction and classification through SVM via cubic kernel function. Two different experiments are performed for the evaluation of this method. The segmentation performance is evaluated on PH2, ISBI2016, and ISIC2017 with an accuracy of 95.86, 94.79, and 94.92%, respectively. The classification performance is evaluated on PH2 and ISBI2016 dataset with an accuracy of 98.20 and 95.42%, respectively. The results of the proposed automated systems are outstanding as compared to the current techniques reported in state of art, which demonstrate the validity of the proposed method.

An Efficient Melanoma Diagnosis Approach Using Integrated HMF Multi-Atlas Map Based Segmentation

Melanoma is a life threading disease when it grows outside the corium layer of the skin. Mortality rates of the Melanoma cases are maximum among the skin cancer patients. The cost required for the treatment of advanced melanoma cases is very high and the survival rate is low. Numerous computerized dermoscopy systems are developed based on the combination of shape, texture and color features to facilitate early diagnosis of melanoma. The availability and cost of the dermoscopic imaging system is still an issue. To mitigate this issue, this paper presented an integrated segmentation and Third Dimensional (3D) feature extraction approach for the accurate diagnosis of melanoma. A multi-atlas method is applied for the image segmentation. The patch-based label fusion model is expressed in a Bayesian framework to improve the segmentation accuracy. A depth map is obtained from the Two-dimensional (2D) dermoscopic image for reconstructing the 3D skin lesion represented as structure tensors. The 3D shape features including the relative depth features are obtained. Streaks are the significant morphological terms of the melanoma in the radial growth phase. The proposed method yields maximum segmentation accuracy, sensibility, specificity and minimum cost function than the existing segmentation technique and classifier.

Melanoma Detection in Dermoscopic Images using Color Features

Color is considered to be a major characteristic feature that is used for distinguishing benign and malignant melanocytic lesions. Most of malignant melanomas are characterized by the presence of six suspicious colors inspired from the ABCD dermoscopic rule. The presence of these suspicious colors histopathologically indicates the presence of melanin in the deeper layers of the epidermis and dermis. The objective of the proposed work is to evaluate the role of color features, a set of fifteen color features have been extracted from the region of interest to determine the role of color in malignancy detection. Further, a set of ensemble classifiers with dynamic selection techniques are used for classification of the extracted features, yielding an average accuracy of 87.5% for classifying benign and malignant lesions.

An implementation of normal distribution based segmentation and entropy controlled features selection for skin lesion detection and classification

Background

Melanoma is the deadliest type of skin cancer with highest mortality rate. However, the annihilation in its early stage implies a high survival rate therefore, it demands early diagnosis. The accustomed diagnosis methods are costly and cumbersome due to the involvement of experienced experts as well as the requirements for the highly equipped environment. The recent advancements in computerized solutions for this diagnosis are highly promising with improved accuracy and efficiency.

Methods

In this article, a method for the identification and classification of the lesion based on probabilistic distribution and best features selection is proposed. The probabilistic distribution such as normal distribution and uniform distribution are implemented for segmentation of lesion in the dermoscopic images. Then multi-level features are extracted and parallel strategy is performed for fusion. A novel entropy-based method with the combination of Bhattacharyya distance and variance are calculated for the selection of best features. Only selected features are classified using multi-class support vector machine, which is selected as a base classifier.

Results

The proposed method is validated on three publicly available datasets such as PH2, ISIC (i.e. ISIC MSK-2 and ISIC UDA), and Combined (ISBI 2016 and ISBI 2017), including multi-resolution RGB images and achieved accuracy of 97.5%, 97.75%, and 93.2%, respectively.

Conclusion

The base classifier performs significantly better on proposed features fusion and selection method as compared to other methods in terms of sensitivity, specificity, and accuracy. Furthermore, the presented method achieved satisfactory segmentation results on selected datasets.