1
|
Tognetti L, Miracapillo C, Leonardelli S, Luschi A, Iadanza E, Cevenini G, Rubegni P, Cartocci A. Deep Learning Techniques for the Dermoscopic Differential Diagnosis of Benign/Malignant Melanocytic Skin Lesions: From the Past to the Present. Bioengineering (Basel) 2024; 11:758. [PMID: 39199716 PMCID: PMC11351129 DOI: 10.3390/bioengineering11080758] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/20/2024] [Revised: 07/15/2024] [Accepted: 07/17/2024] [Indexed: 09/01/2024] Open
Abstract
There has been growing scientific interest in the research field of deep learning techniques applied to skin cancer diagnosis in the last decade. Though encouraging data have been globally reported, several discrepancies have been observed in terms of study methodology, result presentations and validation in clinical settings. The present review aimed to screen the scientific literature on the application of DL techniques to dermoscopic melanoma/nevi differential diagnosis and extrapolate those original studies adequately by reporting on a DL model, comparing them among clinicians and/or another DL architecture. The second aim was to examine those studies together according to a standard set of statistical measures, and the third was to provide dermatologists with a comprehensive explanation and definition of the most used artificial intelligence (AI) terms to better/further understand the scientific literature on this topic and, in parallel, to be updated on the newest applications in the medical dermatologic field, along with a historical perspective. After screening nearly 2000 records, a subset of 54 was selected. Comparing the 20 studies reporting on convolutional neural network (CNN)/deep convolutional neural network (DCNN) models, we have a scenario of highly performant DL algorithms, especially in terms of low false positive results, with average values of accuracy (83.99%), sensitivity (77.74%), and specificity (80.61%). Looking at the comparison with diagnoses by clinicians (13 studies), the main difference relies on the specificity values, with a +15.63% increase for the CNN/DCNN models (average specificity of 84.87%) compared to humans (average specificity of 64.24%) with a 14,85% gap in average accuracy; the sensitivity values were comparable (79.77% for DL and 79.78% for humans). To obtain higher diagnostic accuracy and feasibility in clinical practice, rather than in experimental retrospective settings, future DL models should be based on a large dataset integrating dermoscopic images with relevant clinical and anamnestic data that is prospectively tested and adequately compared with physicians.
Collapse
Affiliation(s)
- Linda Tognetti
- Dermatology Unit, Deparment of Medical, Surgical and Neurosciences, University of Siena, Viale Bracci 16, 53100 Siena, Italy (P.R.)
| | - Chiara Miracapillo
- Dermatology Unit, Deparment of Medical, Surgical and Neurosciences, University of Siena, Viale Bracci 16, 53100 Siena, Italy (P.R.)
| | - Simone Leonardelli
- Dermatology Unit, Deparment of Medical, Surgical and Neurosciences, University of Siena, Viale Bracci 16, 53100 Siena, Italy (P.R.)
| | - Alessio Luschi
- Bioengineering and Biomedical Data Science Lab, Department of Medical Biotechnologies, University of Siena, 53100 Siena, Italy (E.I.)
| | - Ernesto Iadanza
- Bioengineering and Biomedical Data Science Lab, Department of Medical Biotechnologies, University of Siena, 53100 Siena, Italy (E.I.)
| | - Gabriele Cevenini
- Bioengineering and Biomedical Data Science Lab, Department of Medical Biotechnologies, University of Siena, 53100 Siena, Italy (E.I.)
| | - Pietro Rubegni
- Dermatology Unit, Deparment of Medical, Surgical and Neurosciences, University of Siena, Viale Bracci 16, 53100 Siena, Italy (P.R.)
| | - Alessandra Cartocci
- Dermatology Unit, Deparment of Medical, Surgical and Neurosciences, University of Siena, Viale Bracci 16, 53100 Siena, Italy (P.R.)
- Bioengineering and Biomedical Data Science Lab, Department of Medical Biotechnologies, University of Siena, 53100 Siena, Italy (E.I.)
| |
Collapse
|
2
|
Ashour AS, Abd El-Wahab BS, Wahba MA, Fotiadis DI. DMpDP: a Diagnostic Multiple-patient DermoFeature Profile store-and-forward teledermoscopy system. Med Biol Eng Comput 2024; 62:973-996. [PMID: 38110832 PMCID: PMC10948560 DOI: 10.1007/s11517-023-02982-0] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/03/2023] [Accepted: 11/28/2023] [Indexed: 12/20/2023]
Abstract
Telehealth demand is rapidly growing along with the necessity of providing wide-scale services covering multiple patients at the same time. In this work, the development of a store-and-forward (SAF) teledermoscopy system was considered. The dermoFeatures profile (DP) was proposed to decrease the size of the original dermoscopy image using its most significant features in the form of a newly generated diagonal alignment to generate a small-sized image DP, which is based on the extraction of a weighted intensity-difference frequency (WIDF) features along with morphological features (MOFs). These DPs were assembled to establish a Diagnostic Multiple-patient DermoFeature Profile (DMpDP). Different arrangements are proposed, namely the horizontally aligned, the diagonal-based, and the sequential-based DMpDPs to support the SAF systems. The DMpDPs are then embedded in a recorded patient-information signal (RPS) using a weight factor β to boost the transmitted patient-information signal. The effect of the different transform domains, β values, and number of DPs within the DMpDP were investigated in terms of the diagnostic classification accuracy at the receiver based on the extracted DPs, along with the recorded signal quality evaluation metrics of the recovered RPS. The sequential-based DMpDP achieved the highest classification accuracy, under - 5 dB additive white Gaussian noise, with a realized signal-to-noise ratio of 98.79% during the transmission of 248 DPs using β = 100, and spectral subtraction filtering.
Collapse
Affiliation(s)
- Amira S Ashour
- Department of Electronics and Electrical Communications Engineering, Faculty of Engineering, Tanta University, Tanta, Egypt.
| | - Basant S Abd El-Wahab
- Department of Electronics and Electrical Communications Engineering, Faculty of Engineering, Tanta University, Tanta, Egypt
| | - Maram A Wahba
- Department of Electronics and Electrical Communications Engineering, Faculty of Engineering, Tanta University, Tanta, Egypt
| | - Dimitrios I Fotiadis
- Unit of Medical Technology and Intelligent Information Systems, Dept. of Materials Science and Engineering, University of Ioannina, GR 45110, Ioannina, Greece
- Department of Biomedical Research, Institute of Molecular Biology and Biotechnology, FORTH, GR 45110, Ioannina, Greece
| |
Collapse
|
3
|
Pérez E, Ventura S. Progressive growing of Generative Adversarial Networks for improving data augmentation and skin cancer diagnosis. Artif Intell Med 2023; 141:102556. [PMID: 37295899 DOI: 10.1016/j.artmed.2023.102556] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/12/2022] [Revised: 04/06/2023] [Accepted: 04/14/2023] [Indexed: 06/12/2023]
Abstract
Early melanoma diagnosis is the most important factor in the treatment of skin cancer and can effectively reduce mortality rates. Recently, Generative Adversarial Networks have been used to augment data, prevent overfitting and improve the diagnostic capacity of models. However, its application remains a challenging task due to the high levels of inter and intra-class variance seen in skin images, limited amounts of data, and model instability. We present a more robust Progressive Growing of Adversarial Networks based on residual learning, which is highly recommended to ease the training of deep networks. The stability of the training process was increased by receiving additional inputs from preceding blocks. The architecture is able to produce plausible photorealistic synthetic 512 × 512 skin images, even with small dermoscopic and non-dermoscopic skin image datasets as problem domains. In this manner, we tackle the lack of data and the imbalance problems. Additionally, the proposed approach leverages a skin lesion boundary segmentation algorithm and transfer learning to enhance the diagnosis of melanoma. Inception score and Matthews Correlation Coefficient were used to measure the performance of the models. The architecture was evaluated qualitatively and quantitatively through the use of an extensive experimental study on sixteen datasets, illustrating its effectiveness in the diagnosis of melanoma. Finally, four state-of-the-art data augmentation techniques applied in five convolutional neural network models were significantly outperformed. The results indicated that a bigger number of trainable parameters will not necessarily obtain a better performance in melanoma diagnosis.
Collapse
Affiliation(s)
- Eduardo Pérez
- Andalusian Research Institute in Data Science and Computational Intelligence (DaSCI). University of Córdoba, Córdoba, Spain; Maimónides Biomedical Research Institute of Córdoba (IMIBIC). University of Córdoba, Córdoba, Spain
| | - Sebastián Ventura
- Andalusian Research Institute in Data Science and Computational Intelligence (DaSCI). University of Córdoba, Córdoba, Spain; Maimónides Biomedical Research Institute of Córdoba (IMIBIC). University of Córdoba, Córdoba, Spain.
| |
Collapse
|
4
|
Oliveira B, Torres HR, Morais P, Veloso F, Baptista AL, Fonseca JC, Vilaça JL. A multi-task convolutional neural network for classification and segmentation of chronic venous disorders. Sci Rep 2023; 13:761. [PMID: 36641527 PMCID: PMC9840616 DOI: 10.1038/s41598-022-27089-8] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/08/2022] [Accepted: 12/26/2022] [Indexed: 01/16/2023] Open
Abstract
Chronic Venous Disorders (CVD) of the lower limbs are one of the most prevalent medical conditions, affecting 35% of adults in Europe and North America. Due to the exponential growth of the aging population and the worsening of CVD with age, it is expected that the healthcare costs and the resources needed for the treatment of CVD will increase in the coming years. The early diagnosis of CVD is fundamental in treatment planning, while the monitoring of its treatment is fundamental to assess a patient's condition and quantify the evolution of CVD. However, correct diagnosis relies on a qualitative approach through visual recognition of the various venous disorders, being time-consuming and highly dependent on the physician's expertise. In this paper, we propose a novel automatic strategy for the joint segmentation and classification of CVDs. The strategy relies on a multi-task deep learning network, denominated VENet, that simultaneously solves segmentation and classification tasks, exploiting the information of both tasks to increase learning efficiency, ultimately improving their performance. The proposed method was compared against state-of-the-art strategies in a dataset of 1376 CVD images. Experiments showed that the VENet achieved a classification performance of 96.4%, 96.4%, and 97.2% for accuracy, precision, and recall, respectively, and a segmentation performance of 75.4%, 76.7.0%, 76.7% for the Dice coefficient, precision, and recall, respectively. The joint formulation increased the robustness of both tasks when compared to the conventional classification or segmentation strategies, proving its added value, mainly for the segmentation of small lesions.
Collapse
Affiliation(s)
- Bruno Oliveira
- Life and Health Sciences Research Institute (ICVS), School of Medicine, University of Minho, Campus de Gualtar, 4710-057, Braga, Portugal. .,ICVS/3B's - PT Government Associate Laboratory, Braga/Guimarães, Portugal. .,Algoritmi Center, School of Engineering, University of Minho, Guimarães, Portugal. .,2Ai - School of Technology, IPCA, Barcelos, Portugal. .,LASI-Associate Laboratory of Intelligent Systems, 4800-058, Guimarães, Portugal.
| | - Helena R Torres
- Life and Health Sciences Research Institute (ICVS), School of Medicine, University of Minho, Campus de Gualtar, 4710-057, Braga, Portugal.,ICVS/3B's - PT Government Associate Laboratory, Braga/Guimarães, Portugal.,Algoritmi Center, School of Engineering, University of Minho, Guimarães, Portugal.,2Ai - School of Technology, IPCA, Barcelos, Portugal.,LASI-Associate Laboratory of Intelligent Systems, 4800-058, Guimarães, Portugal
| | - Pedro Morais
- 2Ai - School of Technology, IPCA, Barcelos, Portugal.,LASI-Associate Laboratory of Intelligent Systems, 4800-058, Guimarães, Portugal
| | - Fernando Veloso
- Life and Health Sciences Research Institute (ICVS), School of Medicine, University of Minho, Campus de Gualtar, 4710-057, Braga, Portugal.,ICVS/3B's - PT Government Associate Laboratory, Braga/Guimarães, Portugal.,2Ai - School of Technology, IPCA, Barcelos, Portugal.,LASI-Associate Laboratory of Intelligent Systems, 4800-058, Guimarães, Portugal.,Department of Mechanical Engineering, School of Engineering, University of Minho, Guimarães, Portugal
| | | | - Jaime C Fonseca
- Algoritmi Center, School of Engineering, University of Minho, Guimarães, Portugal.,LASI-Associate Laboratory of Intelligent Systems, 4800-058, Guimarães, Portugal
| | - João L Vilaça
- 2Ai - School of Technology, IPCA, Barcelos, Portugal.,LASI-Associate Laboratory of Intelligent Systems, 4800-058, Guimarães, Portugal
| |
Collapse
|
5
|
Foahom Gouabou AC, Collenne J, Monnier J, Iguernaissi R, Damoiseaux JL, Moudafi A, Merad D. Computer Aided Diagnosis of Melanoma Using Deep Neural Networks and Game Theory: Application on Dermoscopic Images of Skin Lesions. Int J Mol Sci 2022; 23:ijms232213838. [PMID: 36430315 PMCID: PMC9696950 DOI: 10.3390/ijms232213838] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/29/2022] [Revised: 10/31/2022] [Accepted: 11/07/2022] [Indexed: 11/12/2022] Open
Abstract
Early detection of melanoma remains a daily challenge due to the increasing number of cases and the lack of dermatologists. Thus, AI-assisted diagnosis is considered as a possible solution for this issue. Despite the great advances brought by deep learning and especially convolutional neural networks (CNNs), computer-aided diagnosis (CAD) systems are still not used in clinical practice. This may be explained by the dermatologist's fear of being misled by a false negative and the assimilation of CNNs to a "black box", making their decision process difficult to understand by a non-expert. Decision theory, especially game theory, is a potential solution as it focuses on identifying the best decision option that maximizes the decision-maker's expected utility. This study presents a new framework for automated melanoma diagnosis. Pursuing the goal of improving the performance of existing systems, our approach also attempts to bring more transparency in the decision process. The proposed framework includes a multi-class CNN and six binary CNNs assimilated to players. The players' strategies is to first cluster the pigmented lesions (melanoma, nevus, and benign keratosis), using the introduced method of evaluating the confidence of the predictions, into confidence level (confident, medium, uncertain). Then, a subset of players has the strategy to refine the diagnosis for difficult lesions with medium and uncertain prediction. We used EfficientNetB5 as the backbone of our networks and evaluated our approach on the public ISIC dataset consisting of 8917 lesions: melanoma (1113), nevi (6705) and benign keratosis (1099). The proposed framework achieved an area under the receiver operating curve (AUROC) of 0.93 for melanoma, 0.96 for nevus and 0.97 for benign keratosis. Furthermore, our approach outperformed existing methods in this task, improving the balanced accuracy (BACC) of the best compared method from 77% to 86%. These results suggest that our framework provides an effective and explainable decision-making strategy. This approach could help dermatologists in their clinical practice for patients with atypical and difficult-to-diagnose pigmented lesions. We also believe that our system could serve as a didactic tool for less experienced dermatologists.
Collapse
Affiliation(s)
| | - Jules Collenne
- LIS, CNRS, Aix Marseille University, 13288 Marseille, France
| | - Jilliana Monnier
- LIS, CNRS, Aix Marseille University, 13288 Marseille, France
- Research Cancer Centre of Marseille, Inserm, CNRS, Aix-Marseille University, 13273 Marseille, France
- Dermatology and Skin Cancer Department, La Timone Hospital, AP-HM, Aix-Marseille University, 13385 Marseille, France
| | | | | | | | - Djamal Merad
- LIS, CNRS, Aix Marseille University, 13288 Marseille, France
- Correspondence: (A.C.F.G.); (D.M.)
| |
Collapse
|
6
|
Pérez E, Ventura S. A framework to build accurate Convolutional Neural Network models for melanoma diagnosis. Knowl Based Syst 2022. [DOI: 10.1016/j.knosys.2022.110157] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/27/2022]
|
7
|
Deep Residual Learning Image Recognition Model for Skin Cancer Disease Detection and Classification. ACTA INFORMATICA PRAGENSIA 2022. [DOI: 10.18267/j.aip.189] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/18/2022] Open
|
8
|
Wang Y, Feng Y, Zhang L, Zhou JT, Liu Y, Goh RSM, Zhen L. Adversarial multimodal fusion with attention mechanism for skin lesion classification using clinical and dermoscopic images. Med Image Anal 2022; 81:102535. [PMID: 35872361 DOI: 10.1016/j.media.2022.102535] [Citation(s) in RCA: 12] [Impact Index Per Article: 6.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/02/2021] [Revised: 07/07/2022] [Accepted: 07/11/2022] [Indexed: 10/17/2022]
Abstract
Accurate skin lesion diagnosis requires a great effort from experts to identify the characteristics from clinical and dermoscopic images. Deep multimodal learning-based methods can reduce intra- and inter-reader variability and improve diagnostic accuracy compared to the single modality-based methods. This study develops a novel method, named adversarial multimodal fusion with attention mechanism (AMFAM), to perform multimodal skin lesion classification. Specifically, we adopt a discriminator that uses adversarial learning to enforce the feature extractor to learn the correlated information explicitly. Moreover, we design an attention-based reconstruction strategy to encourage the feature extractor to concentrate on learning the features of the lesion area, thus, enhancing the feature vector from each modality with more discriminative information. Unlike existing multimodal-based approaches, which only focus on learning complementary features from dermoscopic and clinical images, our method considers both correlated and complementary information of the two modalities for multimodal fusion. To verify the effectiveness of our method, we conduct comprehensive experiments on a publicly available multimodal and multi-task skin lesion classification dataset: 7-point criteria evaluation database. The experimental results demonstrate that our proposed method outperforms the current state-of-the-art methods and improves the average AUC score by above 2% on the test set.
Collapse
Affiliation(s)
- Yan Wang
- Institute of High Performance Computing, Agency for Science, Technology and Research (A*STAR), Singapore 138632, Singapore
| | - Yangqin Feng
- Institute of High Performance Computing, Agency for Science, Technology and Research (A*STAR), Singapore 138632, Singapore
| | - Lei Zhang
- Machine Intelligence Laboratory, College of Computer Science, Sichuan University, Chengdu, 610065, P.R.China
| | - Joey Tianyi Zhou
- Institute of High Performance Computing, Agency for Science, Technology and Research (A*STAR), Singapore 138632, Singapore
| | - Yong Liu
- Institute of High Performance Computing, Agency for Science, Technology and Research (A*STAR), Singapore 138632, Singapore
| | - Rick Siow Mong Goh
- Institute of High Performance Computing, Agency for Science, Technology and Research (A*STAR), Singapore 138632, Singapore
| | - Liangli Zhen
- Institute of High Performance Computing, Agency for Science, Technology and Research (A*STAR), Singapore 138632, Singapore.
| |
Collapse
|
9
|
Serrano C, Lazo M, Serrano A, Toledo-Pastrana T, Barros-Tornay R, Acha B. Clinically Inspired Skin Lesion Classification through the Detection of Dermoscopic Criteria for Basal Cell Carcinoma. J Imaging 2022; 8:197. [PMID: 35877641 PMCID: PMC9319034 DOI: 10.3390/jimaging8070197] [Citation(s) in RCA: 8] [Impact Index Per Article: 4.0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/08/2022] [Revised: 07/05/2022] [Accepted: 07/08/2022] [Indexed: 12/10/2022] Open
Abstract
Background and Objective. Skin cancer is the most common cancer worldwide. One of the most common non-melanoma tumors is basal cell carcinoma (BCC), which accounts for 75% of all skin cancers. There are many benign lesions that can be confused with these types of cancers, leading to unnecessary biopsies. In this paper, a new method to identify the different BCC dermoscopic patterns present in a skin lesion is presented. In addition, this information is applied to classify skin lesions into BCC and non-BCC. Methods. The proposed method combines the information provided by the original dermoscopic image, introduced in a convolutional neural network (CNN), with deep and handcrafted features extracted from color and texture analysis of the image. This color analysis is performed by transforming the image into a uniform color space and into a color appearance model. To demonstrate the validity of the method, a comparison between the classification obtained employing exclusively a CNN with the original image as input and the classification with additional color and texture features is presented. Furthermore, an exhaustive comparison of classification employing different color and texture measures derived from different color spaces is presented. Results. Results show that the classifier with additional color and texture features outperforms a CNN whose input is only the original image. Another important achievement is that a new color cooccurrence matrix, proposed in this paper, improves the results obtained with other texture measures. Finally, sensitivity of 0.99, specificity of 0.94 and accuracy of 0.97 are achieved when lesions are classified into BCC or non-BCC. Conclusions. To the best of our knowledge, this is the first time that a methodology to detect all the possible patterns that can be present in a BCC lesion is proposed. This detection leads to a clinically explainable classification into BCC and non-BCC lesions. In this sense, the classification of the proposed tool is based on the detection of the dermoscopic features that dermatologists employ for their diagnosis.
Collapse
Affiliation(s)
- Carmen Serrano
- Dpto. Teoría de la Señal y Comunicaciones, Universidad de Sevilla, Camino de los Descubrimientos s/n, 41092 Seville, Spain; (M.L.); (B.A.)
| | - Manuel Lazo
- Dpto. Teoría de la Señal y Comunicaciones, Universidad de Sevilla, Camino de los Descubrimientos s/n, 41092 Seville, Spain; (M.L.); (B.A.)
| | - Amalia Serrano
- Hospital Universitario Virgen Macarena, Calle Dr. Fedriani, 3, 41009 Seville, Spain;
| | - Tomás Toledo-Pastrana
- Hospitales Quironsalud Infanta Luisa y Sagrado Corazón, Calle San Jacinto, 87, 41010 Seville, Spain;
| | | | - Begoña Acha
- Dpto. Teoría de la Señal y Comunicaciones, Universidad de Sevilla, Camino de los Descubrimientos s/n, 41092 Seville, Spain; (M.L.); (B.A.)
| |
Collapse
|
10
|
Oliveira B, Torres HR, Morais P, Baptista A, Fonseca J, Vilaca JL. Classification of Chronic Venous Disorders using an Ensemble Optimization of Convolutional Neural Networks. ANNUAL INTERNATIONAL CONFERENCE OF THE IEEE ENGINEERING IN MEDICINE AND BIOLOGY SOCIETY. IEEE ENGINEERING IN MEDICINE AND BIOLOGY SOCIETY. ANNUAL INTERNATIONAL CONFERENCE 2022; 2022:516-519. [PMID: 36086619 DOI: 10.1109/embc48229.2022.9871502] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/15/2023]
Abstract
Chronic Venous Disorders (CVD) of lower limbs are one of the most prevalent medical conditions, affecting 35% of adults in Europe and North America. The early diagnosis of CVD is critical, however, the diagnosis relies on a visual recognition of the various venous disorders which is time- consuming and dependent on the physician's expertise. Thus, automatic strategies for the classification of the CVD severity are claimed. This paper proposed an automatic ensemble-based strategy of Deep Convolutional Neural Networks (DCNN) for the classification of CVDs severity from medical images. First, a clinical dataset containing 1376 images of patients' legs with CVD of 5 different levels of severity was constructed. Then, the constructed dataset was randomly split into training, testing, and validation datasets. Subsequently, a set of DCNN were individually applied to the images for classification. Finally, instead of a traditional voting ensemble strategy, extracted feature vectors from each DCNN were concatenated and fed into a new ensemble optimization network. Experiments showed that the proposed strategy achieved a classification with 93.8%, 93.4%, 92.4% of accuracy, precision, and recall, respectively. Moreover, compared to the traditional ensemble strategy, improvement in the accuracy of ~2% was registered. The proposed strategy showed to be accurate and robust for the diagnosis of CVD severity from medical images. Nevertheless, further research using an extensive clinical database is required to validate the potential of this strategy. Clinical Relevance- An automatic classification of CVD to reduce the probability of underdiagnoses and promote the treatment of CVD in the early stages.
Collapse
|
11
|
Deep Learning-Based Classification for Melanoma Detection Using XceptionNet. JOURNAL OF HEALTHCARE ENGINEERING 2022; 2022:2196096. [PMID: 35360474 PMCID: PMC8964214 DOI: 10.1155/2022/2196096] [Citation(s) in RCA: 5] [Impact Index Per Article: 2.5] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 12/16/2021] [Revised: 01/04/2022] [Accepted: 02/19/2022] [Indexed: 12/31/2022]
Abstract
Skin cancer is one of the most common types of cancer in the world, accounting for at least 40% of all cancers. Melanoma is considered as the 19th most commonly occurring cancer among the other cancers in the human society, such that about 300,000 new cases were found in 2018. While cancer diagnosis is based on interventional methods such as surgery, radiotherapy, and chemotherapy, studies show that the use of new computer technologies such as image processing mechanisms in processes related to early diagnosis of this cancer can help the physicians heal this cancer. This paper proposes an automatic method for diagnosis of skin cancer from dermoscopy images. The proposed model is based on an improved XceptionNet, which utilized swish activation function and depthwise separable convolutions. This system shows an improvement in the classification accuracy of the network compared to the original Xception and other dome architectures. Simulations of the proposed method are compared with some other related skin cancer diagnosis state-of-the-art solutions, and the results show that the suggested method achieves higher accuracy compared to the other comparative methods.
Collapse
|
12
|
Zhang X, Lee VCS, Rong J, Liu F, Kong H. Multi-channel convolutional neural network architectures for thyroid cancer detection. PLoS One 2022; 17:e0262128. [PMID: 35061759 PMCID: PMC8782508 DOI: 10.1371/journal.pone.0262128] [Citation(s) in RCA: 2] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/03/2021] [Accepted: 12/17/2021] [Indexed: 02/05/2023] Open
Abstract
Early detection of malignant thyroid nodules leading to patient-specific treatments can reduce morbidity and mortality rates. Currently, thyroid specialists use medical images to diagnose then follow the treatment protocols, which have limitations due to unreliable human false-positive diagnostic rates. With the emergence of deep learning, advances in computer-aided diagnosis techniques have yielded promising earlier detection and prediction accuracy; however, clinicians' adoption is far lacking. The present study adopts Xception neural network as the base structure and designs a practical framework, which comprises three adaptable multi-channel architectures that were positively evaluated using real-world data sets. The proposed architectures outperform existing statistical and machine learning techniques and reached a diagnostic accuracy rate of 0.989 with ultrasound images and 0.975 with computed tomography scans through the single input dual-channel architecture. Moreover, the patient-specific design was implemented for thyroid cancer detection and has obtained an accuracy of 0.95 for double inputs dual-channel architecture and 0.94 for four-channel architecture. Our evaluation suggests that ultrasound images and computed tomography (CT) scans yield comparable diagnostic results through computer-aided diagnosis applications. With ultrasound images obtained slightly higher results, CT, on the other hand, can achieve the patient-specific diagnostic design. Besides, with the proposed framework, clinicians can select the best fitting architecture when making decisions regarding a thyroid cancer diagnosis. The proposed framework also incorporates interpretable results as evidence, which potentially improves clinicians' trust and hence their adoption of the computer-aided diagnosis techniques proposed with increased efficiency and accuracy.
Collapse
Affiliation(s)
- Xinyu Zhang
- Department of Data Science and AI/Faculty of IT, Monash University, Melbourne, Victoria, Australia
| | - Vincent C. S. Lee
- Department of Data Science and AI/Faculty of IT, Monash University, Melbourne, Victoria, Australia
| | - Jia Rong
- Department of Data Science and AI/Faculty of IT, Monash University, Melbourne, Victoria, Australia
| | - Feng Liu
- West China Hospital of Sichuan University, Chengdu City, Sichuan Province, China
| | - Haoyu Kong
- Department of Human-Centred Computing/Faculty of IT, Monash University, Melbourne, Victoria, Australia
| |
Collapse
|
13
|
Popescu D, El-Khatib M, El-Khatib H, Ichim L. New Trends in Melanoma Detection Using Neural Networks: A Systematic Review. SENSORS (BASEL, SWITZERLAND) 2022; 22:496. [PMID: 35062458 PMCID: PMC8778535 DOI: 10.3390/s22020496] [Citation(s) in RCA: 21] [Impact Index Per Article: 10.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 11/22/2021] [Revised: 12/28/2021] [Accepted: 01/05/2022] [Indexed: 12/29/2022]
Abstract
Due to its increasing incidence, skin cancer, and especially melanoma, is a serious health disease today. The high mortality rate associated with melanoma makes it necessary to detect the early stages to be treated urgently and properly. This is the reason why many researchers in this domain wanted to obtain accurate computer-aided diagnosis systems to assist in the early detection and diagnosis of such diseases. The paper presents a systematic review of recent advances in an area of increased interest for cancer prediction, with a focus on a comparative perspective of melanoma detection using artificial intelligence, especially neural network-based systems. Such structures can be considered intelligent support systems for dermatologists. Theoretical and applied contributions were investigated in the new development trends of multiple neural network architecture, based on decision fusion. The most representative articles covering the area of melanoma detection based on neural networks, published in journals and impact conferences, were investigated between 2015 and 2021, focusing on the interval 2018-2021 as new trends. Additionally presented are the main databases and trends in their use in teaching neural networks to detect melanomas. Finally, a research agenda was highlighted to advance the field towards the new trends.
Collapse
Affiliation(s)
- Dan Popescu
- Faculty of Automatic Control and Computers, University Politehnica of Bucharest, 060042 Bucharest, Romania; (M.E.-K.); (H.E.-K.); (L.I.)
| | | | | | | |
Collapse
|
14
|
Moglia A, Cerri A, Moglia A, Berchiolli R, Ferrari M, Betti R. Machine learning for the identification of decision boundaries during the transition from radial to vertical growth phase superficial spreading melanomas. Melanoma Res 2021; 31:533-540. [PMID: 34686646 DOI: 10.1097/cmr.0000000000000774] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/26/2022]
Abstract
The objective of this study was to compute threshold values for the diameter of superficial spreading melanomas (SSMs) at which the radial growth phase (RGP) evolves into an invasive vertical growth phase (VGP). We examined reports from 1995 to 2019 of 834 primary SSMs. All the patients underwent complete surgical removal of the tumor and the diagnosis was confirmed after histologic examination. Machine learning was used to compute the thresholds. For invasive non-naevus-associated SSMs, a threshold for the diameter was found at 13.2 mm (n = 634). For the lower limb (n = 209) the threshold was at 9.8 mm, whereas for the upper limb (n = 117) at 14.1 mm. For the back (n = 106) and the trunk (n = 173), the threshold was at 16.2 mm and 17.1 mm, respectively. When considering non-naevus-associated and naevus-associated SSMs together (n = 834) a threshold for the diameter was found at 16.8 mm. For the lower limb (n = 248) the threshold was at 11.7 mm, whereas for the upper limb (n = 146) at 16.4 mm. For the back (n = 170) and the trunk (n = 236), the threshold was at 18.6 mm and 14.1 mm, respectively. Thresholds for various anatomic locations and for each gender were defined. They were based on the diameter of the melanoma and computed to suggest a transition from RGP to VGP. The transition from a radial to a more invasive vertical phase is detected by an increase of tumor size with a numeric cutoff. Besides the anamnestic, clinical and dermatoscopic findings, our proposed approach may have practical relevance in vivo during clinical presurgical inspections.
Collapse
Affiliation(s)
- Andrea Moglia
- Department of Translational Research and New Technologies in Medicine and Surgery, University of Pisa, EndoCAS, Center for Computer Assisted Surgery, Pisa
| | - Amilcare Cerri
- Department of Health Sciences, University of Milan, Dermatology Unit ASST Santi Paolo e Carlo, Milan, Italy
| | | | - Raffaella Berchiolli
- Department of Translational Research and New Technologies in Medicine and Surgery, University of Pisa, Vascular Surgery Unit, Pisa, Italy
| | - Mauro Ferrari
- Department of Translational Research and New Technologies in Medicine and Surgery, University of Pisa, EndoCAS, Center for Computer Assisted Surgery, Pisa
- Department of Translational Research and New Technologies in Medicine and Surgery, University of Pisa, Vascular Surgery Unit, Pisa, Italy
| | - Roberto Betti
- Department of Health Sciences, University of Milan, Dermatology Unit ASST Santi Paolo e Carlo, Milan, Italy
| |
Collapse
|
15
|
An ensemble-based convolutional neural network model powered by a genetic algorithm for melanoma diagnosis. Neural Comput Appl 2021. [DOI: 10.1007/s00521-021-06655-7] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/26/2022]
Abstract
AbstractMelanoma is one of the main causes of cancer-related deaths. The development of new computational methods as an important tool for assisting doctors can lead to early diagnosis and effectively reduce mortality. In this work, we propose a convolutional neural network architecture for melanoma diagnosis inspired by ensemble learning and genetic algorithms. The architecture is designed by a genetic algorithm that finds optimal members of the ensemble. Additionally, the abstract features of all models are merged and, as a result, additional prediction capabilities are obtained. The diagnosis is achieved by combining all individual predictions. In this manner, the training process is implicitly regularized, showing better convergence, mitigating the overfitting of the model, and improving the generalization performance. The aim is to find the models that best contribute to the ensemble. The proposed approach also leverages data augmentation, transfer learning, and a segmentation algorithm. The segmentation can be performed without training and with a central processing unit, thus avoiding a significant amount of computational power, while maintaining its competitive performance. To evaluate the proposal, an extensive experimental study was conducted on sixteen skin image datasets, where state-of-the-art models were significantly outperformed. This study corroborated that genetic algorithms can be employed to effectively find suitable architectures for the diagnosis of melanoma, achieving in overall 11% and 13% better prediction performances compared to the closest model in dermoscopic and non-dermoscopic images, respectively. Finally, the proposal was implemented in a web application in order to assist dermatologists and it can be consulted at http://skinensemble.com.
Collapse
|
16
|
Melanoma Recognition by Fusing Convolutional Blocks and Dynamic Routing between Capsules. Cancers (Basel) 2021; 13:cancers13194974. [PMID: 34638456 PMCID: PMC8508435 DOI: 10.3390/cancers13194974] [Citation(s) in RCA: 6] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/17/2021] [Revised: 09/28/2021] [Accepted: 09/29/2021] [Indexed: 11/17/2022] Open
Abstract
Simple Summary The early treatment of skin cancer can effectively reduce mortality rates. Recently, automatic melanoma diagnosis from skin images has gained attention, which was mainly encouraged by the well-known challenge developed by the International Skin Imaging Collaboration project. The majority of contestant submitted Convolutional Neural Network based solutions. However, this type of model presents disadvantages. As a consequence, Dynamic Routing between Capsules has been proposed to overcome such limitations. The aim of our proposal was to assess the advantages of combining both architectures. An extensive experimental study showed the proposal significantly outperformed state-of-the-art models, achieving 166% higher predictive performance compared to ResNet in non-dermoscopic images. In addition, the pixels activated during prediction were shown, which allows to assess the rationale to give such a conclusion. Finally, more research should be conducted in order to demonstrate the potential of this neural network architecture in other areas. Abstract Skin cancer is one of the most common types of cancers in the world, with melanoma being the most lethal form. Automatic melanoma diagnosis from skin images has recently gained attention within the machine learning community, due to the complexity involved. In the past few years, convolutional neural network models have been commonly used to approach this issue. This type of model, however, presents disadvantages that sometimes hamper its application in real-world situations, e.g., the construction of transformation-invariant models and their inability to consider spatial hierarchies between entities within an image. Recently, Dynamic Routing between Capsules architecture (CapsNet) has been proposed to overcome such limitations. This work is aimed at proposing a new architecture which combines convolutional blocks with a customized CapsNet architecture, allowing for the extraction of richer abstract features. This architecture uses high-quality 299×299×3 skin lesion images, and a hyper-tuning of the main parameters is performed in order to ensure effective learning under limited training data. An extensive experimental study on eleven image datasets was conducted where the proposal significantly outperformed several state-of-the-art models. Finally, predictions made by the model were validated through the application of two modern model-agnostic interpretation tools.
Collapse
|
17
|
A Comprehensive Evaluation and Benchmarking of Convolutional Neural Networks for Melanoma Diagnosis. Cancers (Basel) 2021; 13:cancers13174494. [PMID: 34503300 PMCID: PMC8431618 DOI: 10.3390/cancers13174494] [Citation(s) in RCA: 9] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/03/2021] [Revised: 09/02/2021] [Accepted: 09/04/2021] [Indexed: 01/22/2023] Open
Abstract
Melanoma is the most invasive skin cancer with the highest risk of death. While it is a serious skin cancer, it is highly curable if detected early. Melanoma diagnosis is difficult, even for experienced dermatologists, due to the wide range of morphologies in skin lesions. Given the rapid development of deep learning algorithms for melanoma diagnosis, it is crucial to validate and benchmark these models, which is the main challenge of this work. This research presents a new benchmarking and selection approach based on the multi-criteria analysis method (MCDM), which integrates entropy and the preference ranking organization method for enrichment of evaluations (PROMETHEE) methods. The experimental study is carried out in four phases. Firstly, 19 convolution neural networks (CNNs) are trained and evaluated on a public dataset of 991 dermoscopic images. Secondly, to obtain the decision matrix, 10 criteria, including accuracy, classification error, precision, sensitivity, specificity, F1-score, false-positive rate, false-negative rate, Matthews correlation coefficient (MCC), and the number of parameters are established. Third, entropy and PROMETHEE methods are integrated to determine the weights of criteria and rank the models. Fourth, the proposed benchmarking framework is validated using the VIKOR method. The obtained results reveal that the ResNet101 model is selected as the optimal diagnosis model for melanoma in our case study data. Thus, the presented benchmarking framework is proven to be useful at exposing the optimal melanoma diagnosis model targeting to ease the selection process of the proper convolutional neural network architecture.
Collapse
|
18
|
Kitajima K, Matsuo H, Kono A, Kuribayashi K, Kijima T, Hashimoto M, Hasegawa S, Murakami T, Yamakado K. Deep learning with deep convolutional neural network using FDG-PET/CT for malignant pleural mesothelioma diagnosis. Oncotarget 2021; 12:1187-1196. [PMID: 34136087 PMCID: PMC8202770 DOI: 10.18632/oncotarget.27979] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/20/2021] [Accepted: 05/14/2021] [Indexed: 11/25/2022] Open
Abstract
OBJECTIVES This study analyzed an artificial intelligence (AI) deep learning method with a three-dimensional deep convolutional neural network (3D DCNN) in regard to diagnostic accuracy to differentiate malignant pleural mesothelioma (MPM) from benign pleural disease using FDG-PET/CT results. RESULTS For protocol A, the area under the ROC curve (AUC)/sensitivity/specificity/accuracy values were 0.825/77.9% (81/104)/76.4% (55/72)/77.3% (136/176), while those for protocol B were 0.854/80.8% (84/104)/77.8% (56/72)/79.5% (140/176), for protocol C were 0.881/85.6% (89/104)/75.0% (54/72)/81.3% (143/176), and for protocol D were 0.896/88.5% (92/104)/73.6% (53/72)/82.4% (145/176). Protocol D showed significantly better diagnostic performance as compared to A, B, and C in ROC analysis (p = 0.031, p = 0.0020, p = 0.041, respectively). MATERIALS AND METHODS Eight hundred seventy-five consecutive patients with histologically proven or suspected MPM, shown by history, physical examination findings, and chest CT results, who underwent FDG-PET/CT examinations between 2007 and 2017 were investigated in a retrospective manner. There were 525 patients (314 MPM, 211 benign pleural disease) in the deep learning training set, 174 (102 MPM, 72 benign pleural disease) in the validation set, and 176 (104 MPM, 72 benign pleural disease) in the test set. Using AI with PET/CT alone (protocol A), human visual reading (protocol B), a quantitative method that incorporated maximum standardized uptake value (SUVmax) (protocol C), and a combination of PET/CT, SUVmax, gender, and age (protocol D), obtained data were subjected to ROC curve analyses. CONCLUSIONS Deep learning with 3D DCNN in combination with FDG-PET/CT imaging results as well as clinical features comprise a novel potential tool shows flexibility for differential diagnosis of MPM.
Collapse
Affiliation(s)
- Kazuhiro Kitajima
- Department of Radiology, Hyogo College of Medicine, Nishinomiya, Hyogo, Japan
| | - Hidetoshi Matsuo
- Department of Radiology, Kobe University Graduate School of Medicine, Kobe, Hyogo, Japan
| | - Atsushi Kono
- Department of Radiology, Kobe University Graduate School of Medicine, Kobe, Hyogo, Japan
| | - Kozo Kuribayashi
- Division of Respiratory Medicine, Department of Internal Medicine, Hyogo College of Medicine, Nishinomiya, Hyogo, Japan
| | - Takashi Kijima
- Division of Respiratory Medicine, Department of Internal Medicine, Hyogo College of Medicine, Nishinomiya, Hyogo, Japan
| | - Masaki Hashimoto
- Department of Thoracic Surgery, Hyogo College of Medicine, Nishinomiya, Hyogo, Japan
| | - Seiki Hasegawa
- Department of Thoracic Surgery, Hyogo College of Medicine, Nishinomiya, Hyogo, Japan
| | - Takamichi Murakami
- Department of Radiology, Kobe University Graduate School of Medicine, Kobe, Hyogo, Japan
| | - Koichiro Yamakado
- Department of Radiology, Hyogo College of Medicine, Nishinomiya, Hyogo, Japan
| |
Collapse
|
19
|
Towards Accurate Diagnosis of Skin Lesions Using Feedforward Back Propagation Neural Networks. Diagnostics (Basel) 2021; 11:diagnostics11060936. [PMID: 34067493 PMCID: PMC8224667 DOI: 10.3390/diagnostics11060936] [Citation(s) in RCA: 14] [Impact Index Per Article: 4.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/06/2021] [Revised: 05/19/2021] [Accepted: 05/21/2021] [Indexed: 01/10/2023] Open
Abstract
In the automatic detection framework, there have been many attempts to develop models for real-time melanoma detection. To effectively discriminate benign and malign skin lesions, this work investigates sixty different architectures of the Feedforward Back Propagation Network (FFBPN), based on shape asymmetry for an optimal structural design that includes both the hidden neuron number and the input data selection. The reason for the choice of shape asymmetry was based on the 5–10% disagreement between dermatologists regarding the efficacy of asymmetry in the diagnosis of malignant melanoma. Asymmetry is quantified based on lesion shape (contour), moment of inertia of the lesion shape and histograms. The FFBPN has a high architecture flexibility, which indicates it as a favorable tool to avoid the over-parameterization of the ANN and, equally, to discard those redundant input datasets that usually result in poor test performance. The FFBPN was tested on four public image datasets containing melanoma, dysplastic nevus and nevus images. Experimental results on multiple benchmark data sets demonstrate that asymmetry A2 is a meaningful feature for skin lesion classification, and FFBPN with 16 neurons in the hidden layer can model the data without compromising prediction accuracy.
Collapse
|
20
|
Song M, Liu C, Chen S, Zhang W. Nanocarrier-Based Drug Delivery for Melanoma Therapeutics. Int J Mol Sci 2021; 22:1873. [PMID: 33668591 PMCID: PMC7918190 DOI: 10.3390/ijms22041873] [Citation(s) in RCA: 16] [Impact Index Per Article: 5.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/25/2021] [Revised: 02/10/2021] [Accepted: 02/11/2021] [Indexed: 02/07/2023] Open
Abstract
Melanoma, as a tumor cell derived from melanocyte transformation, has the characteristics of malignant proliferation, high metastasis, rapid recurrence, and a low survival rate. Traditional therapy has many shortcomings, including drug side effects and poor patient compliance, and so on. Therefore, the development of an effective treatment is necessary. Currently, nanotechnologies are a promising oncology treatment strategy because of their ability to effectively deliver drugs and other bioactive molecules to targeted tissues with low toxicity, thereby improving the clinical efficacy of cancer therapy. In this review, the application of nanotechnology in the treatment of melanoma is reviewed and discussed. First, the pathogenesis and molecular targets of melanoma are elucidated, and the current clinical treatment strategies and deficiencies of melanoma are then introduced. Following this, we discuss the main features of developing efficient nanosystems and introduce the latest reports in the literature on nanoparticles for the treatment of melanoma. Subsequently, we review and discuss the application of nanoparticles in chemotherapeutic agents, immunotherapy, mRNA vaccines, and photothermal therapy, as well as the potential of nanotechnology in the early diagnosis of melanoma.
Collapse
Affiliation(s)
| | | | - Siyu Chen
- State Key Laboratory of Natural Medicines and School of Life Science and Technology, China Pharmaceutical University, Nanjing 211198, China; (M.S.); (C.L.)
| | - Wenxiang Zhang
- State Key Laboratory of Natural Medicines and School of Life Science and Technology, China Pharmaceutical University, Nanjing 211198, China; (M.S.); (C.L.)
| |
Collapse
|