1
|
Tognetti L, Miracapillo C, Leonardelli S, Luschi A, Iadanza E, Cevenini G, Rubegni P, Cartocci A. Deep Learning Techniques for the Dermoscopic Differential Diagnosis of Benign/Malignant Melanocytic Skin Lesions: From the Past to the Present. Bioengineering (Basel) 2024; 11:758. [PMID: 39199716 PMCID: PMC11351129 DOI: 10.3390/bioengineering11080758] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/20/2024] [Revised: 07/15/2024] [Accepted: 07/17/2024] [Indexed: 09/01/2024] Open
Abstract
There has been growing scientific interest in the research field of deep learning techniques applied to skin cancer diagnosis in the last decade. Though encouraging data have been globally reported, several discrepancies have been observed in terms of study methodology, result presentations and validation in clinical settings. The present review aimed to screen the scientific literature on the application of DL techniques to dermoscopic melanoma/nevi differential diagnosis and extrapolate those original studies adequately by reporting on a DL model, comparing them among clinicians and/or another DL architecture. The second aim was to examine those studies together according to a standard set of statistical measures, and the third was to provide dermatologists with a comprehensive explanation and definition of the most used artificial intelligence (AI) terms to better/further understand the scientific literature on this topic and, in parallel, to be updated on the newest applications in the medical dermatologic field, along with a historical perspective. After screening nearly 2000 records, a subset of 54 was selected. Comparing the 20 studies reporting on convolutional neural network (CNN)/deep convolutional neural network (DCNN) models, we have a scenario of highly performant DL algorithms, especially in terms of low false positive results, with average values of accuracy (83.99%), sensitivity (77.74%), and specificity (80.61%). Looking at the comparison with diagnoses by clinicians (13 studies), the main difference relies on the specificity values, with a +15.63% increase for the CNN/DCNN models (average specificity of 84.87%) compared to humans (average specificity of 64.24%) with a 14,85% gap in average accuracy; the sensitivity values were comparable (79.77% for DL and 79.78% for humans). To obtain higher diagnostic accuracy and feasibility in clinical practice, rather than in experimental retrospective settings, future DL models should be based on a large dataset integrating dermoscopic images with relevant clinical and anamnestic data that is prospectively tested and adequately compared with physicians.
Collapse
Affiliation(s)
- Linda Tognetti
- Dermatology Unit, Deparment of Medical, Surgical and Neurosciences, University of Siena, Viale Bracci 16, 53100 Siena, Italy (P.R.)
| | - Chiara Miracapillo
- Dermatology Unit, Deparment of Medical, Surgical and Neurosciences, University of Siena, Viale Bracci 16, 53100 Siena, Italy (P.R.)
| | - Simone Leonardelli
- Dermatology Unit, Deparment of Medical, Surgical and Neurosciences, University of Siena, Viale Bracci 16, 53100 Siena, Italy (P.R.)
| | - Alessio Luschi
- Bioengineering and Biomedical Data Science Lab, Department of Medical Biotechnologies, University of Siena, 53100 Siena, Italy (E.I.)
| | - Ernesto Iadanza
- Bioengineering and Biomedical Data Science Lab, Department of Medical Biotechnologies, University of Siena, 53100 Siena, Italy (E.I.)
| | - Gabriele Cevenini
- Bioengineering and Biomedical Data Science Lab, Department of Medical Biotechnologies, University of Siena, 53100 Siena, Italy (E.I.)
| | - Pietro Rubegni
- Dermatology Unit, Deparment of Medical, Surgical and Neurosciences, University of Siena, Viale Bracci 16, 53100 Siena, Italy (P.R.)
| | - Alessandra Cartocci
- Dermatology Unit, Deparment of Medical, Surgical and Neurosciences, University of Siena, Viale Bracci 16, 53100 Siena, Italy (P.R.)
- Bioengineering and Biomedical Data Science Lab, Department of Medical Biotechnologies, University of Siena, 53100 Siena, Italy (E.I.)
| |
Collapse
|
2
|
Yazdanparast T, Shamsipour M, Ayatollahi A, Delavar S, Ahmadi M, Samadi A, Firooz A. Comparison of the Diagnostic Accuracy of Teledermoscopy, Face-to-Face Examinations and Artificial Intelligence in the Diagnosis of Melanoma. Indian J Dermatol 2024; 69:296-300. [PMID: 39296707 PMCID: PMC11407570 DOI: 10.4103/ijd.ijd_61_24] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/01/2024] [Accepted: 03/01/2024] [Indexed: 09/21/2024] Open
Abstract
Background Rapid diagnosis of melanoma is necessary for a good prognosis. Using teledermatology and artificial intelligence for this issue is developing, but its diagnostic accuracy is less measured in a clinical setting. Objective The purpose of this study was to assess the diagnostic accuracy of the teledermoscopy method using the FotoFinder device as well as the Moleanalyzer Pro artificial intelligence (AI) Assistant and to compare them with the face-to-face clinical examination for the diagnosis of melanoma confirmed with histopathology. Methods Thirty melanocytic moles of 29 patients were included in the study. Each mole was assessed face-to-face, using FotoFinder teledermoscopy and Moleanalyzer Pro software methods. The results obtained from each method were compared with the results of the gold standard (pathology). The sensitivity and specificity of the three methods were calculated for malignant and borderline versus benign lesions. Inter-method reliability between a gold standard and other methods was evaluated using per cent agreement and Cohen's kappa coefficient. Results Five moles had a histopathological diagnosis of melanoma, and six and 19 moles were diagnosed as borderline and benign, respectively. Sensitivities and specificities were, respectively, as follows: face-to-face (90.9%, 57.9%), FotoFinder teledermoscopy (63.6%, 78.9%), FotoFinder® Moleanalyzer Pro (36.4%, 42.1%). Agreement with biopsy-obtained diagnosis categories of benign, borderline and malignant for face-to-face was 63.33%, FotoFinder teledermoscopy 73.33%, and FotoFinder® Moleanalyzer Pro 40%. Conclusions Teledermoscopy had the highest agreement with reference diagnosis as well as the highest specificities that caused a reduction of biopsy referrals. The FotoFinder® Moleanalyzer Pro had the lowest agreement. Therefore, it cannot replace dermatologist decision making.
Collapse
Affiliation(s)
- Taraneh Yazdanparast
- From the Center for Research and Training in Skin Diseases and Leprosy, Tehran University of Medical Sciences, Tehran, Iran
| | - Mansour Shamsipour
- Department of Research Methodology and Data Analysis, Tehran University of Medical Sciences, Tehran, Iran
- Center for Air Pollution Research (CAPR), Institute for Environmental Research, Tehran University of Medical Sciences, Tehran, Iran
| | - Azin Ayatollahi
- From the Center for Research and Training in Skin Diseases and Leprosy, Tehran University of Medical Sciences, Tehran, Iran
| | - Shohreh Delavar
- From the Center for Research and Training in Skin Diseases and Leprosy, Tehran University of Medical Sciences, Tehran, Iran
- Dermatology Department, Faculty of Medicine Tehran Medical Sciences, Islamic Azad University, Tehran, Iran
| | - Maryam Ahmadi
- From the Center for Research and Training in Skin Diseases and Leprosy, Tehran University of Medical Sciences, Tehran, Iran
| | - Aniseh Samadi
- From the Center for Research and Training in Skin Diseases and Leprosy, Tehran University of Medical Sciences, Tehran, Iran
| | - Alireza Firooz
- From the Center for Research and Training in Skin Diseases and Leprosy, Tehran University of Medical Sciences, Tehran, Iran
| |
Collapse
|
3
|
Salinas MP, Sepúlveda J, Hidalgo L, Peirano D, Morel M, Uribe P, Rotemberg V, Briones J, Mery D, Navarrete-Dechent C. A systematic review and meta-analysis of artificial intelligence versus clinicians for skin cancer diagnosis. NPJ Digit Med 2024; 7:125. [PMID: 38744955 PMCID: PMC11094047 DOI: 10.1038/s41746-024-01103-x] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/22/2023] [Accepted: 04/04/2024] [Indexed: 05/16/2024] Open
Abstract
Scientific research of artificial intelligence (AI) in dermatology has increased exponentially. The objective of this study was to perform a systematic review and meta-analysis to evaluate the performance of AI algorithms for skin cancer classification in comparison to clinicians with different levels of expertise. Based on PRISMA guidelines, 3 electronic databases (PubMed, Embase, and Cochrane Library) were screened for relevant articles up to August 2022. The quality of the studies was assessed using QUADAS-2. A meta-analysis of sensitivity and specificity was performed for the accuracy of AI and clinicians. Fifty-three studies were included in the systematic review, and 19 met the inclusion criteria for the meta-analysis. Considering all studies and all subgroups of clinicians, we found a sensitivity (Sn) and specificity (Sp) of 87.0% and 77.1% for AI algorithms, respectively, and a Sn of 79.78% and Sp of 73.6% for all clinicians (overall); differences were statistically significant for both Sn and Sp. The difference between AI performance (Sn 92.5%, Sp 66.5%) vs. generalists (Sn 64.6%, Sp 72.8%), was greater, when compared with expert clinicians. Performance between AI algorithms (Sn 86.3%, Sp 78.4%) vs expert dermatologists (Sn 84.2%, Sp 74.4%) was clinically comparable. Limitations of AI algorithms in clinical practice should be considered, and future studies should focus on real-world settings, and towards AI-assistance.
Collapse
Affiliation(s)
- Maria Paz Salinas
- Department of Dermatology, Escuela de Medicina, Pontificia Universidad Católica de Chile, Santiago, Chile
| | - Javiera Sepúlveda
- Department of Dermatology, Escuela de Medicina, Pontificia Universidad Católica de Chile, Santiago, Chile
| | - Leonel Hidalgo
- Department of Dermatology, Escuela de Medicina, Pontificia Universidad Católica de Chile, Santiago, Chile
| | - Dominga Peirano
- Department of Dermatology, Escuela de Medicina, Pontificia Universidad Católica de Chile, Santiago, Chile
| | - Macarena Morel
- Universidad Catolica-Evidence Center, Cochrane Chile Associated Center, Pontificia Universidad Católica de Chile, Santiago, Chile
| | - Pablo Uribe
- Department of Dermatology, Escuela de Medicina, Pontificia Universidad Católica de Chile, Santiago, Chile
- Melanoma and Skin Cancer Unit, Escuela de Medicina, Pontificia Universidad Católica de Chile, Santiago, Chile
| | - Veronica Rotemberg
- Dermatology Service, Department of Medicine, Memorial Sloan Kettering Cancer Center, New York, NY, USA
| | - Juan Briones
- Department of Oncology, Escuela de Medicina, Pontificia Universidad Católica de Chile, Santiago, Chile
| | - Domingo Mery
- Department of Computer Science, Pontificia Universidad Católica de Chile, Santiago, Chile
| | - Cristian Navarrete-Dechent
- Department of Dermatology, Escuela de Medicina, Pontificia Universidad Católica de Chile, Santiago, Chile.
- Melanoma and Skin Cancer Unit, Escuela de Medicina, Pontificia Universidad Católica de Chile, Santiago, Chile.
| |
Collapse
|
4
|
Goessinger EV, Cerminara SE, Mueller AM, Gottfrois P, Huber S, Amaral M, Wenz F, Kostner L, Weiss L, Kunz M, Maul JT, Wespi S, Broman E, Kaufmann S, Patpanathapillai V, Treyer I, Navarini AA, Maul LV. Consistency of convolutional neural networks in dermoscopic melanoma recognition: A prospective real-world study about the pitfalls of augmented intelligence. J Eur Acad Dermatol Venereol 2024; 38:945-953. [PMID: 38158385 DOI: 10.1111/jdv.19777] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/01/2023] [Accepted: 10/23/2023] [Indexed: 01/03/2024]
Abstract
BACKGROUND Deep-learning convolutional neural networks (CNNs) have outperformed even experienced dermatologists in dermoscopic melanoma detection under controlled conditions. It remains unexplored how real-world dermoscopic image transformations affect CNN robustness. OBJECTIVES To investigate the consistency of melanoma risk assessment by two commercially available CNNs to help formulate recommendations for current clinical use. METHODS A comparative cohort study was conducted from January to July 2022 at the Department of Dermatology, University Hospital Basel. Five dermoscopic images of 116 different lesions on the torso of 66 patients were captured consecutively by the same operator without deliberate rotation. Classification was performed by two CNNs (CNN-1/CNN-2). Lesions were divided into four subgroups based on their initial risk scoring and clinical dignity assessment. Reliability was assessed by variation and intraclass correlation coefficients. Excisions were performed for melanoma suspicion or two consecutively elevated CNN risk scores, and benign lesions were confirmed by expert consensus (n = 3). RESULTS 117 repeated image series of 116 melanocytic lesions (2 melanomas, 16 dysplastic naevi, 29 naevi, 1 solar lentigo, 1 suspicious and 67 benign) were classified. CNN-1 demonstrated superior measurement repeatability for clinically benign lesions with an initial malignant risk score (mean variation coefficient (mvc): CNN-1: 49.5(±34.3)%; CNN-2: 71.4(±22.5)%; p = 0.03), while CNN-2 outperformed for clinically benign lesions with benign scoring (mvc: CNN-1: 49.7(±22.7)%; CNN-2: 23.8(±29.3)%; p = 0.002). Both systems exhibited lowest score consistency for lesions with an initial malignant risk score and benign assessment. In this context, averaging three initial risk scores achieved highest sensitivity of dignity assessment (CNN-1: 94%; CNN-2: 89%). Intraclass correlation coefficients indicated 'moderate'-to-'good' reliability for both systems (CNN-1: 0.80, 95% CI:0.71-0.87, p < 0.001; CNN-2: 0.67, 95% CI:0.55-0.77, p < 0.001). CONCLUSIONS Potential user-induced image changes can significantly influence CNN classification. For clinical application, we recommend using the average of three initial risk scores. Furthermore, we advocate for CNN robustness optimization by cross-validation with repeated image sets. TRIAL REGISTRATION ClinicalTrials.gov (NCT04605822).
Collapse
Affiliation(s)
- E V Goessinger
- Department of Dermatology, University Hospital Basel, Basel, Switzerland
| | - S E Cerminara
- Department of Dermatology, University Hospital Basel, Basel, Switzerland
| | - A M Mueller
- Department of Dermatology, University Hospital Basel, Basel, Switzerland
| | - P Gottfrois
- Department of Dermatology, University Hospital Basel, Basel, Switzerland
| | - S Huber
- Department of Dermatology, University Hospital Basel, Basel, Switzerland
| | - M Amaral
- Department of Dermatology, University Hospital Basel, Basel, Switzerland
| | - F Wenz
- Department of Dermatology, University Hospital Basel, Basel, Switzerland
| | - L Kostner
- Department of Dermatology, University Hospital Basel, Basel, Switzerland
| | - L Weiss
- Department of Dermatology, University Hospital Basel, Basel, Switzerland
| | - M Kunz
- Department of Dermatology, University Hospital Basel, Basel, Switzerland
| | - J-T Maul
- Department of Dermatology, University Hospital Zurich, Zurich, Switzerland
| | - S Wespi
- Department of Dermatology, University Hospital Basel, Basel, Switzerland
| | - E Broman
- Department of Dermatology, University Hospital Basel, Basel, Switzerland
| | - S Kaufmann
- Department of Dermatology, University Hospital Basel, Basel, Switzerland
| | - V Patpanathapillai
- Department of Dermatology, University Hospital Basel, Basel, Switzerland
| | - I Treyer
- Department of Dermatology, University Hospital Basel, Basel, Switzerland
| | - A A Navarini
- Department of Dermatology, University Hospital Basel, Basel, Switzerland
| | - L V Maul
- Department of Dermatology, University Hospital Basel, Basel, Switzerland
- Department of Dermatology, University Hospital Zurich, Zurich, Switzerland
| |
Collapse
|
5
|
Miller I, Rosic N, Stapelberg M, Hudson J, Coxon P, Furness J, Walsh J, Climstein M. Performance of Commercial Dermatoscopic Systems That Incorporate Artificial Intelligence for the Identification of Melanoma in General Practice: A Systematic Review. Cancers (Basel) 2024; 16:1443. [PMID: 38611119 PMCID: PMC11011068 DOI: 10.3390/cancers16071443] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/29/2024] [Revised: 04/03/2024] [Accepted: 04/04/2024] [Indexed: 04/14/2024] Open
Abstract
BACKGROUND Cutaneous melanoma remains an increasing global public health burden, particularly in fair-skinned populations. Advancing technologies, particularly artificial intelligence (AI), may provide an additional tool for clinicians to help detect malignancies with a more accurate success rate. This systematic review aimed to report the performance metrics of commercially available convolutional neural networks (CNNs) tasked with detecting MM. METHODS A systematic literature search was performed using CINAHL, Medline, Scopus, ScienceDirect and Web of Science databases. RESULTS A total of 16 articles reporting MM were included in this review. The combined number of melanomas detected was 1160, and non-melanoma lesions were 33,010. The performance of market-approved technology and clinician performance for classifying melanoma was highly heterogeneous, with sensitivity ranging from 16.4 to 100.0%, specificity between 40.0 and 98.3% and accuracy between 44.0 and 92.0%. Less heterogeneity was observed when clinicians worked in unison with AI, with sensitivity ranging between 83.3 and 100.0%, specificity between 83.7 and 87.3%, and accuracy between 86.4 and 86.9%. CONCLUSION Instead of focusing on the performance of AI versus clinicians for classifying melanoma, more consistent performance has been obtained when clinicians' work is supported by AI, facilitating management decisions and improving health outcomes.
Collapse
Affiliation(s)
- Ian Miller
- Aquatic Based Research, Southern Cross University, Bilinga, QLD 4225, Australia; (I.M.); (N.R.)
- Faculty of Health, Southern Cross University, Bilinga, QLD 4225, Australia (P.C.)
- Specialist Suite, John Flynn Hospital, Tugun, QLD 4224, Australia
| | - Nedeljka Rosic
- Aquatic Based Research, Southern Cross University, Bilinga, QLD 4225, Australia; (I.M.); (N.R.)
- Faculty of Health, Southern Cross University, Bilinga, QLD 4225, Australia (P.C.)
| | - Michael Stapelberg
- Aquatic Based Research, Southern Cross University, Bilinga, QLD 4225, Australia; (I.M.); (N.R.)
- Faculty of Health, Southern Cross University, Bilinga, QLD 4225, Australia (P.C.)
- Specialist Suite, John Flynn Hospital, Tugun, QLD 4224, Australia
| | - Jeremy Hudson
- Faculty of Health, Southern Cross University, Bilinga, QLD 4225, Australia (P.C.)
- North Queensland Skin Centre, Townsville, QLD 4810, Australia
| | - Paul Coxon
- Faculty of Health, Southern Cross University, Bilinga, QLD 4225, Australia (P.C.)
- North Queensland Skin Centre, Townsville, QLD 4810, Australia
| | - James Furness
- Water Based Research Unit, Bond University, Robina, QLD 4226, Australia;
| | - Joe Walsh
- Sport Science Institute, Sydney, NSW 2000, Australia;
- AI Consulting Group, Sydney, NSW 2000, Australia
| | - Mike Climstein
- Aquatic Based Research, Southern Cross University, Bilinga, QLD 4225, Australia; (I.M.); (N.R.)
- Faculty of Health, Southern Cross University, Bilinga, QLD 4225, Australia (P.C.)
- Physical Activity, Lifestyle, Ageing and Wellbeing Faculty Research Group, University of Sydney, Sydney, NSW 2050, Australia
| |
Collapse
|
6
|
Yee J, Rosendahl C, Aoude LG. The role of artificial intelligence and convolutional neural networks in the management of melanoma: a clinical, pathological, and radiological perspective. Melanoma Res 2024; 34:96-104. [PMID: 38141179 PMCID: PMC10906187 DOI: 10.1097/cmr.0000000000000951] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/16/2023] [Accepted: 11/29/2023] [Indexed: 12/25/2023]
Abstract
Clinical dermatoscopy and pathological slide assessment are essential in the diagnosis and management of patients with cutaneous melanoma. For those presenting with stage IIC disease and beyond, radiological investigations are often considered. The dermatoscopic, whole slide and radiological images used during clinical care are often stored digitally, enabling artificial intelligence (AI) and convolutional neural networks (CNN) to learn, analyse and contribute to the clinical decision-making. A keyword search of the Medline database was performed to assess the progression, capabilities and limitations of AI and CNN and its use in diagnosis and management of cutaneous melanoma. Full-text articles were reviewed if they related to dermatoscopy, pathological slide assessment or radiology. Through analysis of 95 studies, we demonstrate that diagnostic accuracy of AI/CNN can be superior (or at least equal) to clinicians. However, variability in image acquisition, pre-processing, segmentation, and feature extraction remains challenging. With current technological abilities, AI/CNN and clinicians synergistically working together are better than one another in all subspecialty domains relating to cutaneous melanoma. AI has the potential to enhance the diagnostic capabilities of junior dermatology trainees, primary care skin cancer clinicians and general practitioners. For experienced clinicians, AI provides a cost-efficient second opinion. From a pathological and radiological perspective, CNN has the potential to improve workflow efficiency, allowing clinicians to achieve more in a finite amount of time. Until the challenges of AI/CNN are reliably met, however, they can only remain an adjunct to clinical decision-making.
Collapse
Affiliation(s)
- Joshua Yee
- Faculty of Medicine, University of Queensland, St Lucia
| | - Cliff Rosendahl
- Primary Care Clinical Unit, Medical School, The University of Queensland, Herston
| | - Lauren G. Aoude
- Frazer Institute, The University of Queensland, Woolloongabba, QLD, Australia
| |
Collapse
|
7
|
Joly-Chevrier M, Nguyen AXL, Liang L, Lesko-Krleza M, Lefrançois P. The State of Artificial Intelligence in Skin Cancer Publications. J Cutan Med Surg 2024; 28:146-152. [PMID: 38323537 PMCID: PMC11015717 DOI: 10.1177/12034754241229361] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 02/08/2024]
Abstract
BACKGROUND Artificial intelligence (AI) in skin cancer is a promising research field to assist physicians and to provide support to patients remotely. Physicians' awareness to new developments in AI research is important to define the best practices and scope of integrating AI-enabled technologies within a clinical setting. OBJECTIVES To analyze the characteristics and trends of AI skin cancer publications from dermatology journals. METHODS AI skin cancer publications were retrieved in June 2022 from the Web of Science. Publications were screened by title, abstract, and keywords to assess eligibility. Publications were fully reviewed. Publications were divided between nonmelanoma skin cancer (NMSC), melanoma, and skin cancer studies. The primary measured outcome was the number of citations. The secondary measured outcomes were articles' general characteristics and features related to AI. RESULTS A total of 168 articles were included: 25 on NMSC, 77 on melanoma, and 66 on skin cancer. The most common types of skin cancers were melanoma (134, 79.8%), basal cell carcinoma (61, 36.3%), and squamous cell carcinoma (45, 26.9%). All articles were published between 2000 and 2022, with 49 (29.2%) of them being published in 2021. Original studies that developed or assessed an algorithm predominantly used supervised learning (66, 97.0%) and deep neural networks (42, 67.7%). The most used imaging modalities were standard dermoscopy (76, 45.2%) and clinical images (39, 23.2%). CONCLUSIONS Most publications focused on developing or assessing screening technologies with mainly deep neural network algorithms. This indicates the eminent need for dermatologists to label or annotate images used by novel AI systems.
Collapse
Affiliation(s)
| | | | - Laurence Liang
- Faculty of Engineering, McGill University, Montreal, QC, Canada
| | - Michael Lesko-Krleza
- Division of Computer Engineering, Department of Electrical and Computer Engineering, Concordia University, Montreal, QC, Canada
| | - Philippe Lefrançois
- Division of Dermatology, Department of Medicine, McGill University, Montreal, QC, Canada
- Division of Dermatology, Department of Medicine, Jewish General Hospital, Montreal, QC, Canada
- Lady Davis Institute for Medical Research, Montreal, QC, Canada
| |
Collapse
|
8
|
Helenason J, Ekström C, Falk M, Papachristou P. Exploring the feasibility of an artificial intelligence based clinical decision support system for cutaneous melanoma detection in primary care - a mixed method study. Scand J Prim Health Care 2024; 42:51-60. [PMID: 37982736 PMCID: PMC10851794 DOI: 10.1080/02813432.2023.2283190] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 09/28/2022] [Accepted: 11/08/2023] [Indexed: 11/21/2023] Open
Abstract
Objective: Skin examination to detect cutaneous melanomas is commonly performed in primary care. In recent years, clinical decision support systems (CDSS) based on artificial intelligence (AI) have been introduced within several diagnostic fields.Setting: This study employs a variety of qualitative and quantitative methodologies to investigate the feasibility of an AI-based CDSS to detect cutaneous melanoma in primary care.Subjects and Design: Fifteen primary care physicians (PCPs) underwent near-live simulations using the CDSS on a simulated patient, and subsequent individual semi-structured interviews were explored with a hybrid thematic analysis approach. Additionally, twenty-five PCPs performed a reader study (diagnostic assessment on the basis of image interpretation) of 18 dermoscopic images, both with and without help from AI, investigating the value of adding AI support to a PCPs decision. Perceived instrument usability was rated on the System Usability Scale (SUS).Results: From the interviews, the importance of trust in the CDSS emerged as a central concern. Scientific evidence supporting sufficient diagnostic accuracy of the CDSS was expressed as an important factor that could increase trust. Access to AI decision support when evaluating dermoscopic images proved valuable as it formally increased the physician's diagnostic accuracy. A mean SUS score of 84.8, corresponding to 'good' usability, was measured.Conclusion: AI-based CDSS might play an important future role in cutaneous melanoma diagnostics, provided sufficient evidence of diagnostic accuracy and usability supporting its trustworthiness among the users.
Collapse
Affiliation(s)
| | | | - Magnus Falk
- Department of Health, Medicine and Caring Sciences, Linköping University, Linköping, Sweden
| | - Panagiotis Papachristou
- Department of Neurobiology, Care Sciences and Society, Karolinska Institutet, Stockholm, Sweden
| |
Collapse
|
9
|
Goessinger EV, Niederfeilner JC, Cerminara S, Maul JT, Kostner L, Kunz M, Huber S, Koral E, Habermacher L, Sabato G, Tadic A, Zimmermann C, Navarini A, Maul LV. Patient and dermatologists' perspectives on augmented intelligence for melanoma screening: A prospective study. J Eur Acad Dermatol Venereol 2024. [PMID: 38411348 DOI: 10.1111/jdv.19905] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/05/2023] [Accepted: 01/22/2024] [Indexed: 02/28/2024]
Abstract
BACKGROUND Artificial intelligence (AI) shows promising potential to enhance human decision-making as synergistic augmented intelligence (AuI), but requires critical evaluation for skin cancer screening in a real-world setting. OBJECTIVES To investigate the perspectives of patients and dermatologists after skin cancer screening by human, artificial and augmented intelligence. METHODS A prospective comparative cohort study conducted at the University Hospital Basel included 205 patients (at high-risk of developing melanoma, with resected or advanced disease) and 8 dermatologists. Patients underwent skin cancer screening by a dermatologist with subsequent 2D and 3D total-body photography (TBP). Any suspicious and all melanocytic skin lesions ≥3 mm were imaged with digital dermoscopes and classified by corresponding convolutional neural networks (CNNs). Excisions were performed based on dermatologist's melanoma suspicion, study-defined elevated CNN risk-scores and/or melanoma suspicion by AuI. Subsequently, all patients and dermatologists were surveyed about their experience using questionnaires, including quantification of patient's safety sense following different examinations (subjective safety score (SSS): 0-10). RESULTS Most patients believed AI could improve diagnostic performance (95.5%, n = 192/201). In total, 83.4% preferred AuI-based skin cancer screening compared to examination by AI or dermatologist alone (3D-TBP: 61.3%; 2D-TBP: 22.1%, n = 199). Regarding SSS, AuI induced a significantly higher feeling of safety than AI (mean-SSS (mSSS): 9.5 vs. 7.7, p < 0.0001) or dermatologist screening alone (mSSS: 9.5 vs. 9.1, p = 0.001). Most dermatologists expressed high trust in AI examination results (3D-TBP: 90.2%; 2D-TBP: 96.1%, n = 205). In 68.3% of the examinations, dermatologists felt that diagnostic accuracy improved through additional AI-assessment (n = 140/205). Especially beginners (<2 years' dermoscopic experience; 61.8%, n = 94/152) felt AI facilitated their clinical work compared to experts (>5 years' dermoscopic experience; 20.9%, n = 9/43). Contrarily, in divergent risk assessments, only 1.5% of dermatologists trusted a benign CNN-classification more than personal malignancy suspicion (n = 3/205). CONCLUSIONS While patients already prefer AuI with 3D-TBP for melanoma recognition, dermatologists continue to rely largely on their own decision-making despite high confidence in AI-results. TRIAL REGISTRATION ClinicalTrials.gov (NCT04605822).
Collapse
Affiliation(s)
- Elisabeth Victoria Goessinger
- Department of Dermatology, University Hospital Basel, Basel, Switzerland
- Faculty of Medicine, University of Basel, Basel, Switzerland
| | | | - Sara Cerminara
- Department of Dermatology, University Hospital Basel, Basel, Switzerland
- Faculty of Medicine, University of Basel, Basel, Switzerland
| | - Julia-Tatjana Maul
- Department of Dermatology, University Hospital Zurich, Zurich, Switzerland
- Faculty of Medicine, University of Zurich, Zurich, Switzerland
| | - Lisa Kostner
- Department of Dermatology, University Hospital Basel, Basel, Switzerland
- Faculty of Medicine, University of Basel, Basel, Switzerland
| | - Michael Kunz
- Department of Dermatology, University Hospital Basel, Basel, Switzerland
- Faculty of Medicine, University of Basel, Basel, Switzerland
| | - Stephanie Huber
- Department of Dermatology, University Hospital Basel, Basel, Switzerland
| | - Emrah Koral
- Department of Dermatology, University Hospital Basel, Basel, Switzerland
| | - Lea Habermacher
- Faculty of Medicine, University of Basel, Basel, Switzerland
| | - Gianna Sabato
- Faculty of Medicine, University of Basel, Basel, Switzerland
| | - Andrea Tadic
- Faculty of Medicine, University of Basel, Basel, Switzerland
| | | | - Alexander Navarini
- Department of Dermatology, University Hospital Basel, Basel, Switzerland
- Faculty of Medicine, University of Basel, Basel, Switzerland
| | - Lara Valeska Maul
- Department of Dermatology, University Hospital Basel, Basel, Switzerland
- Faculty of Medicine, University of Basel, Basel, Switzerland
- Department of Dermatology, University Hospital Zurich, Zurich, Switzerland
- Faculty of Medicine, University of Zurich, Zurich, Switzerland
| |
Collapse
|
10
|
Crawford ME, Kamali K, Dorey RA, MacIntyre OC, Cleminson K, MacGillivary ML, Green PJ, Langley RG, Purdy KS, DeCoste RC, Gruchy JR, Pasternak S, Oakley A, Hull PR. Using Artificial Intelligence as a Melanoma Screening Tool in Self-Referred Patients. J Cutan Med Surg 2024; 28:37-43. [PMID: 38156628 PMCID: PMC10908200 DOI: 10.1177/12034754231216967] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/03/2024]
Abstract
INTRODUCTION Early detection of melanoma requires timely access to medical care. In this study, we examined the feasibility of using artificial intelligence (AI) to flag possible melanomas in self-referred patients concerned that a skin lesion might be cancerous. METHODS Patients were recruited for the study through advertisements in 2 hospitals in Halifax, Nova Scotia, Canada. Lesions of concern were initially examined by a trained medical student and if the study criteria were met, the lesions were then scanned using the FotoFinder System®. The images were analyzed using their proprietary computer software. Macroscopic and dermoscopic images were evaluated by 3 experienced dermatologists and a senior dermatology resident, all blinded to the AI results. Suspicious lesions identified by the AI or any of the 3 dermatologists were then excised. RESULTS Seventeen confirmed malignancies were found, including 10 melanomas. Six melanomas were not flagged by the AI. These lesions showed ambiguous atypical melanocytic proliferations, and all were diagnostically challenging to the dermatologists and to the dermatopathologists. Eight malignancies were seen in patients with a family history of melanoma. The AI's ability to diagnose malignancy is not inferior to the dermatologists examining dermoscopic images. CONCLUSION AI, used in this study, may serve as a practical skin cancer screening aid. While it does have technical and diagnostic limitations, its inclusion in a melanoma screening program, directed at those with a concern about a particular lesion would be valuable in providing timely access to the diagnosis of skin cancer.
Collapse
Affiliation(s)
- Madeleine E. Crawford
- Division of Clinical Dermatology and Cutaneous Science, Department of Medicine, Dalhousie University, Halifax, NS, Canada
| | - Kiyana Kamali
- Division of Clinical Dermatology and Cutaneous Science, Department of Medicine, Dalhousie University, Halifax, NS, Canada
| | - Rachel A. Dorey
- Division of Clinical Dermatology and Cutaneous Science, Department of Medicine, Dalhousie University, Halifax, NS, Canada
| | - Olivia C. MacIntyre
- Division of Clinical Dermatology and Cutaneous Science, Department of Medicine, Dalhousie University, Halifax, NS, Canada
| | - Kristyna Cleminson
- Division of Clinical Dermatology and Cutaneous Science, Department of Medicine, Dalhousie University, Halifax, NS, Canada
| | - Michael L. MacGillivary
- Division of Clinical Dermatology and Cutaneous Science, Department of Medicine, Dalhousie University, Halifax, NS, Canada
| | - Peter J. Green
- Division of Clinical Dermatology and Cutaneous Science, Department of Medicine, Dalhousie University, Halifax, NS, Canada
| | - Richard G. Langley
- Division of Clinical Dermatology and Cutaneous Science, Department of Medicine, Dalhousie University, Halifax, NS, Canada
| | - Kerri S. Purdy
- Division of Clinical Dermatology and Cutaneous Science, Department of Medicine, Dalhousie University, Halifax, NS, Canada
| | - Ryan C. DeCoste
- Department of Pathology and Laboratory Medicine, Dalhousie University, Halifax, NS, Canada
| | - Jennette R. Gruchy
- Department of Pathology and Laboratory Medicine, Dalhousie University, Halifax, NS, Canada
| | - Sylvia Pasternak
- Department of Pathology and Laboratory Medicine, Dalhousie University, Halifax, NS, Canada
| | - Amanda Oakley
- Department of Medicine, Waikato Clinical Campus, University of Auckland, Hamilton, New Zealand
| | - Peter R. Hull
- Division of Clinical Dermatology and Cutaneous Science, Department of Medicine, Dalhousie University, Halifax, NS, Canada
| |
Collapse
|
11
|
Zhu AQ, Wang Q, Shi YL, Ren WW, Cao X, Ren TT, Wang J, Zhang YQ, Sun YK, Chen XW, Lai YX, Ni N, Chen YC, Hu JL, Mou LC, Zhao YJ, Liu YQ, Sun LP, Zhu XX, Xu HX, Guo LH. A deep learning fusion network trained with clinical and high-frequency ultrasound images in the multi-classification of skin diseases in comparison with dermatologists: a prospective and multicenter study. EClinicalMedicine 2024; 67:102391. [PMID: 38274117 PMCID: PMC10808933 DOI: 10.1016/j.eclinm.2023.102391] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 10/03/2023] [Revised: 12/07/2023] [Accepted: 12/07/2023] [Indexed: 01/27/2024] Open
Abstract
Background Clinical appearance and high-frequency ultrasound (HFUS) are indispensable for diagnosing skin diseases by providing internal and external information. However, their complex combination brings challenges for primary care physicians and dermatologists. Thus, we developed a deep multimodal fusion network (DMFN) model combining analysis of clinical close-up and HFUS images for binary and multiclass classification in skin diseases. Methods Between Jan 10, 2017, and Dec 31, 2020, the DMFN model was trained and validated using 1269 close-ups and 11,852 HFUS images from 1351 skin lesions. The monomodal convolutional neural network (CNN) model was trained and validated with the same close-up images for comparison. Subsequently, we did a prospective and multicenter study in China. Both CNN models were tested prospectively on 422 cases from 4 hospitals and compared with the results from human raters (general practitioners, general dermatologists, and dermatologists specialized in HFUS). The performance of binary classification (benign vs. malignant) and multiclass classification (the specific diagnoses of 17 types of skin diseases) measured by the area under the receiver operating characteristic curve (AUC) were evaluated. This study is registered with www.chictr.org.cn (ChiCTR2300074765). Findings The performance of the DMFN model (AUC, 0.876) was superior to that of the monomodal CNN model (AUC, 0.697) in the binary classification (P = 0.0063), which was also better than that of the general practitioner (AUC, 0.651, P = 0.0025) and general dermatologists (AUC, 0.838; P = 0.0038). By integrating close-up and HFUS images, the DMFN model attained an almost identical performance in comparison to dermatologists (AUC, 0.876 vs. AUC, 0.891; P = 0.0080). For the multiclass classification, the DMFN model (AUC, 0.707) exhibited superior prediction performance compared with general dermatologists (AUC, 0.514; P = 0.0043) and dermatologists specialized in HFUS (AUC, 0.640; P = 0.0083), respectively. Compared to dermatologists specialized in HFUS, the DMFN model showed better or comparable performance in diagnosing 9 of the 17 skin diseases. Interpretation The DMFN model combining analysis of clinical close-up and HFUS images exhibited satisfactory performance in the binary and multiclass classification compared with the dermatologists. It may be a valuable tool for general dermatologists and primary care providers. Funding This work was supported in part by the National Natural Science Foundation of China and the Clinical research project of Shanghai Skin Disease Hospital.
Collapse
Affiliation(s)
- An-Qi Zhu
- Department of Medical Ultrasound, Shanghai Skin Disease Hospital, School of Medicine, Tongji University, Shanghai, China
- Department of Medical Ultrasound, Shanghai Tenth People's Hospital, School of Medicine, Tongji University, Shanghai, China
- Department of Ultrasound, Zhongshan Hospital, Institute of Ultrasound in Medicine and Engineering, Fudan University, Shanghai, China
| | - Qiao Wang
- Department of Medical Ultrasound, Shanghai Skin Disease Hospital, School of Medicine, Tongji University, Shanghai, China
- Department of Medical Ultrasound, Shanghai Tenth People's Hospital, School of Medicine, Tongji University, Shanghai, China
- Shanghai Engineering Research Center of Ultrasound Diagnosis and Treatment, Shanghai, China
| | - Yi-Lei Shi
- MedAI Technology (Wuxi) Co., Ltd., Wuxi, China
| | - Wei-Wei Ren
- Department of Medical Ultrasound, Shanghai Skin Disease Hospital, School of Medicine, Tongji University, Shanghai, China
- Department of Medical Ultrasound, Shanghai Tenth People's Hospital, School of Medicine, Tongji University, Shanghai, China
- Shanghai Engineering Research Center of Ultrasound Diagnosis and Treatment, Shanghai, China
| | - Xu Cao
- MedAI Technology (Wuxi) Co., Ltd., Wuxi, China
| | - Tian-Tian Ren
- Department of Medical Ultrasound, Ma'anshan People's Hospital, Ma'anshan, China
| | - Jing Wang
- Department of Ultrasound, Jiading District Central Hospital Affiliated Shanghai University of Medicine & Health Sciences, Shanghai, China
| | - Ya-Qin Zhang
- Department of Ultrasound, Zhongshan Hospital, Institute of Ultrasound in Medicine and Engineering, Fudan University, Shanghai, China
| | - Yi-Kang Sun
- Department of Ultrasound, Zhongshan Hospital, Institute of Ultrasound in Medicine and Engineering, Fudan University, Shanghai, China
| | - Xue-Wen Chen
- Department of Dermatological Surgery, Shanghai Skin Disease Hospital, School of Medicine, Tongji University, Shanghai, China
| | - Yong-Xian Lai
- Department of Dermatological Surgery, Shanghai Skin Disease Hospital, School of Medicine, Tongji University, Shanghai, China
| | - Na Ni
- Department of Dermatological Surgery, Shanghai Skin Disease Hospital, School of Medicine, Tongji University, Shanghai, China
| | - Yu-Chong Chen
- Department of Dermatological Surgery, Shanghai Skin Disease Hospital, School of Medicine, Tongji University, Shanghai, China
| | | | - Li-Chao Mou
- MedAI Technology (Wuxi) Co., Ltd., Wuxi, China
| | - Yu-Jing Zhao
- Department of Medical Ultrasound, Shanghai Skin Disease Hospital, School of Medicine, Tongji University, Shanghai, China
| | - Ye-Qiang Liu
- Department of Pathology, Shanghai Skin Disease Hospital, School of Medicine, Tongji University, Shanghai, China
| | - Li-Ping Sun
- Department of Medical Ultrasound, Shanghai Tenth People's Hospital, School of Medicine, Tongji University, Shanghai, China
- Shanghai Engineering Research Center of Ultrasound Diagnosis and Treatment, Shanghai, China
| | - Xiao-Xiang Zhu
- Chair of Data Science in Earth Observation, Technical University of Munich, Munich, Germany
| | - Hui-Xiong Xu
- Department of Ultrasound, Zhongshan Hospital, Institute of Ultrasound in Medicine and Engineering, Fudan University, Shanghai, China
| | - Le-Hang Guo
- Department of Medical Ultrasound, Shanghai Skin Disease Hospital, School of Medicine, Tongji University, Shanghai, China
- Department of Medical Ultrasound, Shanghai Tenth People's Hospital, School of Medicine, Tongji University, Shanghai, China
- Shanghai Engineering Research Center of Ultrasound Diagnosis and Treatment, Shanghai, China
| | - China Alliance of Multi-Center Clinical Study for Ultrasound (Ultra-Chance)
- Department of Medical Ultrasound, Shanghai Skin Disease Hospital, School of Medicine, Tongji University, Shanghai, China
- Department of Medical Ultrasound, Shanghai Tenth People's Hospital, School of Medicine, Tongji University, Shanghai, China
- Shanghai Engineering Research Center of Ultrasound Diagnosis and Treatment, Shanghai, China
- MedAI Technology (Wuxi) Co., Ltd., Wuxi, China
- Department of Medical Ultrasound, Ma'anshan People's Hospital, Ma'anshan, China
- Department of Ultrasound, Jiading District Central Hospital Affiliated Shanghai University of Medicine & Health Sciences, Shanghai, China
- Department of Ultrasound, Zhongshan Hospital, Institute of Ultrasound in Medicine and Engineering, Fudan University, Shanghai, China
- Department of Dermatological Surgery, Shanghai Skin Disease Hospital, School of Medicine, Tongji University, Shanghai, China
- Department of Pathology, Shanghai Skin Disease Hospital, School of Medicine, Tongji University, Shanghai, China
- Chair of Data Science in Earth Observation, Technical University of Munich, Munich, Germany
| |
Collapse
|
12
|
Thomas L, Hyde C, Mullarkey D, Greenhalgh J, Kalsi D, Ko J. Real-world post-deployment performance of a novel machine learning-based digital health technology for skin lesion assessment and suggestions for post-market surveillance. Front Med (Lausanne) 2023; 10:1264846. [PMID: 38020164 PMCID: PMC10645139 DOI: 10.3389/fmed.2023.1264846] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/21/2023] [Accepted: 10/10/2023] [Indexed: 12/01/2023] Open
Abstract
Introduction Deep Ensemble for Recognition of Malignancy (DERM) is an artificial intelligence as a medical device (AIaMD) tool for skin lesion assessment. Methods We report prospective real-world performance from its deployment within skin cancer pathways at two National Health Service hospitals (UK) between July 2021 and October 2022. Results A total of 14,500 cases were seen, including patients 18-100 years old with Fitzpatrick skin types I-VI represented. Based on 8,571 lesions assessed by DERM with confirmed outcomes, versions A and B demonstrated very high sensitivity for detecting melanoma (95.0-100.0%) or malignancy (96.0-100.0%). Benign lesion specificity was 40.7-49.4% (DERM-vA) and 70.1-73.4% (DERM-vB). DERM identified 15.0-31.0% of cases as eligible for discharge. Discussion We show DERM performance in-line with sensitivity targets and pre-marketing authorisation research, and it reduced the caseload for hospital specialists in two pathways. Based on our experience we offer suggestions on key elements of post-market surveillance for AIaMDs.
Collapse
Affiliation(s)
- Lucy Thomas
- Chelsea and Westminster Hospital NHS Foundation Trust, London, United Kingdom
| | - Chris Hyde
- Exeter Test Group, Department of Health and Community Sciences, University of Exeter Medical School, Exeter, United Kingdom
| | | | | | | | - Justin Ko
- Department of Dermatology, Stanford Medicine, Stanford, CA, United States
| |
Collapse
|
13
|
Luo N, Zhong X, Su L, Cheng Z, Ma W, Hao P. Artificial intelligence-assisted dermatology diagnosis: From unimodal to multimodal. Comput Biol Med 2023; 165:107413. [PMID: 37703714 DOI: 10.1016/j.compbiomed.2023.107413] [Citation(s) in RCA: 2] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/12/2023] [Revised: 08/02/2023] [Accepted: 08/28/2023] [Indexed: 09/15/2023]
Abstract
Artificial Intelligence (AI) is progressively permeating medicine, notably in the realm of assisted diagnosis. However, the traditional unimodal AI models, reliant on large volumes of accurately labeled data and single data type usage, prove insufficient to assist dermatological diagnosis. Augmenting these models with text data from patient narratives, laboratory reports, and image data from skin lesions, dermoscopy, and pathologies could significantly enhance their diagnostic capacity. Large-scale pre-training multimodal models offer a promising solution, exploiting the burgeoning reservoir of clinical data and amalgamating various data types. This paper delves into unimodal models' methodologies, applications, and shortcomings while exploring how multimodal models can enhance accuracy and reliability. Furthermore, integrating cutting-edge technologies like federated learning and multi-party privacy computing with AI can substantially mitigate patient privacy concerns in dermatological datasets and further fosters a move towards high-precision self-diagnosis. Diagnostic systems underpinned by large-scale pre-training multimodal models can facilitate dermatology physicians in formulating effective diagnostic and treatment strategies and herald a transformative era in healthcare.
Collapse
Affiliation(s)
- Nan Luo
- Hospital of Chengdu University of Traditional Chinese Medicine, No. 39 Shi-er-qiao Road, Chengdu, 610075, Sichuan, China.
| | - Xiaojing Zhong
- Hospital of Chengdu University of Traditional Chinese Medicine, No. 39 Shi-er-qiao Road, Chengdu, 610075, Sichuan, China.
| | - Luxin Su
- Hospital of Chengdu University of Traditional Chinese Medicine, No. 39 Shi-er-qiao Road, Chengdu, 610075, Sichuan, China.
| | - Zilin Cheng
- Hospital of Chengdu University of Traditional Chinese Medicine, No. 39 Shi-er-qiao Road, Chengdu, 610075, Sichuan, China.
| | - Wenyi Ma
- Hospital of Chengdu University of Traditional Chinese Medicine, No. 39 Shi-er-qiao Road, Chengdu, 610075, Sichuan, China.
| | - Pingsheng Hao
- Hospital of Chengdu University of Traditional Chinese Medicine, No. 39 Shi-er-qiao Road, Chengdu, 610075, Sichuan, China.
| |
Collapse
|
14
|
Xue P, Si M, Qin D, Wei B, Seery S, Ye Z, Chen M, Wang S, Song C, Zhang B, Ding M, Zhang W, Bai A, Yan H, Dang L, Zhao Y, Rezhake R, Zhang S, Qiao Y, Qu Y, Jiang Y. Unassisted Clinicians Versus Deep Learning-Assisted Clinicians in Image-Based Cancer Diagnostics: Systematic Review With Meta-analysis. J Med Internet Res 2023; 25:e43832. [PMID: 36862499 PMCID: PMC10020907 DOI: 10.2196/43832] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/26/2022] [Revised: 01/19/2023] [Accepted: 02/13/2023] [Indexed: 02/16/2023] Open
Abstract
BACKGROUND A number of publications have demonstrated that deep learning (DL) algorithms matched or outperformed clinicians in image-based cancer diagnostics, but these algorithms are frequently considered as opponents rather than partners. Despite the clinicians-in-the-loop DL approach having great potential, no study has systematically quantified the diagnostic accuracy of clinicians with and without the assistance of DL in image-based cancer identification. OBJECTIVE We systematically quantified the diagnostic accuracy of clinicians with and without the assistance of DL in image-based cancer identification. METHODS PubMed, Embase, IEEEXplore, and the Cochrane Library were searched for studies published between January 1, 2012, and December 7, 2021. Any type of study design was permitted that focused on comparing unassisted clinicians and DL-assisted clinicians in cancer identification using medical imaging. Studies using medical waveform-data graphics material and those investigating image segmentation rather than classification were excluded. Studies providing binary diagnostic accuracy data and contingency tables were included for further meta-analysis. Two subgroups were defined and analyzed, including cancer type and imaging modality. RESULTS In total, 9796 studies were identified, of which 48 were deemed eligible for systematic review. Twenty-five of these studies made comparisons between unassisted clinicians and DL-assisted clinicians and provided sufficient data for statistical synthesis. We found a pooled sensitivity of 83% (95% CI 80%-86%) for unassisted clinicians and 88% (95% CI 86%-90%) for DL-assisted clinicians. Pooled specificity was 86% (95% CI 83%-88%) for unassisted clinicians and 88% (95% CI 85%-90%) for DL-assisted clinicians. The pooled sensitivity and specificity values for DL-assisted clinicians were higher than for unassisted clinicians, at ratios of 1.07 (95% CI 1.05-1.09) and 1.03 (95% CI 1.02-1.05), respectively. Similar diagnostic performance by DL-assisted clinicians was also observed across the predefined subgroups. CONCLUSIONS The diagnostic performance of DL-assisted clinicians appears better than unassisted clinicians in image-based cancer identification. However, caution should be exercised, because the evidence provided in the reviewed studies does not cover all the minutiae involved in real-world clinical practice. Combining qualitative insights from clinical practice with data-science approaches may improve DL-assisted practice, although further research is required. TRIAL REGISTRATION PROSPERO CRD42021281372; https://www.crd.york.ac.uk/prospero/display_record.php?RecordID=281372.
Collapse
Affiliation(s)
- Peng Xue
- Department of Epidemiology and Biostatistics, School of Population Medicine and Public Health, Chinese Academy of Medical Sciences and Peking Union Medical College, Beijing, China
| | - Mingyu Si
- Department of Epidemiology and Biostatistics, School of Population Medicine and Public Health, Chinese Academy of Medical Sciences and Peking Union Medical College, Beijing, China
| | - Dongxu Qin
- Department of Epidemiology and Biostatistics, School of Population Medicine and Public Health, Chinese Academy of Medical Sciences and Peking Union Medical College, Beijing, China
| | - Bingrui Wei
- Department of Epidemiology and Biostatistics, School of Population Medicine and Public Health, Chinese Academy of Medical Sciences and Peking Union Medical College, Beijing, China
| | - Samuel Seery
- Faculty of Health and Medicine, Division of Health Research, Lancaster University, Lancaster, United Kingdom
| | - Zichen Ye
- Department of Epidemiology and Biostatistics, School of Population Medicine and Public Health, Chinese Academy of Medical Sciences and Peking Union Medical College, Beijing, China
| | - Mingyang Chen
- Department of Epidemiology and Biostatistics, School of Population Medicine and Public Health, Chinese Academy of Medical Sciences and Peking Union Medical College, Beijing, China
| | - Sumeng Wang
- Department of Cancer Epidemiology, National Cancer Center/National Clinical Research Center for Cancer/Cancer Hospital, Chinese Academy of Medical Sciences and Peking Union Medical College, Beijing, China
| | - Cheng Song
- Department of Epidemiology and Biostatistics, School of Population Medicine and Public Health, Chinese Academy of Medical Sciences and Peking Union Medical College, Beijing, China
| | - Bo Zhang
- Department of Epidemiology and Biostatistics, School of Population Medicine and Public Health, Chinese Academy of Medical Sciences and Peking Union Medical College, Beijing, China
| | - Ming Ding
- Department of Epidemiology and Biostatistics, School of Population Medicine and Public Health, Chinese Academy of Medical Sciences and Peking Union Medical College, Beijing, China
| | - Wenling Zhang
- Department of Epidemiology and Biostatistics, School of Population Medicine and Public Health, Chinese Academy of Medical Sciences and Peking Union Medical College, Beijing, China
| | - Anying Bai
- Department of Epidemiology and Biostatistics, School of Population Medicine and Public Health, Chinese Academy of Medical Sciences and Peking Union Medical College, Beijing, China
| | - Huijiao Yan
- Department of Epidemiology and Biostatistics, School of Population Medicine and Public Health, Chinese Academy of Medical Sciences and Peking Union Medical College, Beijing, China
| | - Le Dang
- Peking Union Medical College Hospital, Chinese Academy of Medical Sciences and Peking Union Medical College, Beijing, China
| | - Yuqian Zhao
- Sichuan Cancer Hospital & Institute, Sichuan Cancer Center, School of Medicine, University of Electronic Science & Technology of China, Sichuan, China
| | - Remila Rezhake
- Affiliated Cancer Hospital, The 3rd Affiliated Teaching Hospital of Xinjiang Medical University, Xinjiang, China
| | - Shaokai Zhang
- Henan Cancer Hospital, Affiliated Cancer Hospital of Zhengzhou University, Henan, China
| | - Youlin Qiao
- Center for Global Health, School of Population Medicine and Public Health, Chinese Academy of Medical Sciences and Peking Union Medical College, Beijing, China
| | - Yimin Qu
- Department of Epidemiology and Biostatistics, School of Population Medicine and Public Health, Chinese Academy of Medical Sciences and Peking Union Medical College, Beijing, China
| | - Yu Jiang
- Department of Epidemiology and Biostatistics, School of Population Medicine and Public Health, Chinese Academy of Medical Sciences and Peking Union Medical College, Beijing, China
| |
Collapse
|
15
|
Li Z, Koban KC, Schenck TL, Giunta RE, Li Q, Sun Y. Artificial Intelligence in Dermatology Image Analysis: Current Developments and Future Trends. J Clin Med 2022; 11:jcm11226826. [PMID: 36431301 PMCID: PMC9693628 DOI: 10.3390/jcm11226826] [Citation(s) in RCA: 36] [Impact Index Per Article: 18.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/24/2022] [Revised: 10/24/2022] [Accepted: 10/28/2022] [Indexed: 11/22/2022] Open
Abstract
BACKGROUND Thanks to the rapid development of computer-based systems and deep-learning-based algorithms, artificial intelligence (AI) has long been integrated into the healthcare field. AI is also particularly helpful in image recognition, surgical assistance and basic research. Due to the unique nature of dermatology, AI-aided dermatological diagnosis based on image recognition has become a modern focus and future trend. Key scientific concepts of review: The use of 3D imaging systems allows clinicians to screen and label skin pigmented lesions and distributed disorders, which can provide an objective assessment and image documentation of lesion sites. Dermatoscopes combined with intelligent software help the dermatologist to easily correlate each close-up image with the corresponding marked lesion in the 3D body map. In addition, AI in the field of prosthetics can assist in the rehabilitation of patients and help to restore limb function after amputation in patients with skin tumors. THE AIM OF THE STUDY For the benefit of patients, dermatologists have an obligation to explore the opportunities, risks and limitations of AI applications. This study focuses on the application of emerging AI in dermatology to aid clinical diagnosis and treatment, analyzes the current state of the field and summarizes its future trends and prospects so as to help dermatologists realize the impact of new technological innovations on traditional practices so that they can embrace and use AI-based medical approaches more quickly.
Collapse
Affiliation(s)
- Zhouxiao Li
- Department of Plastic and Reconstructive Surgery, Shanghai 9th People’s Hospital, Shanghai Jiao Tong University School of Medicine, Shanghai 200023, China
- Division of Hand, Plastic and Aesthetic Surgery, University Hospital, LMU Munich, 80339 Munich, Germany
| | | | - Thilo Ludwig Schenck
- Division of Hand, Plastic and Aesthetic Surgery, University Hospital, LMU Munich, 80339 Munich, Germany
| | - Riccardo Enzo Giunta
- Division of Hand, Plastic and Aesthetic Surgery, University Hospital, LMU Munich, 80339 Munich, Germany
| | - Qingfeng Li
- Department of Plastic and Reconstructive Surgery, Shanghai 9th People’s Hospital, Shanghai Jiao Tong University School of Medicine, Shanghai 200023, China
- Correspondence: (Q.L.); (Y.S.)
| | - Yangbai Sun
- Department of Plastic and Reconstructive Surgery, Shanghai 9th People’s Hospital, Shanghai Jiao Tong University School of Medicine, Shanghai 200023, China
- Correspondence: (Q.L.); (Y.S.)
| |
Collapse
|
16
|
Distinguish the Value of the Benign Nevus and Melanomas Using Machine Learning: A Meta-Analysis and Systematic Review. Mediators Inflamm 2022; 2022:1734327. [PMID: 36274972 PMCID: PMC9586788 DOI: 10.1155/2022/1734327] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/18/2022] [Revised: 09/22/2022] [Accepted: 10/01/2022] [Indexed: 11/26/2022] Open
Abstract
Background Melanomas, the most common human malignancy, are primarily diagnosed visually, beginning with an initial clinical screening and followed potentially by dermoscopic analysis, a biopsy, and histopathological examination. We aimed to systematically review the performance and quality of machine learning-based methods in distinguishing melanoma and benign nevus in the relevant literature. Method Four databases (Web of Science, PubMed, Embase, and the Cochrane library) were searched to retrieve the relevant studies published until March 26, 2022. The Predictive model Deviation Risk Assessment tool (PROBAST) was used to assess the deviation risk of opposing law. Result This systematic review included thirty researches with 114007 subjects and 71 machine learning models. The convolutional neural network was the main machine learning method. The pooled sensitivity was 85% (95% CI 82–87%), the specificity was 86% (82–88%), and the C-index was 0.87 (0.84–0.90). Conclusion The findings of our study showed that ML algorithms had high sensitivity and specificity for distinguishing between melanoma and benign nevi. This suggests that state-of-the-art ML-based algorithms for distinguishing melanoma from benign nevi may be ready for clinical use. However, a large proportion of the earlier published studies had methodological flaws, such as lack of external validation and lack of clinician comparisons. The results of these studies should be interpreted with caution.
Collapse
|
17
|
Rasheed A, Umar AI, Shirazi SH, Khan Z, Nawaz S, Shahzad M. Automatic eczema classification in clinical images based on hybrid deep neural network. Comput Biol Med 2022; 147:105807. [PMID: 35809409 DOI: 10.1016/j.compbiomed.2022.105807] [Citation(s) in RCA: 7] [Impact Index Per Article: 3.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/15/2021] [Revised: 05/09/2022] [Accepted: 05/13/2022] [Indexed: 11/24/2022]
Abstract
The healthcare sector is the highest priority sector, and people demand the highest services and care. The fast rise of deep learning, particularly in clinical decision support tools, has provided exciting solutions primarily in medical imaging. In the past, ANNs (artificial neural networks) have been used extensively in dermatology and have shown promising results for detecting various skin diseases. Eczema represents a group of skin conditions characterized by irritated, dry, inflamed, and itchy skin. This study extends great help to automate the diagnosis process of various kinds of eczema through a Hybrid model that uses concatenated ReliefF optimized handcrafted and deep activated features and a support vector machine for classification. Deep learning models and standard image processing techniques have been used to classify eczema from images automatically. This work contributes to the first multiclass image dataset, namely EIR (Eczema image resource). The EIR dataset consists of 2039 labeled eczema images belonging to seven categories. We performed a comparative analysis of multiple ensemble models, attention mechanisms, and data augmentation techniques for this task. The respective accuracy, sensitivity, and specificity, for eczema classification by classifiers were recorded. In comparison, the proposed Hybrid 6 network achieved the highest accuracy of 88.29%, sensitivity of 85.19%, and specificity of 90.33%% among all employed models. Our findings suggest that deep learning models can classify eczema with high accuracy, and their performance is comparable to dermatologists. However, many factors have been elucidated that contribute to reducing accuracy and potential scope for improvement.
Collapse
Affiliation(s)
- Assad Rasheed
- Department of Information Technology, Hazara University Mansehra, Pakistan.
| | - Arif Iqbal Umar
- Department of Information Technology, Hazara University Mansehra, Pakistan
| | - Syed Hamad Shirazi
- Department of Information Technology, Hazara University Mansehra, Pakistan.
| | - Zakir Khan
- Department of Information Technology, Hazara University Mansehra, Pakistan
| | - Shah Nawaz
- Department of Information Technology, Hazara University Mansehra, Pakistan
| | - Muhammad Shahzad
- Department of Information Technology, Hazara University Mansehra, Pakistan
| |
Collapse
|
18
|
Yu Z, Nguyen J, Nguyen TD, Kelly J, Mclean C, Bonnington P, Zhang L, Mar V, Ge Z. Early Melanoma Diagnosis With Sequential Dermoscopic Images. IEEE TRANSACTIONS ON MEDICAL IMAGING 2022; 41:633-646. [PMID: 34648437 DOI: 10.1109/tmi.2021.3120091] [Citation(s) in RCA: 5] [Impact Index Per Article: 2.5] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/13/2023]
Abstract
Dermatologists often diagnose or rule out early melanoma by evaluating the follow-up dermoscopic images of skin lesions. However, existing algorithms for early melanoma diagnosis are developed using single time-point images of lesions. Ignoring the temporal, morphological changes of lesions can lead to misdiagnosis in borderline cases. In this study, we propose a framework for automated early melanoma diagnosis using sequential dermoscopic images. To this end, we construct our method in three steps. First, we align sequential dermoscopic images of skin lesions using estimated Euclidean transformations, extract the lesion growth region by computing image differences among the consecutive images, and then propose a spatio-temporal network to capture the dermoscopic changes from aligned lesion images and the corresponding difference images. Finally, we develop an early diagnosis module to compute probability scores of malignancy for lesion images over time. We collected 179 serial dermoscopic imaging data from 122 patients to verify our method. Extensive experiments show that the proposed model outperforms other commonly used sequence models. We also compared the diagnostic results of our model with those of seven experienced dermatologists and five registrars. Our model achieved higher diagnostic accuracy than clinicians (63.69% vs. 54.33%, respectively) and provided an earlier diagnosis of melanoma (60.7% vs. 32.7% of melanoma correctly diagnosed on the first follow-up images). These results demonstrate that our model can be used to identify melanocytic lesions that are at high-risk of malignant transformation earlier in the disease process and thereby redefine what is possible in the early detection of melanoma.
Collapse
|
19
|
Smartphone-Based Visual Inspection with Acetic Acid: An Innovative Tool to Improve Cervical Cancer Screening in Low-Resource Setting. Healthcare (Basel) 2022; 10:healthcare10020391. [PMID: 35207002 PMCID: PMC8871553 DOI: 10.3390/healthcare10020391] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/30/2021] [Revised: 02/06/2022] [Accepted: 02/11/2022] [Indexed: 11/17/2022] Open
Abstract
Visual inspection with acetic acid (VIA) is recommended by the World Health Organization for primary cervical cancer screening or triage of human papillomavirus-positive women living in low-resource settings. Nonetheless, traditional VIA with the naked-eye is associated with large variabilities in the detection of pre-cancer and with a lack of quality control. Digital-VIA (D-VIA), using high definition cameras, allows magnification and zooming on transformation zones and suspicious cervical regions, as well as simultaneously compare native and post-VIA images in real-time. We searched MEDLINE and LILACS between January 2015 and November 2021 for relevant studies conducted in low-resource settings using a smartphone device for D-VIA. The aim of this review was to provide an evaluation on available data for smartphone use in low-resource settings in the context of D-VIA-based cervical cancer screenings. The available results to date show that the quality of D-VIA images is satisfactory and enables CIN1/CIN2+ diagnosis, and that a smartphone is a promising tool for cervical cancer screening monitoring and for on- and off-site supervision, and training. The use of artificial intelligence algorithms could soon allow automated and accurate cervical lesion detection.
Collapse
|
20
|
Sies K, Winkler JK, Fink C, Bardehle F, Toberer F, Buhl T, Enk A, Blum A, Stolz W, Rosenberger A, Haenssle HA. Does sex matter? Analysis of sex-related differences in the diagnostic performance of a market-approved convolutional neural network for skin cancer detection. Eur J Cancer 2022; 164:88-94. [PMID: 35182926 DOI: 10.1016/j.ejca.2021.12.034] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/19/2021] [Revised: 12/17/2021] [Accepted: 12/29/2021] [Indexed: 11/03/2022]
Abstract
BACKGROUND Advances in biomedical artificial intelligence may introduce or perpetuate sex and gender discriminations. Convolutional neural networks (CNN) have proven a dermatologist-level performance in image classification tasks but have not been assessed for sex and gender biases that may affect training data and diagnostic performance. In this study, we investigated sex-related imbalances in training data and diagnostic performance of a market-approved CNN for skin cancer classification (Moleanalyzer Pro®, Fotofinder Systems GmbH, Bad Birnbach, Germany). METHODS We screened open-access dermoscopic image repositories widely used for CNN training for distribution of sex. Moreover, the sex-related diagnostic performance of the market-approved CNN was tested in 1549 dermoscopic images stratified by sex (female n = 773; male n = 776). RESULTS Most open-access repositories showed a marked under-representation of images originating from female (40%) versus male (60%) patients. Despite these imbalances and well-known sex-related differences in skin anatomy or skin-directed behaviour, the tested CNN achieved a comparable sensitivity of 87.0% [80.9%-91.3%] versus 87.1% [81.1%-91.4%], specificity of 98.7% [97.4%-99.3%] versus 96.9% [95.2%-98.0%] and ROC-AUC of 0.984 [0.975-0.993] versus 0.979 [0.969-0.988] in dermoscopic images of female versus male origin, respectively. In the sample at hand, sex-related differences in ROC-AUCs were not statistically significant in the per-image analysis nor in an additional per-individual analysis (p ≥ 0.59). CONCLUSION Design and training of artificial intelligence algorithms for medical applications should generally acknowledge sex and gender dimensions. Despite sex-related imbalances in open-access training data, the diagnostic performance of the tested CNN showed no sex-related bias in the classification of skin lesions.
Collapse
Affiliation(s)
- Katharina Sies
- Department of Dermatology, University of Heidelberg, Heidelberg, Germany
| | - Julia K Winkler
- Department of Dermatology, University of Heidelberg, Heidelberg, Germany
| | - Christine Fink
- Department of Dermatology, University of Heidelberg, Heidelberg, Germany
| | - Felicitas Bardehle
- Department of Dermatology, University of Heidelberg, Heidelberg, Germany
| | - Ferdinand Toberer
- Department of Dermatology, University of Heidelberg, Heidelberg, Germany
| | - Timo Buhl
- Department of Dermatology, Venereology and Allergology, University Medical Center Göttingen, Göttingen, Germany
| | - Alexander Enk
- Department of Dermatology, University of Heidelberg, Heidelberg, Germany
| | - Andreas Blum
- Public, Private and Teaching Practice of Dermatology, Konstanz, Germany
| | - Wilhelm Stolz
- Department of Dermatology, Allergology and Environmental Medicine II, Hospital Thalkirchner Street, Munich, Germany
| | - Albert Rosenberger
- Department of Genetic Epidemiology, University of Goettingen, Goettingen, Germany
| | - Holger A Haenssle
- Department of Dermatology, University of Heidelberg, Heidelberg, Germany.
| |
Collapse
|
21
|
Winkler JK, Tschandl P, Toberer F, Sies K, Fink C, Enk A, Kittler H, Haenssle HA. Monitoring patients at risk for melanoma: May convolutional neural networks replace the strategy of sequential digital dermoscopy? Eur J Cancer 2021; 160:180-188. [PMID: 34840028 DOI: 10.1016/j.ejca.2021.10.030] [Citation(s) in RCA: 7] [Impact Index Per Article: 2.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/24/2021] [Revised: 10/06/2021] [Accepted: 10/25/2021] [Indexed: 01/11/2023]
Abstract
BACKGROUND Sequential digital dermoscopy (SDD) is applied for early melanoma detection by uncovering dynamic changes of monitored lesions. Convolutional neural networks (CNN) are capable of high diagnostic accuracies similar to trained dermatologists. OBJECTIVES To investigate the capability of CNN to correctly classify melanomas originally diagnosed by mere dynamic changes during SDD. METHODS A retrospective cross-sectional study using image quartets of 59 high-risk patients each containing one melanoma diagnosed by dynamic changes during SDD and three nevi (236 lesions). Two validated CNN classified quartets at baseline or after SDD follow-up at the time of melanoma diagnosis. Moreover, baseline quartets were rated by 26 dermatologists. The main outcome was the number of quartets with correct classifications. RESULTS CNN-1 correctly classified 9 (15.3%) and CNN-2 8 (13.6%) of 59 baseline quartets. In baseline images, CNN-1 attained a sensitivity of 25.4% (16.1%-37.8%) and specificity of 92.7% (87.8%-95.7%), whereas CNN-2 of 28.8% (18.8%-41.4%) and 75.7% (68.9%-81.4%). Expectedly, after SDD follow-up CNN more readily detected melanomas resulting in improved sensitivities (CNN-1: 44.1% [32.2%-56.7%]; CNN-2: 49.2% [36.8%-61.6%]). Dermatologists were told that each baseline quartet contained one melanoma, and on average, correctly classified 24 (22-27) of 59 quartets. Correspondingly, accepting a baseline quartet to be appropriately classified whenever the highest malignancy score was assigned to the melanoma within, CNN-1 and CNN-2 correctly classified 28 (47.5%) and 22 (37.3%) of 59 quartets, respectively. CONCLUSIONS The tested CNN could not replace the strategy of SDD. There is a need for CNN capable of integrating information on dynamic changes into analyses.
Collapse
Affiliation(s)
- Julia K Winkler
- Department of Dermatology, University of Heidelberg, Heidelberg, Germany
| | - Philipp Tschandl
- Department of Dermatology, Medical University of Vienna, Vienna, Austria
| | - Ferdinand Toberer
- Department of Dermatology, University of Heidelberg, Heidelberg, Germany
| | - Katharina Sies
- Department of Dermatology, University of Heidelberg, Heidelberg, Germany
| | - Christine Fink
- Department of Dermatology, University of Heidelberg, Heidelberg, Germany
| | - Alexander Enk
- Department of Dermatology, University of Heidelberg, Heidelberg, Germany
| | - Harald Kittler
- Department of Dermatology, Medical University of Vienna, Vienna, Austria
| | - Holger A Haenssle
- Department of Dermatology, University of Heidelberg, Heidelberg, Germany.
| |
Collapse
|
22
|
Weber P, Sinz C, Rinner C, Kittler H, Tschandl P. Perilesional sun damage as a diagnostic clue for pigmented actinic keratosis and Bowen's disease. J Eur Acad Dermatol Venereol 2021; 35:2022-2026. [PMID: 34146354 PMCID: PMC8518404 DOI: 10.1111/jdv.17464] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/15/2021] [Accepted: 06/10/2021] [Indexed: 11/29/2022]
Abstract
Background Chronic sun damage in the background is common in pigmented actinic keratoses and Bowen’s disease (pAK/BD). While explainable artificial intelligence (AI) demonstrated increased background attention for pAK/BD, humans frequently miss this clue in dermatoscopic images because they tend to focus on the lesion. Aim To analyse whether perilesional sun damage is a robust diagnostic clue for pAK/BD and if teaching this clue to dermatoscopy users improves their diagnostic accuracy. Methods We assessed the interrater agreement and the frequency of perilesional sun damage in 220 dermatoscopic images and conducted a reader study with 124 dermatoscopy users. The readers were randomly assigned to one of two online tutorials; one tutorial pointed to perilesional sun damage as a clue to pAK/BD (group A) the other did not (group B). In both groups, we compared the frequencies of correct diagnoses before and after receiving the tutorial. Results The frequency of perilesional sun damage was higher in pAK/BD than in other types of pigmented skin lesions and interrater agreement was good (kappa = 0.675). The diagnostic accuracy for pAK/BD improved in both groups of readers (group A: +16.1%, 95%‐CI: 9.5–22.7; group B: +13.1%; 95%‐CI: 7.1–19.0; P for both <0.001), but the overall accuracy improved only in group A from (59.1% (95%‐CI: 55.0–63.1) to 63.5% (95%‐CI: 59.5–67.6); P = 0.002). Conclusion Perilesional sun damage is a good clue to differentiate pAK/BD from other pigmented skin lesions in dermatoscopic images, which could be useful for teledermatology. Knowledge of this clue improves the accuracy of dermatoscopy users, which demonstrates that insights from explainable AI can be used to train humans.
Collapse
Affiliation(s)
- P Weber
- Department of Dermatology, Medical University of Vienna, Vienna, Austria
| | - C Sinz
- Department of Dermatology, Medical University of Vienna, Vienna, Austria
| | - C Rinner
- Center for Medical Statistics, Informatics and Intelligent Systems, Medical University of Vienna, Vienna, Austria
| | - H Kittler
- Department of Dermatology, Medical University of Vienna, Vienna, Austria
| | - P Tschandl
- Department of Dermatology, Medical University of Vienna, Vienna, Austria
| |
Collapse
|
23
|
Sies K, Winkler JK, Fink C, Bardehle F, Toberer F, Kommoss FKF, Buhl T, Enk A, Rosenberger A, Haenssle HA. Auswirkungen des „dunklen Rand‐Artefakts“ in dermatoskopischen Bildern auf die diagnostische Leistungsfähigkeit eines deep learning neuronalen Netzwerkes mit Marktzulassung. J Dtsch Dermatol Ges 2021; 19:842-851. [PMID: 34139087 DOI: 10.1111/ddg.14384_g] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/12/2020] [Accepted: 11/27/2020] [Indexed: 02/06/2023]
Abstract
HINTERGRUND UND ZIELE Systeme künstlicher Intelligenz (durch "deep learning" faltende neuronale Netzwerke; engl. convolutional neural networks, CNN) erreichen inzwischen bei der Klassifikation von Hautläsionen vergleichbar gute Ergebnisse wie Dermatologen. Allerdings müssen die Limitationen solcher Systeme vor flächendeckendem klinischem Einsatz bekannt sein. Daher haben wir den Einfluss des "dunklen Rand-Artefakts" (engl. dark corner artefact; DCA) in dermatoskopischen Bildern auf die diagnostische Leistung eines CNN mit Marktzulassung zur Klassifikation von Hautläsionen untersucht. PATIENTEN UND METHODEN Ein Datensatz aus 233 Bildern von Hautläsionen (60 maligne und 173 benigne) ohne DCA (Kontrolle) wurde digital so modifiziert, dass kleine, mittlere oder große DCA zu sehen waren. Alle 932 Bilder wurden dann mittels CNN mit Marktzulassung (Moleanalyzer-Pro® , FotoFinder Systems) auf Malignitätsscores hin analysiert. Das Spektrum reichte von 0-1; ein Score von > 0,5 wurde als maligne klassifiziert. ERGEBNISSE In der Kontrollserie ohne DCA erreichte das CNN eine Sensitivität von 90,0 % (79,9 %-95,3 %), eine Spezifität von 96,5 % (92,6 %-98,4 %) sowie eine Fläche unter der Kurve (AUC, area under the curve) der "receiver operating characteristic" (ROC) von 0,961 (0,932-0,989). In den Datensätzen mit kleinen beziehungsweise mittleren DCA war die diagnostische Leistung vergleichbar. In den Bildersätzen mit großen DCA wurden allerdings signifikant höhere Malignitätsscores erzielt. Dies führte zu einer signifikant verminderten Spezifität (87,9 % [82,2 %-91,9 %], P < 0,001) sowie einer nicht signifikant erhöhten Sensitivität (96,7 % [88,6 %-99,1 %]). Die ROC-AUC blieb mit 0,962 (0,935-0,989) unverändert. SCHLUSSFOLGERUNGEN Die Klassifizierung mittels des CNN war bei dermatoskopischen Bildern mit kleinen oder mittleren DCA nicht beeinträchtigt, das System zeigte jedoch Schwächen bei großen DCA. Wenn Ärzte solche Bilder zur Klassifikation mittels CNN einreichen, sollten sie sich dieser Grenzen der Technologie bewusst sein.
Collapse
Affiliation(s)
| | | | | | | | | | - Felix K F Kommoss
- Abteilung Pathologie, Institut für Pathologie, Universitätsklinikum Heidelberg
| | - Timo Buhl
- Klinik für Dermatologie, Venerologie und Allergologie, Universitätsmedizin Göttingen
| | | | | | | |
Collapse
|
24
|
Sies K, Winkler JK, Fink C, Bardehle F, Toberer F, Kommoss FKF, Buhl T, Enk A, Rosenberger A, Haenssle HA. Dark corner artefact and diagnostic performance of a market-approved neural network for skin cancer classification. J Dtsch Dermatol Ges 2021; 19:842-850. [PMID: 33973372 DOI: 10.1111/ddg.14384] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/12/2020] [Accepted: 11/27/2020] [Indexed: 01/10/2023]
Abstract
BACKGROUND AND OBJECTIVES Convolutional neural networks (CNN) have proven dermatologist-level performance in skin lesion classification. Prior to a broader clinical application, an assessment of limitations is crucial. Therefore, the influence of a dark tubular periphery in dermatoscopic images (also called dark corner artefact [DCA]) on the diagnostic performance of a market-approved CNN for skin lesion classification was investigated. PATIENTS AND METHODS A prospective image set of 233 skin lesions (60 malignant, 173 benign) without DCA (control-set) was modified to show small, medium or large DCA. All 932 images were analyzed by a market-approved CNN (Moleanalyzer-Pro® , FotoFinder Systems), providing malignancy scores (range 0-1) with the cut-off > 0.5 indicating malignancy. RESULTS In the control-set the CNN achieved a sensitivity of 90.0 % (79.9 % - 95.3 %), a specificity of 96.5 % (92.6 % - 98.4 %), and an area under the curve (AUC) of receiver operating characteristics (ROC) of 0.961 (0.932 - 0.989). Comparable diagnostic performance was observed in the DCAsmall-set and DCAmedium-set. Conversely, in the DCAlarge-set significantly increased malignancy scores triggered a significantly decreased specificity (87.9 % [82.2 % - 91.9 %], P < 0.001), non-significantly increased sensitivity (96.7 % [88.6 % - 99.1 %]) and unchanged ROC-AUC of 0.962 (0.935 - 0.989). CONCLUSIONS Convolutional neural network classification was robust in images with small and medium DCA, but impaired in images with large DCA. Physicians should be aware of this limitation when submitting images to CNN classification.
Collapse
Affiliation(s)
- Katharina Sies
- Department of Dermatology, University of Heidelberg, Heidelberg, Germany
| | - Julia K Winkler
- Department of Dermatology, University of Heidelberg, Heidelberg, Germany
| | - Christine Fink
- Department of Dermatology, University of Heidelberg, Heidelberg, Germany
| | - Felicitas Bardehle
- Department of Dermatology, University of Heidelberg, Heidelberg, Germany
| | - Ferdinand Toberer
- Department of Dermatology, University of Heidelberg, Heidelberg, Germany
| | - Felix K F Kommoss
- Department of Pathology, Institute of Pathology, Heidelberg University Hospital, Heidelberg, Germany
| | - Timo Buhl
- Department of Dermatology, Venereology and Allergology, University Medical Center Göttingen, Goettingen, Germany
| | - Alexander Enk
- Department of Dermatology, University of Heidelberg, Heidelberg, Germany
| | - Albert Rosenberger
- Department of Genetic Epidemiology, University of Göttingen, Goettingen, Germany
| | - Holger A Haenssle
- Department of Dermatology, University of Heidelberg, Heidelberg, Germany
| |
Collapse
|
25
|
Felmingham CM, Adler NR, Ge Z, Morton RL, Janda M, Mar VJ. The Importance of Incorporating Human Factors in the Design and Implementation of Artificial Intelligence for Skin Cancer Diagnosis in the Real World. Am J Clin Dermatol 2021; 22:233-242. [PMID: 33354741 DOI: 10.1007/s40257-020-00574-4] [Citation(s) in RCA: 21] [Impact Index Per Article: 7.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/15/2022]
Abstract
Artificial intelligence (AI) algorithms have been shown to diagnose skin lesions with impressive accuracy in experimental settings. The majority of the literature to date has compared AI and dermatologists as opponents in skin cancer diagnosis. However, in the real-world clinical setting, the clinician will work in collaboration with AI. Existing evidence regarding the integration of such AI diagnostic tools into clinical practice is limited. Human factors, such as cognitive style, personality, experience, preferences, and attitudes may influence clinicians' use of AI. In this review, we consider these human factors and the potential cognitive errors, biases, and unintended consequences that could arise when using an AI skin cancer diagnostic tool in the real world. Integrating this knowledge in the design and implementation of AI technology will assist in ensuring that the end product can be used effectively. Dermatologist leadership in the development of these tools will further improve their clinical relevance and safety.
Collapse
Affiliation(s)
- Claire M Felmingham
- School of Public Health and Preventive Medicine, Monash University, Melbourne, VIC, Australia.
- Victorian Melanoma Service, Alfred Hospital, 55 Commercial Road, Melbourne, VIC, 3004, Australia.
| | - Nikki R Adler
- School of Public Health and Preventive Medicine, Monash University, Melbourne, VIC, Australia
| | - Zongyuan Ge
- Monash eResearch Centre, Monash University, Clayton, Australia
- Department of Electrical and Computer Systems Engineering, Faculty of Engineering, Monash University, Melbourne, VIC, Australia
- Monash-Airdoc Research Centre, Monash University, Melbourne, VIC, Australia
| | - Rachael L Morton
- NHMRC Clinical Trials Centre, Faculty of Medicine and Health, University of Sydney, Camperdown, NSW, Australia
| | - Monika Janda
- Centre for Health Services Research, Faculty of Medicine, The University of Queensland, Brisbane, QLD, Australia
| | - Victoria J Mar
- School of Public Health and Preventive Medicine, Monash University, Melbourne, VIC, Australia
- Victorian Melanoma Service, Alfred Hospital, 55 Commercial Road, Melbourne, VIC, 3004, Australia
| |
Collapse
|
26
|
Winkler JK, Sies K, Fink C, Toberer F, Enk A, Abassi MS, Fuchs T, Haenssle HA. Association between different scale bars in dermoscopic images and diagnostic performance of a market-approved deep learning convolutional neural network for melanoma recognition. Eur J Cancer 2021; 145:146-154. [PMID: 33465706 DOI: 10.1016/j.ejca.2020.12.010] [Citation(s) in RCA: 13] [Impact Index Per Article: 4.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/18/2020] [Revised: 12/01/2020] [Accepted: 12/03/2020] [Indexed: 12/22/2022]
Abstract
BACKGROUND Studies systematically unravelling possible causes for false diagnoses of deep learning convolutional neural networks (CNNs) are scarce, yet needed before broader application. OBJECTIVES The objective of the study was to investigate whether scale bars in dermoscopic images are associated with the diagnostic accuracy of a market-approved CNN. METHODS This cross-sectional analysis applied a CNN trained with more than 150,000 images (Moleanalyzer-pro®, FotoFinder Systems Inc., Bad Birnbach, Germany) to investigate seven dermoscopic image sets depicting the same 130 melanocytic lesions (107 nevi, 23 melanomas) without or with digitally superimposed scale bars of different manufacturers. Sensitivity, specificity and area under the curve (AUC) of receiver operating characteristics (ROC) for the CNN's binary classification of images with or without superimposed scale bars were assessed. RESULTS Six dermoscopic image sets with different scale bars and one control set without scale bars (overall 910 images) were submitted to CNN analysis. In images without scale bars, the CNN attained a sensitivity [95% confidence interval] of 87.0% [67.9%-95.5%] and a specificity of 87.9% [80.3%-92.8%]. ROC AUC was 0.953 [0.914-0.992]. Scale bars were not associated with significant changes in sensitivity (range 87%-95.7%, all p ≥ 1.0). However, four scale bars induced a decrease of the CNN's specificity (range 0%-43.9%, all p < 0.001). Moreover, ROC AUC was significantly reduced by two scale bars (range 0.520-0.848, both p ≤ 0.042). CONCLUSIONS Superimposed scale bars in dermoscopic images may impair the CNN's diagnostic accuracy, mostly by increasing the rate of the false-positive diagnoses. We recommend avoiding scale bars in images intended for CNN analysis unless specific measures counteracting effects are implemented. CLINICAL TRIAL NUMBER This study was registered at the German Clinical Trial Register (DRKS-Study-ID: DRKS00013570; URL: https://www.drks.de/drks_web/).
Collapse
Affiliation(s)
- Julia K Winkler
- Department of Dermatology, University of Heidelberg, Heidelberg, Germany
| | - Katharina Sies
- Department of Dermatology, University of Heidelberg, Heidelberg, Germany
| | - Christine Fink
- Department of Dermatology, University of Heidelberg, Heidelberg, Germany
| | - Ferdinand Toberer
- Department of Dermatology, University of Heidelberg, Heidelberg, Germany
| | - Alexander Enk
- Department of Dermatology, University of Heidelberg, Heidelberg, Germany
| | - Mohamed S Abassi
- Department of Research and Development, FotoFinder Systems GmbH, Bad Birnbach, Germany
| | - Tobias Fuchs
- Department of Research and Development, FotoFinder Systems GmbH, Bad Birnbach, Germany
| | - Holger A Haenssle
- Department of Dermatology, University of Heidelberg, Heidelberg, Germany.
| |
Collapse
|
27
|
Haenssle HA, Winkler JK, Fink C, Toberer F, Enk A, Stolz W, Deinlein T, Hofmann-Wellenhof R, Kittler H, Tschandl P, Rosendahl C, Lallas A, Blum A, Abassi MS, Thomas L, Tromme I, Rosenberger A. Skin lesions of face and scalp - Classification by a market-approved convolutional neural network in comparison with 64 dermatologists. Eur J Cancer 2020; 144:192-199. [PMID: 33370644 DOI: 10.1016/j.ejca.2020.11.034] [Citation(s) in RCA: 15] [Impact Index Per Article: 3.8] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/16/2020] [Revised: 11/02/2020] [Accepted: 11/22/2020] [Indexed: 02/06/2023]
Abstract
BACKGROUND The clinical differentiation of face and scalp lesions (FSLs) is challenging even for trained dermatologists. Studies comparing the diagnostic performance of a convolutional neural network (CNN) with dermatologists in FSL are lacking. METHODS A market-approved CNN (Moleanalyzer-Pro, FotoFinder Systems) was used for binary classifications of 100 dermoscopic images of FSL. The same lesions were used in a two-level reader study including 64 dermatologists (level I: dermoscopy only; level II: dermoscopy, clinical close-up images, textual information). Primary endpoints were the CNN's sensitivity and specificity in comparison with the dermatologists' management decisions in level II. Generalizability of the CNN results was tested by using four additional external data sets. RESULTS The CNN's sensitivity, specificity and ROC AUC were 96.2% [87.0%-98.9%], 68.8% [54.7%-80.1%] and 0.929 [0.880-0.978], respectively. In level II, the dermatologists' management decisions showed a mean sensitivity of 84.2% [82.2%-86.2%] and specificity of 69.4% [66.0%-72.8%]. When fixing the CNN's specificity at the dermatologists' mean specificity (69.4%), the CNN's sensitivity (96.2% [87.0%-98.9%]) was significantly higher than that of dermatologists (84.2% [82.2%-86.2%]; p < 0.001). Dermatologists of all training levels were outperformed by the CNN (all p < 0.001). In confirmation, the CNN's accuracy (83.0%) was significantly higher than dermatologists' accuracies in level II management decisions (all p < 0.001). The CNN's performance was largely confirmed in three additional external data sets but particularly showed a reduced specificity in one Australian data set including FSL on severely sun-damaged skin. CONCLUSIONS When applied as an assistant system, the CNN's higher sensitivity at an equivalent specificity may result in an improved early detection of face and scalp skin cancers.
Collapse
Affiliation(s)
| | | | - Christine Fink
- Department of Dermatology, University of Heidelberg, Heidelberg, Germany
| | - Ferdinand Toberer
- Department of Dermatology, University of Heidelberg, Heidelberg, Germany
| | - Alexander Enk
- Department of Dermatology, University of Heidelberg, Heidelberg, Germany
| | - Wilhelm Stolz
- Department of Dermatology, Allergology and Environmental Medicine II, Hospital Thalkirchner Street, Munich, Germany
| | - Teresa Deinlein
- Department of Dermatology and Venerology, Medical University of Graz, Graz, Austria
| | | | - Harald Kittler
- Department of Dermatology, Medical University of Vienna, Vienna, Austria
| | - Philipp Tschandl
- Department of Dermatology, Medical University of Vienna, Vienna, Austria
| | - Cliff Rosendahl
- School of Medicine, The University of Queensland, Queensland, Australia
| | - Aimilios Lallas
- First Department of Dermatology, Aristotle University, Thessaloniki, Greece
| | - Andreas Blum
- Office Based Clinic of Dermatology, Konstanz, Germany
| | | | - Luc Thomas
- Department of Dermatology, Lyons Cancer Research Center, Lyon 1 University, Lyon, France
| | - Isabelle Tromme
- Department of Dermatology, Université Catholique de Louvain, St. Luc University Hospital, Brussels, Belgium
| | - Albert Rosenberger
- Department of Genetic Epidemiology, University of Goettingen, Goettingen, Germany
| | | |
Collapse
|
28
|
Stojkovic-Filipovic J, Tiodorovic D, Lallas A, Akay BN, Longo C, Rosendahl C, Dobrosavljevic D, Nazzaro G, Argenziano G, Zalaudek I, Tromme I, Tschandl P, Puig S, Lanssens S, Kittler H. Dermatoscopy of combined blue nevi: a multicentre study of the International Dermoscopy Society. J Eur Acad Dermatol Venereol 2020; 35:900-905. [PMID: 33274487 DOI: 10.1111/jdv.17059] [Citation(s) in RCA: 3] [Impact Index Per Article: 0.8] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/30/2020] [Accepted: 11/13/2020] [Indexed: 01/31/2023]
Abstract
BACKGROUND Combined blue nevi (CBN) may mimic melanoma and are relatively often biopsied for diagnostic reasons. OBJECTIVE To better characterize CBN and to compare it with melanoma. METHODS We collected clinical and dermatoscopic images of 111 histologically confirmed CBN and contrasted their dermatoscopic characteristics with 132 partly blue coloured melanomas. Furthermore, we compared the accuracy of human experts using pattern analysis with a computer algorithm based on deep learning. RESULTS Combined blue nevi are usually flat or slightly elevated and, in comparison with melanoma, more frequent on the head and neck. Dermatoscopically, they are typified by a blue structureless part in combination with either brown clods (n = 52, 46.8%), lines (n = 28, 25.2%) or skin-coloured or brown structureless areas (n = 31, 27.9%). In contrast with melanoma, the blue part of CBN is more often well defined (18.9% vs. 4.5%, P < 0.001) and more often located in the centre (22.5% vs. 5.3%, P < 0.001). Melanomas are more often chaotic (OR: 28.7, 95% CI: 14.8-55.7, P < 0.001), have at least one melanoma clue (OR: 10.8, 95% CI: 5.2-22.2 P < 0.001) in particular white lines (OR: 37.1, 95% CI: 13.4-102.9, P < 0.001). Using simplified pattern analysis (chaos and clues), two raters reached sensitivities of 93.9% (95% CI: 88.4-97.3%) and 92.4% (95% CI: 86.5-96.3%) at corresponding specificities of 59.5% (95% CI: 49.7-68.7%) and 65.8% (95% CI: 56.2-74.5%). The human accuracy with pattern analysis was on par with a state-of-the-art computer algorithm based on deep learning that achieved an area under the curve of (0.92, 95% CI: 0.87-0.96) and a specificity of 85.3% (95% CI: 76.5-91.7%) at a given sensitivity of 83.6% (95% CI: 72.5-91.5%). CONCLUSION CBN usually lack melanoma clues, in particular white lines. The accuracy of pattern analysis for combined nevi is acceptable, and histopathologic confirmation may not be necessary in exemplary cases.
Collapse
Affiliation(s)
- J Stojkovic-Filipovic
- Clinic of Dermatology and Venereology, Clinical Center of Serbia, Department of Dermatology, Faculty of Medicine, University of Belgrade, Belgrade, Serbia
| | - D Tiodorovic
- Clinic of Dermatovenereology, Clinical Center of Nis, Medical Faculty, University of Nis, Nis, Serbia
| | - A Lallas
- First Department of Dermatology, Aristotle University, Thessaloniki, Greece
| | - B N Akay
- Medicine Faculty, Department of Dermatology, Ankara University, Ankara, Turkey
| | - C Longo
- Department of Dermatology, University of Modena and Reggio Emilia, Modena, Italy.,Azienda Unità Sanitaria Locale - IRCCS di Reggio Emilia, Centro Oncologico ad Alta Tecnologia Diagnostica-Dermatologia, Reggio Emilia, Italy
| | - C Rosendahl
- Faculty of Medicine, The University of Queensland, Capalaba, QLD, Australia
| | - D Dobrosavljevic
- Clinic of Dermatology and Venereology, Clinical Center of Serbia, Department of Dermatology, Faculty of Medicine, University of Belgrade, Belgrade, Serbia
| | - G Nazzaro
- Dermatology Unit, Fondazione IRCCS Ca' Granda Ospedale Maggiore Policlinico, Milan, Italy
| | - G Argenziano
- Dermatology Unit, University of Campania, Naples, Italy
| | - I Zalaudek
- Department of Dermatology, University of Trieste, Trieste, Italy
| | - I Tromme
- Dermatology Department, King Albert II Institute, Cliniques Universitaires St Luc, Brussels, Belgium
| | - P Tschandl
- Department of Dermatology, Division of General Dermatology, Medical University of Vienna, Vienna, Austria
| | - S Puig
- Dermatology Department, Melanoma Unit, Hospital Clínic de Barcelona, IDIBAPS, Universitat de Barcelona, Barcelona, Spain.,Centro de Investigación Biomédica en Red de Enfermedades Raras, Instituto de Salud Carlos III, Barcelona, Spain
| | - S Lanssens
- Dermatologie Maldegem, Maldegem, Belgium
| | - H Kittler
- Department of Dermatology, Division of General Dermatology, Medical University of Vienna, Vienna, Austria
| |
Collapse
|
29
|
Ilan Y. Second-Generation Digital Health Platforms: Placing the Patient at the Center and Focusing on Clinical Outcomes. Front Digit Health 2020; 2:569178. [PMID: 34713042 PMCID: PMC8521820 DOI: 10.3389/fdgth.2020.569178] [Citation(s) in RCA: 25] [Impact Index Per Article: 6.3] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/03/2020] [Accepted: 10/02/2020] [Indexed: 12/13/2022] Open
Abstract
Artificial intelligence (AI) digital health systems have drawn much attention over the last decade. However, their implementation into medical practice occurs at a much slower pace than expected. This paper reviews some of the achievements of first-generation AI systems, and the barriers facing their implementation into medical practice. The development of second-generation AI systems is discussed with a focus on overcoming some of these obstacles. Second-generation systems are aimed at focusing on a single subject and on improving patients' clinical outcomes. A personalized closed-loop system designed to improve end-organ function and the patient's response to chronic therapies is presented. The system introduces a platform which implements a personalized therapeutic regimen and introduces quantifiable individualized-variability patterns into its algorithm. The platform is designed to achieve a clinically meaningful endpoint by ensuring that chronic therapies will have sustainable effect while overcoming compensatory mechanisms associated with disease progression and drug resistance. Second-generation systems are expected to assist patients and providers in adopting and implementing of these systems into everyday care.
Collapse
|
30
|
Tognetti L, Bonechi S, Andreini P, Bianchini M, Scarselli F, Cevenini G, Moscarella E, Farnetani F, Longo C, Lallas A, Carrera C, Puig S, Tiodorovic D, Perrot JL, Pellacani G, Argenziano G, Cinotti E, Cataldo G, Balistreri A, Mecocci A, Gori M, Rubegni P, Cartocci A. A new deep learning approach integrated with clinical data for the dermoscopic differentiation of early melanomas from atypical nevi. J Dermatol Sci 2020; 101:115-122. [PMID: 33358096 DOI: 10.1016/j.jdermsci.2020.11.009] [Citation(s) in RCA: 18] [Impact Index Per Article: 4.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/29/2020] [Revised: 11/23/2020] [Accepted: 11/30/2020] [Indexed: 12/13/2022]
Abstract
BACKGROUND Timely recognition of malignant melanoma (MM) is challenging for dermatologists worldwide and represents the main determinant for mortality. Dermoscopic examination is influenced by dermatologists' experience and fails to achieve adequate accuracy and reproducibility in discriminating atypical nevi (AN) from early melanomas (EM). OBJECTIVE We aimed to develop a Deep Convolutional Neural Network (DCNN) model able to support dermatologists in the classification and management of atypical melanocytic skin lesions (aMSL). METHODS A training set (630 images), a validation set (135) and a testing set (214) were derived from the idScore dataset of 979 challenging aMSL cases in which the dermoscopic image is integrated with clinical data (age, sex, body site and diameter) and associated with histological data. A DCNN_aMSL architecture was designed and then trained on both dermoscopic images of aMSL and the clinical/anamnestic data, resulting in the integrated "iDCNN_aMSL" model. Responses of 111 dermatologists with different experience levels on both aMSL classification (intuitive diagnosis) and management decisions (no/long follow-up; short follow-up; excision/preventive excision) were compared with the DCNNs models. RESULTS In the lesion classification study, the iDCNN_aMSL achieved the best accuracy, reaching an AUC = 90.3 %, SE = 86.5 % and SP = 73.6 %, compared to DCNN_aMSL (SE = 89.2 %, SP = 65.7 %) and intuitive diagnosis of dermatologists (SE = 77.0 %; SP = 61.4 %). CONCLUSIONS The iDCNN_aMSL proved to be the best support tool for management decisions reducing the ratio of inappropriate excision. The proposed iDCNN_aMSL model can represent a valid support for dermatologists in discriminating AN from EM with high accuracy and for medical decision making by reducing their rates of inappropriate excisions.
Collapse
Affiliation(s)
- Linda Tognetti
- Dermatology Unit, Department of Medical, Surgical and Neurosciences, University of Siena, Italy.
| | - Simone Bonechi
- Department of Information Engineering and Mathematics, University of Siena, Siena, Italy; Department of Economy Engineering Society and Buisiness, Tuscia University, Viterbo, Italy
| | - Paolo Andreini
- Department of Information Engineering and Mathematics, University of Siena, Siena, Italy
| | - Monica Bianchini
- Department of Information Engineering and Mathematics, University of Siena, Siena, Italy
| | - Franco Scarselli
- Department of Information Engineering and Mathematics, University of Siena, Siena, Italy
| | - Gabriele Cevenini
- Bioengineering Unit, Department of Medical Biotechnology, University of Siena, Italy
| | - Elvira Moscarella
- Dermatology Unit, University of Campania Luigi Vanvitelli, Naples, Italy
| | - Francesca Farnetani
- Department of Dermatology, University of Modena and Reggio Emilia, Modena, Italy
| | - Caterina Longo
- Centro Oncologico ad Alta Tecnologia Diagnostica, Azienda Unità Sanitaria Locale, IRCCS di Reggio Emilia, Reggio Emilia, Italy
| | - Aimilios Lallas
- First Department of Dermatology, Aristotle University, Thessaloniki, Greece
| | - Cristina Carrera
- Melanoma Unit, Department of Dermatology, University of Barcelona, Barcelona, Spain; Centro de Investigación Biomédica en Red de Enfermedades Raras (CIBERER), Instituto de Salud Carlos III, University of Barcelona, Barcelona, Spain
| | - Susana Puig
- Melanoma Unit, Department of Dermatology, University of Barcelona, Barcelona, Spain
| | | | - Jean Luc Perrot
- Dermatology Unit, University Hospital of St-Etienne, Saint Etienne, France
| | - Giovanni Pellacani
- Department of Dermatology, University of Modena and Reggio Emilia, Modena, Italy
| | | | - Elisa Cinotti
- Dermatology Unit, Department of Medical, Surgical and Neurosciences, University of Siena, Italy
| | - Gennaro Cataldo
- Bioengineering Unit, Department of Medical Biotechnology, University of Siena, Italy
| | - Alberto Balistreri
- Bioengineering Unit, Department of Medical Biotechnology, University of Siena, Italy
| | - Alessandro Mecocci
- Department of Information Engineering and Mathematics, University of Siena, Siena, Italy
| | - Marco Gori
- Department of Information Engineering and Mathematics, University of Siena, Siena, Italy
| | - Pietro Rubegni
- Dermatology Unit, Department of Medical, Surgical and Neurosciences, University of Siena, Italy
| | - Alessandra Cartocci
- Dermatology Unit, Department of Medical, Surgical and Neurosciences, University of Siena, Italy; Bioengineering Unit, Department of Medical Biotechnology, University of Siena, Italy
| |
Collapse
|
31
|
Blum A, Bosch S, Haenssle HA, Fink C, Hofmann-Wellenhof R, Zalaudek I, Kittler H, Tschandl P. [Artificial intelligence and smartphone program applications (Apps) : Relevance for dermatological practice]. Hautarzt 2020; 71:691-698. [PMID: 32720165 DOI: 10.1007/s00105-020-04658-4] [Citation(s) in RCA: 3] [Impact Index Per Article: 0.8] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/24/2022]
Abstract
ADVANTAGES OF ARTIFICIAL INTELLIGENCE (AI) With responsible, safe and successful use of artificial intelligence (AI), possible advantages in the field of dermato-oncology include the following: (1) medical work can focus on skin cancer patients, (2) patients can be more quickly and effectively treated despite the increasing incidence of skin cancer and the decreasing number of actively working dermatologists and (3) users can learn from the AI results. POTENTIAL DISADVANTAGES AND RISKS OF AI USE: (1) Lack of mutual trust can develop due to the decreased patient-physician contact, (2) additional time effort will be necessary to promptly evaluate the AI-classified benign lesions, (3) lack of adequate medical experience to recognize misclassified AI decisions and (4) recontacting a patient in due time in the case of incorrect AI classifications. Still problematic in the use of AI are the medicolegal situation and remuneration. Apps using AI currently cannot provide sufficient assistance based on clinical images of skin cancer. REQUIREMENTS AND POSSIBLE USE OF SMARTPHONE PROGRAM APPLICATIONS Smartphone program applications (apps) can be implemented responsibly when the image quality is good, the patient's history can be entered easily, transmission of the image and results are assured and medicolegal aspects as well as remuneration are clarified. Apps can be used for disease-specific information material and can optimize patient care by using teledermatology.
Collapse
Affiliation(s)
- A Blum
- Hautarzt- und Lehrpraxis, Augustinerplatz 7, 78462, Konstanz, Deutschland.
| | - S Bosch
- Hautarztpraxis, Ludwigsburg, Deutschland
| | - H A Haenssle
- Universitäts-Hautklinik Heidelberg, Heidelberg, Deutschland
| | - C Fink
- Universitäts-Hautklinik Heidelberg, Heidelberg, Deutschland
| | - R Hofmann-Wellenhof
- Universitätsklinik für Dermatologie, Medizinische Universität Graz, Graz, Österreich
| | - I Zalaudek
- Dermatology Clinic, University Hospital of Trieste, Hospital Maggiore, Trieste, Italien
| | - H Kittler
- Universitätsklinik für Dermatologie, Medizinische Universität Wien, Wien, Österreich
| | - P Tschandl
- Universitätsklinik für Dermatologie, Medizinische Universität Wien, Wien, Österreich
| |
Collapse
|
32
|
Heppt M, Berking C. The value of convolutional neural networks in the diagnosis of melanoma simulators. J Eur Acad Dermatol Venereol 2020; 34:1134-1135. [DOI: 10.1111/jdv.16577] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/29/2020] [Accepted: 04/29/2020] [Indexed: 11/29/2022]
Affiliation(s)
- M.V. Heppt
- Department of Dermatology Universitätsklinikum Erlangen Friedrich‐Alexander‐Universität Erlangen‐Nürnberg (FAU) Erlangen Germany
| | - C. Berking
- Department of Dermatology Universitätsklinikum Erlangen Friedrich‐Alexander‐Universität Erlangen‐Nürnberg (FAU) Erlangen Germany
| |
Collapse
|