Man against machine reloaded: performance of a market-approved convolutional neural network in classifying a broad spectrum of skin lesions in comparison with 96 dermatologists working under less artificial conditions

Improving Skin Color Diversity in Cancer Detection: Deep Learning Approach

Background

The lack of dark skin images in pathologic skin lesions in dermatology resources hinders the accurate diagnosis of skin lesions in people of color. Artificial intelligence applications have further disadvantaged people of color because those applications are mainly trained with light skin color images.

Objective

The aim of this study is to develop a deep learning approach that generates realistic images of darker skin colors to improve dermatology data diversity for various malignant and benign lesions.

Methods

We collected skin clinical images for common malignant and benign skin conditions from DermNet NZ, the International Skin Imaging Collaboration, and Dermatology Atlas. Two deep learning methods, style transfer (ST) and deep blending (DB), were utilized to generate images with darker skin colors using the lighter skin images. The generated images were evaluated quantitively and qualitatively. Furthermore, a convolutional neural network (CNN) was trained using the generated images to assess the latter’s effect on skin lesion classification accuracy.

Results

Image quality assessment showed that the ST method outperformed DB, as the former achieved a lower loss of realism score of 0.23 (95% CI 0.19-0.27) compared to 0.63 (95% CI 0.59-0.67) for the DB method. In addition, ST achieved a higher disease presentation with a similarity score of 0.44 (95% CI 0.40-0.49) compared to 0.17 (95% CI 0.14-0.21) for the DB method. The qualitative assessment completed on masked participants indicated that ST-generated images exhibited high realism, whereby 62.2% (1511/2430) of the votes for the generated images were classified as real. Eight dermatologists correctly diagnosed the lesions in the generated images with an average rate of 0.75 (360 correct diagnoses out of 480) for several malignant and benign lesions. Finally, the classification accuracy and the area under the curve (AUC) of the model when considering the generated images were 0.76 (95% CI 0.72-0.79) and 0.72 (95% CI 0.67-0.77), respectively, compared to the accuracy of 0.56 (95% CI 0.52-0.60) and AUC of 0.63 (95% CI 0.58-0.68) for the model without considering the generated images.

Conclusions

Deep learning approaches can generate realistic skin lesion images that improve the skin color diversity of dermatology atlases. The diversified image bank, utilized herein to train a CNN, demonstrates the potential of developing generalizable artificial intelligence skin cancer diagnosis applications.

International Registered Report Identifier (IRRID)

RR2-10.2196/34896

Classification of skin cancer using convolutional neural networks analysis of Raman spectra

BACKGROUND AND OBJECTIVE

Skin cancer is the most common malignancy in whites accounting for about one third of all cancers diagnosed per year. Portable Raman spectroscopy setups for skin cancer "optical biopsy" are utilized to detect tumors based on their spectral features caused by the comparative presence of different chemical components. However, low signal-to-noise ratio in such systems may prevent accurate tumors classification. Thus, there is a challenge to develop methods for efficient skin tumors classification.

METHODS

We compare the performance of convolutional neural networks and the projection on latent structures with discriminant analysis for discriminating skin cancer using the analysis of Raman spectra with a high autofluorescence background stimulated by a 785 nm laser. We have registered the spectra of 617 cases of skin neoplasms (615 patients, 70 melanomas, 122 basal cell carcinomas, 12 squamous cell carcinomas and 413 benign tumors) in vivo with a portable Raman setup and created classification models both for convolutional neural networks and projection on latent structures approaches. To check the classification models stability, a 10-fold cross-validation was performed for all created models. To avoid models overfitting, the data was divided into a training set (80% of spectral dataset) and a test set (20% of spectral dataset).

RESULTS

The results for different classification tasks demonstrate that the convolutional neural networks significantly (p<0.01) outperforms the projection on latent structures. For the convolutional neural networks implementation we obtained ROC AUCs of 0.96 (0.94 - 0.97; 95% CI), 0.90 (0.85-0.94; 95% CI), and 0.92 (0.87 - 0.97; 95% CI) for classifying a) malignant vs benign tumors, b) melanomas vs pigmented tumors and c) melanomas vs seborrheic keratosis respectively.

CONCLUSIONS

The performance of the convolutional neural networks classification of skin tumors based on Raman spectra analysis is higher or comparable to the accuracy provided by trained dermatologists. The increased accuracy with the convolutional neural networks implementation is due to a more precise accounting of low intensity Raman bands in the intense autofluorescence background. The achieved high performance of skin tumors classifications with convolutional neural networks analysis opens a possibility for wide implementation of Raman setups in clinical setting.

A deep learning approach to detect blood vessels in basal cell carcinoma

PURPOSE

Blood vessels called telangiectasia are visible in skin lesions with the aid of dermoscopy. Telangiectasia are a pivotal identifying feature of basal cell carcinoma. These vessels appear thready, serpiginous, and may also appear arborizing, that is, wide vessels branch into successively thinner vessels. Due to these intricacies, their detection is not an easy task, neither with manual annotation nor with computerized techniques. In this study, we automate the segmentation of telangiectasia in dermoscopic images with a deep learning U-Net approach.

METHODS

We apply a combination of image processing techniques and a deep learning-based U-Net approach to detect telangiectasia in digital basal cell carcinoma skin cancer images. We compare loss functions and optimize the performance by using a combination loss function to manage class imbalance of skin versus vessel pixels.

RESULTS

We establish a baseline method for pixel-based telangiectasia detection in skin cancer lesion images. An analysis and comparison for human observer variability in annotation is also presented.

CONCLUSION

Our approach yields Jaccard score within the variation of human observers as it addresses a new aspect of the rapidly evolving field of deep learning: automatic identification of cancer-specific structures. Further application of DL techniques to detect dermoscopic structures and handle noisy labels is warranted.

Bonnes pratiques en matière de télémédecine

La télémédecine ou médecine à distance s’est imposée aux soignants à la faveur de la pandémie à SARS-Cov2. Elle doit être considérée comme un outil capable d’améliorer la pratique d’une médecine moderne. Ce texte en rappel les règles d’exercice et incite à en organiser l’enseignement.

A two-stage deep learning architecture for radiographic staging of periodontal bone loss

Background

Radiographic periodontal bone loss is one of the most important basis for periodontitis staging, with problems such as limited accuracy, inconsistency, and low efficiency in imaging diagnosis. Deep learning network may be a solution to improve the accuracy and efficiency of periodontitis imaging staging diagnosis. This study aims to establish a comprehensive and accurate radiological staging model of periodontal alveolar bone loss based on panoramic images.

Methods

A total of 640 panoramic images were included, and 3 experienced periodontal physicians marked the key points needed to calculate the degree of periodontal alveolar bone loss and the specific location and shape of the alveolar bone loss. A two-stage deep learning architecture based on UNet and YOLO-v4 was proposed to localize the tooth and key points, so that the percentage of periodontal alveolar bone loss was accurately calculated and periodontitis was staged. The ability of the model to recognize these features was evaluated and compared with that of general dental practitioners.

Results

The overall classification accuracy of the model was 0.77, and the performance of the model varied for different tooth positions and categories; model classification was generally more accurate than that of general practitioners.

Conclusions

It is feasible to establish deep learning model for assessment and staging radiographic periodontal alveolar bone loss using two-stage architecture based on UNet and YOLO-v4.

Real-time high-resolution millimeter-wave imaging for in-vivo skin cancer diagnosis

High-resolution millimeter-wave imaging (HR-MMWI), with its high discrimination contrast and sufficient penetration depth, can potentially provide affordable tissue diagnostic information noninvasively. In this study, we evaluate the application of a real-time system of HR-MMWI for in-vivo skin cancer diagnosis. 136 benign and malignant skin lesions from 71 patients, including melanoma, basal cell carcinoma, squamous cell carcinoma, actinic keratosis, melanocytic nevi, angiokeratoma, dermatofibroma, solar lentigo, and seborrheic keratosis were measured. Lesions were classified using a 3-D principal component analysis followed by five classifiers including linear discriminant analysis (LDA), K-nearest neighbor (KNN) with different K-values, linear and Gaussian support vector machine (LSVM and GSVM) with different margin factors, and multilayer perception (MLP). Our results suggested that the best classification was achieved by using five PCA components followed by MLP with 97% sensitivity and 98% specificity. Our findings establish that real-time millimeter-wave imaging can be used to distinguish malignant tissues from benign skin lesions with high diagnostic accuracy comparable with clinical examination and other methods.

Leveraging Artificial Intelligence to Improve the Diversity of Dermatological Skin Color Pathology: Protocol for an Algorithm Development and Validation Study

Background

The paucity of dark skin images in dermatological textbooks and atlases is a reflection of racial injustice in medicine. The underrepresentation of dark skin images makes diagnosing skin pathology in people of color challenging. For conditions such as skin cancer, in which early diagnosis makes a difference between life and death, people of color have worse prognoses and lower survival rates than people with lighter skin tones as a result of delayed or incorrect diagnoses. Recent advances in artificial intelligence, such as deep learning, offer a potential solution that can be achieved by diversifying the mostly light-skin image repositories through generating images for darker skin tones. Thus, facilitating the development of inclusive cancer early diagnosis systems that are trained and tested on diverse images that truly represent human skin tones.

Objective

We aim to develop and evaluate an artificial intelligence–based skin cancer early detection system for all skin tones using clinical images.

Methods

This study consists of four phases: (1) Publicly available skin image repositories will be analyzed to quantify the underrepresentation of darker skin tones, (2) Images will be generated for the underrepresented skin tones, (3) Generated images will be extensively evaluated for realism and disease presentation with quantitative image quality assessment as well as qualitative human expert and nonexpert ratings, and (4) The images will be utilized with available light-skin images to develop a robust skin cancer early detection model.

Results

This study started in September 2020. The first phase of quantifying the underrepresentation of darker skin tones was completed in March 2021. The second phase of generating the images is in progress and will be completed by March 2022. The third phase is expected to be completed by May 2022, and the final phase is expected to be completed by September 2022.

Conclusions

This work is the first step toward expanding skin tone diversity in existing image databases to address the current gap in the underrepresentation of darker skin tones. Once validated, the image bank will be a valuable resource that can potentially be utilized in physician education and in research applications. Furthermore, generated images are expected to improve the generalizability of skin cancer detection. When completed, the model will assist family physicians and general practitioners in evaluating skin lesion severity and in efficient triaging for referral to expert dermatologists. In addition, the model can assist dermatologists in diagnosing skin lesions.

International Registered Report Identifier (IRRID)

DERR1-10.2196/34896

Early Melanoma Diagnosis With Sequential Dermoscopic Images

Dermatologists often diagnose or rule out early melanoma by evaluating the follow-up dermoscopic images of skin lesions. However, existing algorithms for early melanoma diagnosis are developed using single time-point images of lesions. Ignoring the temporal, morphological changes of lesions can lead to misdiagnosis in borderline cases. In this study, we propose a framework for automated early melanoma diagnosis using sequential dermoscopic images. To this end, we construct our method in three steps. First, we align sequential dermoscopic images of skin lesions using estimated Euclidean transformations, extract the lesion growth region by computing image differences among the consecutive images, and then propose a spatio-temporal network to capture the dermoscopic changes from aligned lesion images and the corresponding difference images. Finally, we develop an early diagnosis module to compute probability scores of malignancy for lesion images over time. We collected 179 serial dermoscopic imaging data from 122 patients to verify our method. Extensive experiments show that the proposed model outperforms other commonly used sequence models. We also compared the diagnostic results of our model with those of seven experienced dermatologists and five registrars. Our model achieved higher diagnostic accuracy than clinicians (63.69% vs. 54.33%, respectively) and provided an earlier diagnosis of melanoma (60.7% vs. 32.7% of melanoma correctly diagnosed on the first follow-up images). These results demonstrate that our model can be used to identify melanocytic lesions that are at high-risk of malignant transformation earlier in the disease process and thereby redefine what is possible in the early detection of melanoma.

Evaluation of Artificial Intelligence-assisted Diagnosis of Skin Neoplasms - a single-center, paralleled, unmasked, randomized controlled trial

A randomized trial (KCT0005614; https://cris.nih.go.kr) was conducted in a tertiary care institute in South Korea, to validate whether artificial intelligence (AI) could augment the accuracy of non-expert physicians in the real-world settings which included diverse out-of-distribution conditions. Four non-dermatology trainees and four dermatology residents examined the randomly allocated patients with skin lesions suspicious of skin cancer with or without the real-time assistance of AI algorithm (https://b2020.modelderm.com#world; convolutional neural networks). Using 576 consecutive cases (Fitzpatrick skin phototypes III or IV) with suspicious lesions out of the initial 603 recruitments, the accuracy of the AI-assisted group (n=295, 53.9%) was significantly higher than those of the Unaided group (n=281, 43.8%; P=0.019). The augmentation was more significant from 54.7% (n=150) to 30.7% (n=138; P<0.0001) in the non-dermatology trainees who had the least experience in dermatology. The augmentation was not significant in the dermatology residents. The algorithm could help the trainees in the AI-assisted group include more differential diagnoses than the Unaided group (2.09 diagnoses versus 1.95; P=0.0005). In this single-center, unmasked, paralleled, randomized controlled trial, the multiclass AI algorithm augmented the diagnostic accuracy of non-expert physicians in dermatology.

Does sex matter? Analysis of sex-related differences in the diagnostic performance of a market-approved convolutional neural network for skin cancer detection

BACKGROUND

Advances in biomedical artificial intelligence may introduce or perpetuate sex and gender discriminations. Convolutional neural networks (CNN) have proven a dermatologist-level performance in image classification tasks but have not been assessed for sex and gender biases that may affect training data and diagnostic performance. In this study, we investigated sex-related imbalances in training data and diagnostic performance of a market-approved CNN for skin cancer classification (Moleanalyzer Pro®, Fotofinder Systems GmbH, Bad Birnbach, Germany).

METHODS

We screened open-access dermoscopic image repositories widely used for CNN training for distribution of sex. Moreover, the sex-related diagnostic performance of the market-approved CNN was tested in 1549 dermoscopic images stratified by sex (female n = 773; male n = 776).

RESULTS

Most open-access repositories showed a marked under-representation of images originating from female (40%) versus male (60%) patients. Despite these imbalances and well-known sex-related differences in skin anatomy or skin-directed behaviour, the tested CNN achieved a comparable sensitivity of 87.0% [80.9%-91.3%] versus 87.1% [81.1%-91.4%], specificity of 98.7% [97.4%-99.3%] versus 96.9% [95.2%-98.0%] and ROC-AUC of 0.984 [0.975-0.993] versus 0.979 [0.969-0.988] in dermoscopic images of female versus male origin, respectively. In the sample at hand, sex-related differences in ROC-AUCs were not statistically significant in the per-image analysis nor in an additional per-individual analysis (p ≥ 0.59).

CONCLUSION

Design and training of artificial intelligence algorithms for medical applications should generally acknowledge sex and gender dimensions. Despite sex-related imbalances in open-access training data, the diagnostic performance of the tested CNN showed no sex-related bias in the classification of skin lesions.

Augmenting the accuracy of trainee doctors in diagnosing skin lesions suspected of skin neoplasms in a real-world setting: A prospective controlled before-and-after study

BACKGROUND

Although deep neural networks have shown promising results in the diagnosis of skin cancer, a prospective evaluation in a real-world setting could confirm these results. This study aimed to evaluate whether an algorithm (http://b2019.modelderm.com) improves the accuracy of nondermatologists in diagnosing skin neoplasms.

METHODS

A total of 285 cases (random series) with skin neoplasms suspected of malignancy by either physicians or patients were recruited in two tertiary care centers located in South Korea. An artificial intelligence (AI) group (144 cases, mean [SD] age, 57.0 [17.7] years; 62 [43.1%] men) was diagnosed via routine examination with photographic review and assistance by the algorithm, whereas the control group (141 cases, mean [SD] age, 61.0 [15.3] years; 52 [36.9%] men) was diagnosed only via routine examination with a photographic review. The accuracy of the nondermatologists before and after the interventions was compared.

RESULTS

Among the AI group, the accuracy of the first impression (Top-1 accuracy; 58.3%) after the assistance of AI was higher than that before the assistance (46.5%, P = .008). The number of differential diagnoses of the participants increased from 1.9 ± 0.5 to 2.2 ± 0.6 after the assistance (P < .001). In the control group, the difference in the Top-1 accuracy between before and after reviewing photographs was not significant (before, 46.1%; after, 51.8%; P = .19), and the number of differential diagnoses did not significantly increase (before, 2.0 ± 0.4; after, 2.1 ± 0.5; P = .57).

CONCLUSIONS

In real-world settings, AI augmented the diagnostic accuracy of trainee doctors. The limitation of this study is that the algorithm was tested only for Asians recruited from a single region. Additional international randomized controlled trials involving various ethnicities are required.

Artificial Intelligence and Melanoma: A Comprehensive Review of Clinical, Dermoscopic, and Histologic Applications

Melanoma detection, prognosis, and treatment represent challenging and complex areas of cutaneous oncology with considerable impact on patient outcomes and healthcare economics. Artificial intelligence (AI) applications in these tasks are rapidly developing. Neural networks with increasing levels of sophistication are being implemented in clinical image, dermoscopic image, and histopathologic specimen classification of pigmented lesions. These efforts hold promise of earlier and highly accurate melanoma detection, as well as reliable prognostication and prediction of therapeutic response. Herein, we provide a brief introduction to AI, discuss contemporary investigational applications of AI in melanoma, and summarize challenges encountered with AI.

Improving Skin cancer Management with ARTificial Intelligence (SMARTI): protocol for a preintervention/postintervention trial of an artificial intelligence system used as a diagnostic aid for skin cancer management in a specialist dermatology setting

INTRODUCTION

Convolutional neural networks (CNNs) can diagnose skin cancers with impressive accuracy in experimental settings, however, their performance in the real-world clinical setting, including comparison to teledermatology services, has not been validated in prospective clinical studies.

METHODS AND ANALYSIS

Participants will be recruited from dermatology clinics at the Alfred Hospital and Skin Health Institute, Melbourne. Skin lesions will be imaged using a proprietary dermoscopic camera. The artificial intelligence (AI) algorithm, a CNN developed by MoleMap Ltd and Monash eResearch, classifies lesions as benign, malignant or uncertain. This is a preintervention/postintervention study. In the preintervention period, treating doctors are blinded to AI lesion assessment. In the postintervention period, treating doctors review the AI lesion assessment in real time, and have the opportunity to then change their diagnosis and management. Any skin lesions of concern and at least two benign lesions will be selected for imaging. Each participant's lesions will be examined by a registrar, the treating consultant dermatologist and later by a teledermatologist. At the conclusion of the preintervention period, the safety of the AI algorithm will be evaluated in a primary analysis by measuring its sensitivity, specificity and agreement with histopathology where available, or the treating consultant dermatologists' classification. At trial completion, AI classifications will be compared with those of the teledermatologist, registrar, treating dermatologist and histopathology. The impact of the AI algorithm on diagnostic and management decisions will be evaluated by: (1) comparing the initial management decision of the registrar with their AI-assisted decision and (2) comparing the benign to malignant ratio (for lesions biopsied) between the preintervention and postintervention periods.

ETHICS AND DISSEMINATION

Human Research Ethics Committee (HREC) approval received from the Alfred Hospital Ethics Committee on 14 February 2019 (HREC/48865/Alfred-2018). Findings from this study will be disseminated through peer-reviewed publications, non-peer reviewed media and conferences.

TRIAL REGISTRATION NUMBER

NCT04040114.

Machine Learning and Its Application in Skin Cancer

Artificial intelligence (AI) has wide applications in healthcare, including dermatology. Machine learning (ML) is a subfield of AI involving statistical models and algorithms that can progressively learn from data to predict the characteristics of new samples and perform a desired task. Although it has a significant role in the detection of skin cancer, dermatology skill lags behind radiology in terms of AI acceptance. With continuous spread, use, and emerging technologies, AI is becoming more widely available even to the general population. AI can be of use for the early detection of skin cancer. For example, the use of deep convolutional neural networks can help to develop a system to evaluate images of the skin to diagnose skin cancer. Early detection is key for the effective treatment and better outcomes of skin cancer. Specialists can accurately diagnose the cancer, however, considering their limited numbers, there is a need to develop automated systems that can diagnose the disease efficiently to save lives and reduce health and financial burdens on the patients. ML can be of significant use in this regard. In this article, we discuss the fundamentals of ML and its potential in assisting the diagnosis of skin cancer.

Monitoring patients at risk for melanoma: May convolutional neural networks replace the strategy of sequential digital dermoscopy?

BACKGROUND

Sequential digital dermoscopy (SDD) is applied for early melanoma detection by uncovering dynamic changes of monitored lesions. Convolutional neural networks (CNN) are capable of high diagnostic accuracies similar to trained dermatologists.

OBJECTIVES

To investigate the capability of CNN to correctly classify melanomas originally diagnosed by mere dynamic changes during SDD.

METHODS

A retrospective cross-sectional study using image quartets of 59 high-risk patients each containing one melanoma diagnosed by dynamic changes during SDD and three nevi (236 lesions). Two validated CNN classified quartets at baseline or after SDD follow-up at the time of melanoma diagnosis. Moreover, baseline quartets were rated by 26 dermatologists. The main outcome was the number of quartets with correct classifications.

RESULTS

CNN-1 correctly classified 9 (15.3%) and CNN-2 8 (13.6%) of 59 baseline quartets. In baseline images, CNN-1 attained a sensitivity of 25.4% (16.1%-37.8%) and specificity of 92.7% (87.8%-95.7%), whereas CNN-2 of 28.8% (18.8%-41.4%) and 75.7% (68.9%-81.4%). Expectedly, after SDD follow-up CNN more readily detected melanomas resulting in improved sensitivities (CNN-1: 44.1% [32.2%-56.7%]; CNN-2: 49.2% [36.8%-61.6%]). Dermatologists were told that each baseline quartet contained one melanoma, and on average, correctly classified 24 (22-27) of 59 quartets. Correspondingly, accepting a baseline quartet to be appropriately classified whenever the highest malignancy score was assigned to the melanoma within, CNN-1 and CNN-2 correctly classified 28 (47.5%) and 22 (37.3%) of 59 quartets, respectively.

CONCLUSIONS

The tested CNN could not replace the strategy of SDD. There is a need for CNN capable of integrating information on dynamic changes into analyses.

Artificial intelligence in arthroplasty

Artificial intelligence (AI) is altering the world of medicine. Given the rapid advances in technology, computers are now able to learn and improve, imitating humanoid cognitive function. AI applications currently exist in various medical specialties, some of which are already in clinical use. This review presents the potential uses and limitations of AI in arthroplasty to provide a better understanding of the existing technology and future direction of this field.Recent literature demonstrates that the utilization of AI in the field of arthroplasty has the potential to improve patient care through better diagnosis, screening, planning, monitoring, and prediction. The implementation of AI technology will enable arthroplasty surgeons to provide patient-specific management in clinical decision making, preoperative health optimization, resource allocation, decision support, and early intervention. While this technology presents a variety of exciting opportunities, it also has several limitations and challenges that need to be overcome to ensure its safety and effectiveness.

Lack of Transparency and Potential Bias in Artificial Intelligence Data Sets and Algorithms: A Scoping Review

Importance

Clinical artificial intelligence (AI) algorithms have the potential to improve clinical care, but fair, generalizable algorithms depend on the clinical data on which they are trained and tested.

Objective

To assess whether data sets used for training diagnostic AI algorithms addressing skin disease are adequately described and to identify potential sources of bias in these data sets.

Data Sources

In this scoping review, PubMed was used to search for peer-reviewed research articles published between January 1, 2015, and November 1, 2020, with the following paired search terms: deep learning and dermatology, artificial intelligence and dermatology, deep learning and dermatologist, and artificial intelligence and dermatologist.

Study Selection

Studies that developed or tested an existing deep learning algorithm for triage, diagnosis, or monitoring using clinical or dermoscopic images of skin disease were selected, and the articles were independently reviewed by 2 investigators to verify that they met selection criteria.

Consensus Process

Data set audit criteria were determined by consensus of all authors after reviewing existing literature to highlight data set transparency and sources of bias.

Results

A total of 70 unique studies were included. Among these studies, 1 065 291 images were used to develop or test AI algorithms, of which only 257 372 (24.2%) were publicly available. Only 14 studies (20.0%) included descriptions of patient ethnicity or race in at least 1 data set used. Only 7 studies (10.0%) included any information about skin tone in at least 1 data set used. Thirty-six of the 56 studies developing new AI algorithms for cutaneous malignant neoplasms (64.3%) met the gold standard criteria for disease labeling. Public data sets were cited more often than private data sets, suggesting that public data sets contribute more to new development and benchmarks.

Conclusions and Relevance

This scoping review identified 3 issues in data sets that are used to develop and test clinical AI algorithms for skin disease that should be addressed before clinical translation: (1) sparsity of data set characterization and lack of transparency, (2) nonstandard and unverified disease labels, and (3) inability to fully assess patient diversity used for algorithm development and testing.

Classification of Basal Cell Carcinoma in Ex Vivo Confocal Microscopy Images from Freshly Excised Tissues Using a Deep Learning Algorithm

Ex vivo confocal microscopy (EVCM) generates digitally colored purple-pink images similar to H&E without time-consuming tissue processing. It can be used during Mohs surgery for rapid detection of basal cell carcinoma (BCC); however, reading EVCM images requires specialized training. An automated approach using a deep learning algorithm for BCC detection in EVCM images can aid in diagnosis. A total of 40 BCCs and 28 negative (not-BCC) samples were collected at Memorial Sloan Kettering Cancer Center to create three training datasets: (i) EVCM image dataset (663 images), (ii) H&E image dataset (516 images), and (iii) a combination of the two datasets. A total of seven BCCs and four negative samples were collected to create an EVCM test dataset (107 images). The model trained with the EVCM dataset achieved 92% diagnostic accuracy, similar to the H&E model (93%). The area under the receiver operator characteristic curve was 0.94, 0.95, and 0.94 for EVCM-, H&E-, and combination-trained models, respectively. We developed an algorithm for automatic BCC detection in EVCM images (comparable accuracy to dermatologists). This approach could be used to assist with BCC detection during Mohs surgery. Furthermore, we found that a model trained with only H&E images (which are more available than EVCM images) can accurately detect BCC in EVCM images.

A Cloud Approach for Melanoma Detection based on Deep Learning Networks

In the era of digitized images, the goal is to be able to extract information from them and create new knowledge thanks to the use of Computer Vision techniques, Machine Learning and Deep Learning. This allows their use for early diagnosis and subsequent determination of the treatment of many pathologies. In the specific case treated here, deep neural networks are used in the dermatological field to distinguish between melanoma and non-melanoma images. In this work we have underlined two essential points of melanoma detection research. The first aspect taken into consideration is how even a simple modification of the parameters in the dataset determines a change of the accuracy of the classifiers, while working on the same original dataset. The second point is the need to have a system architecture that can be more flexible in updating the training datasets for the classification of this pathology. In this context, the proposed architecture reserves the goal of developing and implementing a hybrid architecture based on Cloud, Fog and Edge Computing in order to provide a Melanoma Detection service based on clinical and/or dermoscopic images. At the same time, this architecture must be able to interface with the amount of data to be analyzed by reducing the running time of the necessary computational operations. This has been highlighted with experiments carried out on a single machine and on different distribution systems, highlighting how a distributed approach guarantees the achievement of an output in a much more acceptable time without the need to fully rely on data scientists skills.

Skin cancer classification via convolutional neural networks: systematic review of studies involving human experts

BACKGROUND

Multiple studies have compared the performance of artificial intelligence (AI)-based models for automated skin cancer classification to human experts, thus setting the cornerstone for a successful translation of AI-based tools into clinicopathological practice.

OBJECTIVE

The objective of the study was to systematically analyse the current state of research on reader studies involving melanoma and to assess their potential clinical relevance by evaluating three main aspects: test set characteristics (holdout/out-of-distribution data set, composition), test setting (experimental/clinical, inclusion of metadata) and representativeness of participating clinicians.

METHODS

PubMed, Medline and ScienceDirect were screened for peer-reviewed studies published between 2017 and 2021 and dealing with AI-based skin cancer classification involving melanoma. The search terms skin cancer classification, deep learning, convolutional neural network (CNN), melanoma (detection), digital biomarkers, histopathology and whole slide imaging were combined. Based on the search results, only studies that considered direct comparison of AI results with clinicians and had a diagnostic classification as their main objective were included.

RESULTS

A total of 19 reader studies fulfilled the inclusion criteria. Of these, 11 CNN-based approaches addressed the classification of dermoscopic images; 6 concentrated on the classification of clinical images, whereas 2 dermatopathological studies utilised digitised histopathological whole slide images.

CONCLUSIONS

All 19 included studies demonstrated superior or at least equivalent performance of CNN-based classifiers compared with clinicians. However, almost all studies were conducted in highly artificial settings based exclusively on single images of the suspicious lesions. Moreover, test sets mainly consisted of holdout images and did not represent the full range of patient populations and melanoma subtypes encountered in clinical practice.

Artificial intelligence: how it works and criteria for assessment

Artificial intelligence is a term used to describe computer technology in the modeling of intelligent behavior and critical thinking comparable to that of humans. To date, some of the first areas of medicine to be influenced by advances in artificial intelligence technologies will be those most dependent on imaging. These include ophthalmology, radiology, and dermatology. In connection with the emergence of numerous medical applications, scientists have formulated criteria for their assessment. This list included: clinical validation, regular application updates, functional focus, cost, availability of an information block for specialists and patients, compliance with the conditions of government regulation, and registration. One of the applications that meet all the requirements is the ProRodinki software package, developed for use by patients and specialists in the Russian Federation. Taking into account a widespread and rapidly developing competitive environment, it is necessary to soberly treat the resources of such applications, not exaggerating their capabilities and not considering them as a substitute for a specialist.

Gastrointestinal cancer classification and prognostication from histology using deep learning: Systematic review

BACKGROUND

Gastrointestinal cancers account for approximately 20% of all cancer diagnoses and are responsible for 22.5% of cancer deaths worldwide. Artificial intelligence-based diagnostic support systems, in particular convolutional neural network (CNN)-based image analysis tools, have shown great potential in medical computer vision. In this systematic review, we summarise recent studies reporting CNN-based approaches for digital biomarkers for characterization and prognostication of gastrointestinal cancer pathology.

METHODS

Pubmed and Medline were screened for peer-reviewed papers dealing with CNN-based gastrointestinal cancer analyses from histological slides, published between 2015 and 2020.Seven hundred and ninety titles and abstracts were screened, and 58 full-text articles were assessed for eligibility.

RESULTS

Sixteen publications fulfilled our inclusion criteria dealing with tumor or precursor lesion characterization or prognostic and predictive biomarkers: 14 studies on colorectal or rectal cancer, three studies on gastric cancer and none on esophageal cancer. These studies were categorised according to their end-points: polyp characterization, tumor characterization and patient outcome. Regarding the translation into clinical practice, we identified several studies demonstrating generalization of the classifier with external tests and comparisons with pathologists, but none presenting clinical implementation.

CONCLUSIONS

Results of recent studies on CNN-based image analysis in gastrointestinal cancer pathology are promising, but studies were conducted in observational and retrospective settings. Large-scale trials are needed to assess performance and predict clinical usefulness. Furthermore, large-scale trials are required for approval of CNN-based prediction models as medical devices.

Integrating Patient Data Into Skin Cancer Classification Using Convolutional Neural Networks: Systematic Review

Background

Recent years have been witnessing a substantial improvement in the accuracy of skin cancer classification using convolutional neural networks (CNNs). CNNs perform on par with or better than dermatologists with respect to the classification tasks of single images. However, in clinical practice, dermatologists also use other patient data beyond the visual aspects present in a digitized image, further increasing their diagnostic accuracy. Several pilot studies have recently investigated the effects of integrating different subtypes of patient data into CNN-based skin cancer classifiers.

Objective

This systematic review focuses on the current research investigating the impact of merging information from image features and patient data on the performance of CNN-based skin cancer image classification. This study aims to explore the potential in this field of research by evaluating the types of patient data used, the ways in which the nonimage data are encoded and merged with the image features, and the impact of the integration on the classifier performance.

Methods

Google Scholar, PubMed, MEDLINE, and ScienceDirect were screened for peer-reviewed studies published in English that dealt with the integration of patient data within a CNN-based skin cancer classification. The search terms skin cancer classification, convolutional neural network(s), deep learning, lesions, melanoma, metadata, clinical information, and patient data were combined.

Results

A total of 11 publications fulfilled the inclusion criteria. All of them reported an overall improvement in different skin lesion classification tasks with patient data integration. The most commonly used patient data were age, sex, and lesion location. The patient data were mostly one-hot encoded. There were differences in the complexity that the encoded patient data were processed with regarding deep learning methods before and after fusing them with the image features for a combined classifier.

Conclusions

This study indicates the potential benefits of integrating patient data into CNN-based diagnostic algorithms. However, how exactly the individual patient data enhance classification performance, especially in the case of multiclass classification problems, is still unclear. Moreover, a substantial fraction of patient data used by dermatologists remains to be analyzed in the context of CNN-based skin cancer classification. Further exploratory analyses in this promising field may optimize patient data integration into CNN-based skin cancer diagnostics for patients’ benefits.

Auswirkungen des „dunklen Rand‐Artefakts“ in dermatoskopischen Bildern auf die diagnostische Leistungsfähigkeit eines deep learning neuronalen Netzwerkes mit Marktzulassung

HINTERGRUND UND ZIELE

Systeme künstlicher Intelligenz (durch "deep learning" faltende neuronale Netzwerke; engl. convolutional neural networks, CNN) erreichen inzwischen bei der Klassifikation von Hautläsionen vergleichbar gute Ergebnisse wie Dermatologen. Allerdings müssen die Limitationen solcher Systeme vor flächendeckendem klinischem Einsatz bekannt sein. Daher haben wir den Einfluss des "dunklen Rand-Artefakts" (engl. dark corner artefact; DCA) in dermatoskopischen Bildern auf die diagnostische Leistung eines CNN mit Marktzulassung zur Klassifikation von Hautläsionen untersucht.

PATIENTEN UND METHODEN

Ein Datensatz aus 233 Bildern von Hautläsionen (60 maligne und 173 benigne) ohne DCA (Kontrolle) wurde digital so modifiziert, dass kleine, mittlere oder große DCA zu sehen waren. Alle 932 Bilder wurden dann mittels CNN mit Marktzulassung (Moleanalyzer-Pro^® , FotoFinder Systems) auf Malignitätsscores hin analysiert. Das Spektrum reichte von 0-1; ein Score von > 0,5 wurde als maligne klassifiziert.

ERGEBNISSE

In der Kontrollserie ohne DCA erreichte das CNN eine Sensitivität von 90,0 % (79,9 %-95,3 %), eine Spezifität von 96,5 % (92,6 %-98,4 %) sowie eine Fläche unter der Kurve (AUC, area under the curve) der "receiver operating characteristic" (ROC) von 0,961 (0,932-0,989). In den Datensätzen mit kleinen beziehungsweise mittleren DCA war die diagnostische Leistung vergleichbar. In den Bildersätzen mit großen DCA wurden allerdings signifikant höhere Malignitätsscores erzielt. Dies führte zu einer signifikant verminderten Spezifität (87,9 % [82,2 %-91,9 %], P < 0,001) sowie einer nicht signifikant erhöhten Sensitivität (96,7 % [88,6 %-99,1 %]). Die ROC-AUC blieb mit 0,962 (0,935-0,989) unverändert.

SCHLUSSFOLGERUNGEN

Die Klassifizierung mittels des CNN war bei dermatoskopischen Bildern mit kleinen oder mittleren DCA nicht beeinträchtigt, das System zeigte jedoch Schwächen bei großen DCA. Wenn Ärzte solche Bilder zur Klassifikation mittels CNN einreichen, sollten sie sich dieser Grenzen der Technologie bewusst sein.

Collective human intelligence outperforms artificial intelligence in a skin lesion classification task

BACKGROUND AND OBJECTIVES

Convolutional neural networks (CNN) enable accurate diagnosis of medical images and perform on or above the level of individual physicians. Recently, collective human intelligence (CoHI) was shown to exceed the diagnostic accuracy of individuals. Thus, diagnostic performance of CoHI (120 dermatologists) versus individual dermatologists versus two state-of-the-art CNN was investigated.

PATIENTS AND METHODS

Cross-sectional reader study with presentation of 30 clinical cases to 120 dermatologists. Six diagnoses were offered and votes collected via remote voting devices (quizzbox®, Quizzbox Solutions GmbH, Stuttgart, Germany). Dermatoscopic images were classified by a binary and multiclass CNN (FotoFinder Systems GmbH, Bad Birnbach, Germany). Three sets of diagnostic classifications were scored against ground truth: (1) CoHI, (2) individual dermatologists, and (3) CNN.

RESULTS

CoHI attained a significantly higher accuracy [95 % confidence interval] (80.0 % [62.7 %-90.5 %]) than individual dermatologists (75.7 % [73.8 %-77.5 %]) and CNN (70.0 % [52.1 %-83.3 %]; all P < 0.001) in binary classifications. Moreover, CoHI achieved a higher sensitivity (82.4 % [59.0 %-93.8 %]) and specificity (76.9 % [49.7 %-91.8 %]) than individual dermatologists (sensitivity 77.8 % [75.3 %-80.2 %], specificity 73.0 % [70.6 %-75.4 %]) and CNN (sensitivity 70.6 % [46.9 %-86.7 %], specificity 69.2 % [42.4 %-87.3 %]). The diagnostic accuracy of CoHI was superior to that of individual dermatologists (P < 0.001) in multiclass evaluation, with the accuracy of the latter comparable to multiclass CNN.

CONCLUSIONS

Our analysis revealed that the majority vote of an interconnected group of dermatologists (CoHI) outperformed individuals and CNN in a demanding skin lesion classification task.

Dark corner artefact and diagnostic performance of a market-approved neural network for skin cancer classification

BACKGROUND AND OBJECTIVES

Convolutional neural networks (CNN) have proven dermatologist-level performance in skin lesion classification. Prior to a broader clinical application, an assessment of limitations is crucial. Therefore, the influence of a dark tubular periphery in dermatoscopic images (also called dark corner artefact [DCA]) on the diagnostic performance of a market-approved CNN for skin lesion classification was investigated.

PATIENTS AND METHODS

A prospective image set of 233 skin lesions (60 malignant, 173 benign) without DCA (control-set) was modified to show small, medium or large DCA. All 932 images were analyzed by a market-approved CNN (Moleanalyzer-Pro^® , FotoFinder Systems), providing malignancy scores (range 0-1) with the cut-off > 0.5 indicating malignancy.

RESULTS

In the control-set the CNN achieved a sensitivity of 90.0 % (79.9 % - 95.3 %), a specificity of 96.5 % (92.6 % - 98.4 %), and an area under the curve (AUC) of receiver operating characteristics (ROC) of 0.961 (0.932 - 0.989). Comparable diagnostic performance was observed in the DCAsmall-set and DCAmedium-set. Conversely, in the DCAlarge-set significantly increased malignancy scores triggered a significantly decreased specificity (87.9 % [82.2 % - 91.9 %], P < 0.001), non-significantly increased sensitivity (96.7 % [88.6 % - 99.1 %]) and unchanged ROC-AUC of 0.962 (0.935 - 0.989).

CONCLUSIONS

Convolutional neural network classification was robust in images with small and medium DCA, but impaired in images with large DCA. Physicians should be aware of this limitation when submitting images to CNN classification.

Patientensicherheit in der Dermatologie: Optionen zu ihrer Optimierung

Um die Patientensicherheit in der Dermatologie zu verbessern, gilt es, „vermeidbare unerwünschte medizinische Ereignisse“, definiert als „Patienten schadende Vorkommnisse, die eher auf der Behandlung als auf der Erkrankung selbst beruhen und die durch einen Fehler verursacht sind“, zu minimieren. Bereits die Problemwahrnehmung von möglichen Behandlungsfehlern ist dabei ein wichtiger erster Schritt. Diese Bewusstseinsschärfung geschieht wesentlich dadurch, dass alle Aspekte der Patientensicherheit in die ärztliche Aus- und Weiterbildung sowie in Fortbildungen integriert werden. Für die tägliche Praxis von medizinischen Einrichtungen spielt nach den Vorgaben des Gesetzgebers das Qualitätsmanagement eine wesentliche Rolle, in dem die Patientensicherheit als eine Priorität Berücksichtigung findet. Diese Qualitäts- und Sicherheitsorientierung muss als Führungsaufgabe verstanden werden, der auch angesichts konkurrierender, insbesondere ökonomischer, Unternehmensziele Vorrang einzuräumen ist. Mit der obligatorischen Einführung von Patientensicherheitsbeauftragten in Krankenhäusern, wie aktuell in Hessen erfolgt, kann dem Thema im Klinikmanagement eine wichtige Stimme verliehen werden. Neben der zu fördernden Patientenpartizipation am Behandlungsprozess auch bez. der Patientensicherheit ergeben sich gerade auch in der Dermatologie erhebliche Potenziale durch die Digitalisierung des Gesundheitswesens (e-Health). Auch wenn diese eigene Risikopotenziale beinhaltet, könnte sie zur Diagnose-, Therapie- und Koordinations- und Kommunikationssicherheit in der Dermatologie beitragen.

A survey of clinicians on the use of artificial intelligence in ophthalmology, dermatology, radiology and radiation oncology

Artificial intelligence technology has advanced rapidly in recent years and has the potential to improve healthcare outcomes. However, technology uptake will be largely driven by clinicians, and there is a paucity of data regarding the attitude that clinicians have to this new technology. In June-August 2019 we conducted an online survey of fellows and trainees of three specialty colleges (ophthalmology, radiology/radiation oncology, dermatology) in Australia and New Zealand on artificial intelligence. There were 632 complete responses (n = 305, 230, and 97, respectively), equating to a response rate of 20.4%, 5.1%, and 13.2% for the above colleges, respectively. The majority (n = 449, 71.0%) believed artificial intelligence would improve their field of medicine, and that medical workforce needs would be impacted by the technology within the next decade (n = 542, 85.8%). Improved disease screening and streamlining of monotonous tasks were identified as key benefits of artificial intelligence. The divestment of healthcare to technology companies and medical liability implications were the greatest concerns. Education was identified as a priority to prepare clinicians for the implementation of artificial intelligence in healthcare. This survey highlights parallels between the perceptions of different clinician groups in Australia and New Zealand about artificial intelligence in medicine. Artificial intelligence was recognized as valuable technology that will have wide-ranging impacts on healthcare.

Designing deep learning studies in cancer diagnostics

The number of publications on deep learning for cancer diagnostics is rapidly increasing, and systems are frequently claimed to perform comparable with or better than clinicians. However, few systems have yet demonstrated real-world medical utility. In this Perspective, we discuss reasons for the moderate progress and describe remedies designed to facilitate transition to the clinic. Recent, presumably influential, deep learning studies in cancer diagnostics, of which the vast majority used images as input to the system, are evaluated to reveal the status of the field. By manipulating real data, we then exemplify that much and varied training data facilitate the generalizability of neural networks and thus the ability to use them clinically. To reduce the risk of biased performance estimation of deep learning systems, we advocate evaluation in external cohorts and strongly advise that the planned analyses, including a predefined primary analysis, are described in a protocol preferentially stored in an online repository. Recommended protocol items should be established for the field, and we present our suggestions.

The Importance of Incorporating Human Factors in the Design and Implementation of Artificial Intelligence for Skin Cancer Diagnosis in the Real World

Artificial intelligence (AI) algorithms have been shown to diagnose skin lesions with impressive accuracy in experimental settings. The majority of the literature to date has compared AI and dermatologists as opponents in skin cancer diagnosis. However, in the real-world clinical setting, the clinician will work in collaboration with AI. Existing evidence regarding the integration of such AI diagnostic tools into clinical practice is limited. Human factors, such as cognitive style, personality, experience, preferences, and attitudes may influence clinicians' use of AI. In this review, we consider these human factors and the potential cognitive errors, biases, and unintended consequences that could arise when using an AI skin cancer diagnostic tool in the real world. Integrating this knowledge in the design and implementation of AI technology will assist in ensuring that the end product can be used effectively. Dermatologist leadership in the development of these tools will further improve their clinical relevance and safety.

The Role of DICOM in Artificial Intelligence for Skin Disease

There is optimism that artificial intelligence (AI) will result in positive clinical outcomes, which is driving research and investment in the use of AI for skin disease. At present, AI for skin disease is embedded in research and development and not practiced widely in clinical dermatology. Clinical dermatology is also undergoing a technological transformation in terms of the development and adoption of standards that optimizes the quality use of imaging. Digital Imaging and Communications in Medicine (DICOM) is the international standard for medical imaging. DICOM is a continually evolving standard. There is considerable effort being invested in developing dermatology-specific extensions to the DICOM standard. The ability to encode relevant metadata and afford interoperability with the digital health ecosystem (e.g., image repositories, electronic medical records) has driven the initial impetus in the adoption of DICOM for dermatology. DICOM has a dedicated working group whose role is to develop a mechanism to support AI workflows and encode AI artifacts. DICOM can improve AI workflows by encoding derived objects (e.g., secondary images, visual explainability maps, AI algorithm output) and the efficient curation of multi-institutional datasets for machine learning training, testing, and validation. This can be achieved using DICOM mechanisms such as standardized image formats and metadata, metadata-based image retrieval, and de-identification protocols. DICOM can address several important technological and workflow challenges for the implementation of AI. However, many other technological, ethical, regulatory, medicolegal, and workforce barriers will need to be addressed before DICOM and AI can be used effectively in dermatology.

Stress testing reveals gaps in clinic readiness of image-based diagnostic artificial intelligence models

Artificial intelligence models match or exceed dermatologists in melanoma image classification. Less is known about their robustness against real-world variations, and clinicians may incorrectly assume that a model with an acceptable area under the receiver operating characteristic curve or related performance metric is ready for clinical use. Here, we systematically assessed the performance of dermatologist-level convolutional neural networks (CNNs) on real-world non-curated images by applying computational "stress tests". Our goal was to create a proxy environment in which to comprehensively test the generalizability of off-the-shelf CNNs developed without training or evaluation protocols specific to individual clinics. We found inconsistent predictions on images captured repeatedly in the same setting or subjected to simple transformations (e.g., rotation). Such transformations resulted in false positive or negative predictions for 6.5-22% of skin lesions across test datasets. Our findings indicate that models meeting conventionally reported metrics need further validation with computational stress tests to assess clinic readiness.

Association between different scale bars in dermoscopic images and diagnostic performance of a market-approved deep learning convolutional neural network for melanoma recognition

BACKGROUND

Studies systematically unravelling possible causes for false diagnoses of deep learning convolutional neural networks (CNNs) are scarce, yet needed before broader application.

OBJECTIVES

The objective of the study was to investigate whether scale bars in dermoscopic images are associated with the diagnostic accuracy of a market-approved CNN.

METHODS

This cross-sectional analysis applied a CNN trained with more than 150,000 images (Moleanalyzer-pro®, FotoFinder Systems Inc., Bad Birnbach, Germany) to investigate seven dermoscopic image sets depicting the same 130 melanocytic lesions (107 nevi, 23 melanomas) without or with digitally superimposed scale bars of different manufacturers. Sensitivity, specificity and area under the curve (AUC) of receiver operating characteristics (ROC) for the CNN's binary classification of images with or without superimposed scale bars were assessed.

RESULTS

Six dermoscopic image sets with different scale bars and one control set without scale bars (overall 910 images) were submitted to CNN analysis. In images without scale bars, the CNN attained a sensitivity [95% confidence interval] of 87.0% [67.9%-95.5%] and a specificity of 87.9% [80.3%-92.8%]. ROC AUC was 0.953 [0.914-0.992]. Scale bars were not associated with significant changes in sensitivity (range 87%-95.7%, all p ≥ 1.0). However, four scale bars induced a decrease of the CNN's specificity (range 0%-43.9%, all p < 0.001). Moreover, ROC AUC was significantly reduced by two scale bars (range 0.520-0.848, both p ≤ 0.042).

CONCLUSIONS

Superimposed scale bars in dermoscopic images may impair the CNN's diagnostic accuracy, mostly by increasing the rate of the false-positive diagnoses. We recommend avoiding scale bars in images intended for CNN analysis unless specific measures counteracting effects are implemented.

CLINICAL TRIAL NUMBER

This study was registered at the German Clinical Trial Register (DRKS-Study-ID: DRKS00013570; URL: https://www.drks.de/drks_web/).

Skin lesions of face and scalp - Classification by a market-approved convolutional neural network in comparison with 64 dermatologists

BACKGROUND

The clinical differentiation of face and scalp lesions (FSLs) is challenging even for trained dermatologists. Studies comparing the diagnostic performance of a convolutional neural network (CNN) with dermatologists in FSL are lacking.

METHODS

A market-approved CNN (Moleanalyzer-Pro, FotoFinder Systems) was used for binary classifications of 100 dermoscopic images of FSL. The same lesions were used in a two-level reader study including 64 dermatologists (level I: dermoscopy only; level II: dermoscopy, clinical close-up images, textual information). Primary endpoints were the CNN's sensitivity and specificity in comparison with the dermatologists' management decisions in level II. Generalizability of the CNN results was tested by using four additional external data sets.

RESULTS

The CNN's sensitivity, specificity and ROC AUC were 96.2% [87.0%-98.9%], 68.8% [54.7%-80.1%] and 0.929 [0.880-0.978], respectively. In level II, the dermatologists' management decisions showed a mean sensitivity of 84.2% [82.2%-86.2%] and specificity of 69.4% [66.0%-72.8%]. When fixing the CNN's specificity at the dermatologists' mean specificity (69.4%), the CNN's sensitivity (96.2% [87.0%-98.9%]) was significantly higher than that of dermatologists (84.2% [82.2%-86.2%]; p < 0.001). Dermatologists of all training levels were outperformed by the CNN (all p < 0.001). In confirmation, the CNN's accuracy (83.0%) was significantly higher than dermatologists' accuracies in level II management decisions (all p < 0.001). The CNN's performance was largely confirmed in three additional external data sets but particularly showed a reduced specificity in one Australian data set including FSL on severely sun-damaged skin.

CONCLUSIONS

When applied as an assistant system, the CNN's higher sensitivity at an equivalent specificity may result in an improved early detection of face and scalp skin cancers.

Perceptions of virtual primary care physicians: A focus group study of medical and data science graduate students

BACKGROUND

Artificial and virtual technologies in healthcare have advanced rapidly, and healthcare systems have been adapting care accordingly. An intriguing new development is the virtual physician, which can diagnose and treat patients independently.

METHODS AND FINDINGS

This qualitative study of advanced degree students aimed to assess their perceptions of using a virtual primary care physician as a patient. Four focus groups were held: first year medical students, fourth year medical students, first year engineering/data science graduate students, and fourth year engineering/data science graduate students. The focus groups were audiotaped, transcribed verbatim, and content analyses of the transcripts was performed using a data-driven inductive approach. Themes identified concerned advantages, disadvantages, and the future of virtual primary care physicians. Within those main categories, 13 themes emerged and 31 sub-themes.

DISCUSSION

While participants appreciated that a virtual primary care physician would be convenient, efficient, and cost-effective, they also expressed concern about data privacy and the potential for misdiagnosis. To garner trust from its potential users, future virtual primary physicians should be programmed with a sufficient amount of trustworthy data and have a high level of transparency and accountability for patients.

A new deep learning approach integrated with clinical data for the dermoscopic differentiation of early melanomas from atypical nevi

BACKGROUND

Timely recognition of malignant melanoma (MM) is challenging for dermatologists worldwide and represents the main determinant for mortality. Dermoscopic examination is influenced by dermatologists' experience and fails to achieve adequate accuracy and reproducibility in discriminating atypical nevi (AN) from early melanomas (EM).

OBJECTIVE

We aimed to develop a Deep Convolutional Neural Network (DCNN) model able to support dermatologists in the classification and management of atypical melanocytic skin lesions (aMSL).

METHODS

A training set (630 images), a validation set (135) and a testing set (214) were derived from the idScore dataset of 979 challenging aMSL cases in which the dermoscopic image is integrated with clinical data (age, sex, body site and diameter) and associated with histological data. A DCNN_aMSL architecture was designed and then trained on both dermoscopic images of aMSL and the clinical/anamnestic data, resulting in the integrated "iDCNN_aMSL" model. Responses of 111 dermatologists with different experience levels on both aMSL classification (intuitive diagnosis) and management decisions (no/long follow-up; short follow-up; excision/preventive excision) were compared with the DCNNs models.

RESULTS

In the lesion classification study, the iDCNN_aMSL achieved the best accuracy, reaching an AUC = 90.3 %, SE = 86.5 % and SP = 73.6 %, compared to DCNN_aMSL (SE = 89.2 %, SP = 65.7 %) and intuitive diagnosis of dermatologists (SE = 77.0 %; SP = 61.4 %).

CONCLUSIONS

The iDCNN_aMSL proved to be the best support tool for management decisions reducing the ratio of inappropriate excision. The proposed iDCNN_aMSL model can represent a valid support for dermatologists in discriminating AN from EM with high accuracy and for medical decision making by reducing their rates of inappropriate excisions.

Artificial intelligence-based image classification methods for diagnosis of skin cancer: Challenges and opportunities

Recently, there has been great interest in developing Artificial Intelligence (AI) enabled computer-aided diagnostics solutions for the diagnosis of skin cancer. With the increasing incidence of skin cancers, low awareness among a growing population, and a lack of adequate clinical expertise and services, there is an immediate need for AI systems to assist clinicians in this domain. A large number of skin lesion datasets are available publicly, and researchers have developed AI solutions, particularly deep learning algorithms, to distinguish malignant skin lesions from benign lesions in different image modalities such as dermoscopic, clinical, and histopathology images. Despite the various claims of AI systems achieving higher accuracy than dermatologists in the classification of different skin lesions, these AI systems are still in the very early stages of clinical application in terms of being ready to aid clinicians in the diagnosis of skin cancers. In this review, we discuss advancements in the digital image-based AI solutions for the diagnosis of skin cancer, along with some challenges and future opportunities to improve these AI systems to support dermatologists and enhance their ability to diagnose skin cancer.