A deep learning system for differential diagnosis of skin diseases

Advancing diagnostic performance and clinical usability of neural networks via adversarial training and dual batch normalization

Unmasking the decision making process of machine learning models is essential for implementing diagnostic support systems in clinical practice. Here, we demonstrate that adversarially trained models can significantly enhance the usability of pathology detection as compared to their standard counterparts. We let six experienced radiologists rate the interpretability of saliency maps in datasets of X-rays, computed tomography, and magnetic resonance imaging scans. Significant improvements are found for our adversarial models, which are further improved by the application of dual-batch normalization. Contrary to previous research on adversarially trained models, we find that accuracy of such models is equal to standard models, when sufficiently large datasets and dual batch norm training are used. To ensure transferability, we additionally validate our results on an external test set of 22,433 X-rays. These findings elucidate that different paths for adversarial and real images are needed during training to achieve state of the art results with superior clinical interpretability.

Digital health, big data and smart technologies for the care of patients with systemic autoimmune diseases: Where do we stand?

The past decade has seen tremendous development in digital health, including in innovative new technologies such as Electronic Health Records, telemedicine, virtual visits, wearable technology and sophisticated analytical tools such as artificial intelligence (AI) and machine learning for the deep-integration of big data. In the field of rare connective tissue diseases (rCTDs), these opportunities include increased access to scarce and remote expertise, improved patient monitoring, increased participation and therapeutic adherence, better patient outcomes and patient empowerment. In this review, we discuss opportunities and key-barriers to improve application of digital health technologies in the field of autoimmune diseases. We also describe what could be the fully digital pathway of rCTD patients. Smart technologies can be used to provide real-world evidence about the natural history of rCTDs, to determine real-life drug utilization, advanced efficacy and safety data for rare diseases and highlight significant unmet needs. Yet, digitalization remains one of the most challenging issues faced by rCTD patients, their physicians and healthcare systems. Digital health technologies offer enormous potential to improve autoimmune rCTD care but this potential has so far been largely unrealized due to those significant obstacles. The need for robust assessments of the efficacy, affordability and scalability of AI in the context of digital health is crucial to improve the care of patients with rare autoimmune diseases.

Model learning analysis of 3D optoacoustic mesoscopy images for the classification of atopic dermatitis

Atopic dermatitis (AD) is a skin inflammatory disease affecting 10% of the population worldwide. Raster-scanning optoacoustic mesoscopy (RSOM) has recently shown promise in dermatological imaging. We conducted a comprehensive analysis using three machine-learning models, random forest (RF), support vector machine (SVM), and convolutional neural network (CNN) for classifying healthy versus AD conditions, and sub-classifying different AD severities using RSOM images and clinical information. CNN model successfully differentiates healthy from AD patients with 97% accuracy. With limited data, RF achieved 65% accuracy in sub-classifying AD patients into mild versus moderate-severe cases. Identification of disease severities is vital in managing AD treatment.

AI-based pathology predicts origins for cancers of unknown primary

Cancer of unknown primary (CUP) origin is an enigmatic group of diagnoses in which the primary anatomical site of tumour origin cannot be determined^1,2. This poses a considerable challenge, as modern therapeutics are predominantly specific to the primary tumour³. Recent research has focused on using genomics and transcriptomics to identify the origin of a tumour^4-9. However, genomic testing is not always performed and lacks clinical penetration in low-resource settings. Here, to overcome these challenges, we present a deep-learning-based algorithm-Tumour Origin Assessment via Deep Learning (TOAD)-that can provide a differential diagnosis for the origin of the primary tumour using routinely acquired histology slides. We used whole-slide images of tumours with known primary origins to train a model that simultaneously identifies the tumour as primary or metastatic and predicts its site of origin. On our held-out test set of tumours with known primary origins, the model achieved a top-1 accuracy of 0.83 and a top-3 accuracy of 0.96, whereas on our external test set it achieved top-1 and top-3 accuracies of 0.80 and 0.93, respectively. We further curated a dataset of 317 cases of CUP for which a differential diagnosis was assigned. Our model predictions resulted in concordance for 61% of cases and a top-3 agreement of 82%. TOAD can be used as an assistive tool to assign a differential diagnosis to complicated cases of metastatic tumours and CUPs and could be used in conjunction with or in lieu of ancillary tests and extensive diagnostic work-ups to reduce the occurrence of CUP.

Quantitative characterization of human breast tissue based on deep learning segmentation of 3D optical coherence tomography images

In this study, we performed dual-modality optical coherence tomography (OCT) characterization (volumetric OCT imaging and quantitative optical coherence elastography) on human breast tissue specimens. We trained and validated a U-Net for automatic image segmentation. Our results demonstrated that U-Net segmentation can be used to assist clinical diagnosis for breast cancer, and is a powerful enabling tool to advance our understanding of the characteristics for breast tissue. Based on the results obtained from U-Net segmentation of 3D OCT images, we demonstrated significant morphological heterogeneity in small breast specimens acquired through diagnostic biopsy. We also found that breast specimens affected by different pathologies had different structural characteristics. By correlating U-Net analysis of structural OCT images with mechanical measurement provided by quantitative optical coherence elastography, we showed that the change of mechanical properties in breast tissue is not directly due to the change in the amount of dense or porous tissue.

Automated Radiographic Evaluation of Adenoid Hypertrophy Based on VGG-Lite

Adenoid hypertrophy is a pathological hyperplasia of the adenoids, which may cause snoring and apnea, as well as impede breathing during sleep. The lateral cephalogram is commonly used by dentists to screen for adenoid hypertrophy, but it is tedious and time-consuming to measure the ratio of adenoid width to nasopharyngeal width for adenoid assessment. The purpose of this study was to develop a screening tool to automatically evaluate adenoid hypertrophy from lateral cephalograms using deep learning. We proposed the deep learning model VGG-Lite, using the largest data set (1,023 X-ray images) yet described to support the automatic detection of adenoid hypertrophy. We demonstrated that our model was able to automatically evaluate adenoid hypertrophy with a sensitivity of 0.898, a specificity of 0.882, positive predictive value of 0.880, negative predictive value of 0.900, and F1 score of 0.889. The comparison of model-only and expert-only detection performance showed that the fully automatic method (0.07 min) was about 522 times faster than the human expert (36.6 min). Comparison of human experts with or without deep learning assistance showed that model-assisted human experts spent an average of 23.3 min to evaluate adenoid hypertrophy using 100 radiographs, compared to an average of 36.6 min using an entirely manual procedure. We therefore concluded that deep learning could improve the accuracy, speed, and efficiency of evaluating adenoid hypertrophy from lateral cephalograms.

A Deep Learning Based Framework for Diagnosing Multiple Skin Diseases in a Clinical Environment

Background: Numerous studies have attempted to apply artificial intelligence (AI) in the dermatological field, mainly on the classification and segmentation of various dermatoses. However, researches under real clinical settings are scarce. Objectives: This study was aimed to construct a novel framework based on deep learning trained by a dataset that represented the real clinical environment in a tertiary class hospital in China, for better adaptation of the AI application in clinical practice among Asian patients. Methods: Our dataset was composed of 13,603 dermatologist-labeled dermoscopic images, containing 14 categories of diseases, namely lichen planus (LP), rosacea (Rosa), viral warts (VW), acne vulgaris (AV), keloid and hypertrophic scar (KAHS), eczema and dermatitis (EAD), dermatofibroma (DF), seborrheic dermatitis (SD), seborrheic keratosis (SK), melanocytic nevus (MN), hemangioma (Hem), psoriasis (Pso), port wine stain (PWS), and basal cell carcinoma (BCC). In this study, we applied Google's EfficientNet-b4 with pre-trained weights on ImageNet as the backbone of our CNN architecture. The final fully-connected classification layer was replaced with 14 output neurons. We added seven auxiliary classifiers to each of the intermediate layer groups. The modified model was retrained with our dataset and implemented using Pytorch. We constructed saliency maps to visualize our network's attention area of input images for its prediction. To explore the visual characteristics of different clinical classes, we also examined the internal image features learned by the proposed framework using t-SNE (t-distributed Stochastic Neighbor Embedding). Results: Test results showed that the proposed framework achieved a high level of classification performance with an overall accuracy of 0.948, a sensitivity of 0.934 and a specificity of 0.950. We also compared the performance of our algorithm with three most widely used CNN models which showed our model outperformed existing models with the highest area under curve (AUC) of 0.985. We further compared this model with 280 board-certificated dermatologists, and results showed a comparable performance level in an 8-class diagnostic task. Conclusions: The proposed framework retrained by the dataset that represented the real clinical environment in our department could accurately classify most common dermatoses that we encountered during outpatient practice including infectious and inflammatory dermatoses, benign and malignant cutaneous tumors.

Using deep learning to predict microvascular invasion in hepatocellular carcinoma based on dynamic contrast-enhanced MRI combined with clinical parameters

PURPOSE

Microvascular invasion (MVI) is a critical determinant of the early recurrence and poor prognosis of patients with hepatocellular carcinoma (HCC). Prediction of MVI status is clinically significant for the decision of treatment strategies and the assessment of patient's prognosis. A deep learning (DL) model was developed to predict the MVI status and grade in HCC patients based on preoperative dynamic contrast-enhanced magnetic resonance imaging (DCE-MRI) and clinical parameters.

METHODS

HCC patients with pathologically confirmed MVI status from January to December 2016 were enrolled and preoperative DCE-MRI of these patients were collected in this study. Then they were randomly divided into the training and testing cohorts. A DL model with eight conventional neural network (CNN) branches for eight MRI sequences was built to predict the presence of MVI, and further combined with clinical parameters for better prediction.

RESULTS

Among 601 HCC patients, 376 patients were pathologically MVI absent, and 225 patients were MVI present. To predict the presence of MVI, the DL model based only on images achieved an area under curve (AUC) of 0.915 in the testing cohort as compared to the radiomics model with an AUC of 0.731. The DL combined with clinical parameters (DLC) model yielded the best predictive performance with an AUC of 0.931. For the MVI-grade stratification, the DLC models achieved an overall accuracy of 0.793. Survival analysis demonstrated that the patients with DLC-predicted MVI status were associated with the poor overall survival (OS) and recurrence-free survival (RFS). Further investigation showed that hepatectomy with the wide resection margin contributes to better OS and RFS in the DLC-predicted MVI present patients.

CONCLUSION

The proposed DLC model can provide a non-invasive approach to evaluate MVI before surgery, which can help surgeons make decisions of surgical strategies and assess patient's prognosis.

Development and Assessment of an Artificial Intelligence-Based Tool for Skin Condition Diagnosis by Primary Care Physicians and Nurse Practitioners in Teledermatology Practices

Importance

Most dermatologic cases are initially evaluated by nondermatologists such as primary care physicians (PCPs) or nurse practitioners (NPs).

Objective

To evaluate an artificial intelligence (AI)-based tool that assists with diagnoses of dermatologic conditions.

Design, Setting, and Participants

This multiple-reader, multiple-case diagnostic study developed an AI-based tool and evaluated its utility. Primary care physicians and NPs retrospectively reviewed an enriched set of cases representing 120 different skin conditions. Randomization was used to ensure each clinician reviewed each case either with or without AI assistance; each clinician alternated between batches of 50 cases in each modality. The reviews occurred from February 21 to April 28, 2020. Data were analyzed from May 26, 2020, to January 27, 2021.

Exposures

An AI-based assistive tool for interpreting clinical images and associated medical history.

Main Outcomes and Measures

The primary analysis evaluated agreement with reference diagnoses provided by a panel of 3 dermatologists for PCPs and NPs. Secondary analyses included diagnostic accuracy for biopsy-confirmed cases, biopsy and referral rates, review time, and diagnostic confidence.

Results

Forty board-certified clinicians, including 20 PCPs (14 women [70.0%]; mean experience, 11.3 [range, 2-32] years) and 20 NPs (18 women [90.0%]; mean experience, 13.1 [range, 2-34] years) reviewed 1048 retrospective cases (672 female [64.2%]; median age, 43 [interquartile range, 30-56] years; 41 920 total reviews) from a teledermatology practice serving 11 sites and provided 0 to 5 differential diagnoses per case (mean [SD], 1.6 [0.7]). The PCPs were located across 12 states, and the NPs practiced in primary care without physician supervision across 9 states. The NPs had a mean of 13.1 (range, 2-34) years of experience and practiced in primary care without physician supervision across 9 states. Artificial intelligence assistance was significantly associated with higher agreement with reference diagnoses. For PCPs, the increase in diagnostic agreement was 10% (95% CI, 8%-11%; P < .001), from 48% to 58%; for NPs, the increase was 12% (95% CI, 10%-14%; P < .001), from 46% to 58%. In secondary analyses, agreement with biopsy-obtained diagnosis categories of maglignant, precancerous, or benign increased by 3% (95% CI, -1% to 7%) for PCPs and by 8% (95% CI, 3%-13%) for NPs. Rates of desire for biopsies decreased by 1% (95% CI, 0-3%) for PCPs and 2% (95% CI, 1%-3%) for NPs; the rate of desire for referrals decreased by 3% (95% CI, 1%-4%) for PCPs and NPs. Diagnostic agreement on cases not indicated for a dermatologist referral increased by 10% (95% CI, 8%-12%) for PCPs and 12% (95% CI, 10%-14%) for NPs, and median review time increased slightly by 5 (95% CI, 0-8) seconds for PCPs and 7 (95% CI, 5-10) seconds for NPs per case.

Conclusions and Relevance

Artificial intelligence assistance was associated with improved diagnoses by PCPs and NPs for 1 in every 8 to 10 cases, indicating potential for improving the quality of dermatologic care.

Novel Transfer Learning Approach for Medical Imaging with Limited Labeled Data

Deep learning requires a large amount of data to perform well. However, the field of medical image analysis suffers from a lack of sufficient data for training deep learning models. Moreover, medical images require manual labeling, usually provided by human annotators coming from various backgrounds. More importantly, the annotation process is time-consuming, expensive, and prone to errors. Transfer learning was introduced to reduce the need for the annotation process by transferring the deep learning models with knowledge from a previous task and then by fine-tuning them on a relatively small dataset of the current task. Most of the methods of medical image classification employ transfer learning from pretrained models, e.g., ImageNet, which has been proven to be ineffective. This is due to the mismatch in learned features between the natural image, e.g., ImageNet, and medical images. Additionally, it results in the utilization of deeply elaborated models. In this paper, we propose a novel transfer learning approach to overcome the previous drawbacks by means of training the deep learning model on large unlabeled medical image datasets and by next transferring the knowledge to train the deep learning model on the small amount of labeled medical images. Additionally, we propose a new deep convolutional neural network (DCNN) model that combines recent advancements in the field. We conducted several experiments on two challenging medical imaging scenarios dealing with skin and breast cancer classification tasks. According to the reported results, it has been empirically proven that the proposed approach can significantly improve the performance of both classification scenarios. In terms of skin cancer, the proposed model achieved an F1-score value of 89.09% when trained from scratch and 98.53% with the proposed approach. Secondly, it achieved an accuracy value of 85.29% and 97.51%, respectively, when trained from scratch and using the proposed approach in the case of the breast cancer scenario. Finally, we concluded that our method can possibly be applied to many medical imaging problems in which a substantial amount of unlabeled image data is available and the labeled image data is limited. Moreover, it can be utilized to improve the performance of medical imaging tasks in the same domain. To do so, we used the pretrained skin cancer model to train on feet skin to classify them into two classes-either normal or abnormal (diabetic foot ulcer (DFU)). It achieved an F1-score value of 86.0% when trained from scratch, 96.25% using transfer learning, and 99.25% using double-transfer learning.

Raising the Bar for Randomized Trials Involving Artificial Intelligence: The SPIRIT-Artificial Intelligence and CONSORT-Artificial Intelligence Guidelines

Artificial intelligence (AI)-based applications have the potential to improve the quality and efficiency of patient care in dermatology. Unique challenges in the development and validation of these technologies may limit their generalizability and real-world applicability. Before the widespread adoption of AI interventions, randomized trials should be conducted to evaluate their efficacy, safety, and cost effectiveness in clinical settings. The recent Standard Protocol Items: Recommendations for Interventional Trials-AI extension and Consolidated Standards of Reporting Trials-AI extension guidelines provide recommendations for reporting the methods and results of trials involving AI interventions. High-quality trials will provide gold standard evidence to support the adoption of AI for the benefit of patient care.

How to evaluate deep learning for cancer diagnostics - factors and recommendations

The large volume of data used in cancer diagnosis presents a unique opportunity for deep learning algorithms, which improve in predictive performance with increasing data. When applying deep learning to cancer diagnosis, the goal is often to learn how to classify an input sample (such as images or biomarkers) into predefined categories (such as benign or cancerous). In this article, we examine examples of how deep learning algorithms have been implemented to make predictions related to cancer diagnosis using clinical, radiological, and pathological image data. We present a systematic approach for evaluating the development and application of clinical deep learning algorithms. Based on these examples and the current state of deep learning in medicine, we discuss the future possibilities in this space and outline a roadmap for implementations of deep learning in cancer diagnosis.

Deep Neural Networks Offer Morphologic Classification and Diagnosis of Bacterial Vaginosis

Bacterial vaginosis (BV) is caused by the excessive and imbalanced growth of bacteria in vagina, affecting 30 to 50% of women. Gram staining followed by Nugent scoring based on bacterial morphotypes under the microscope is considered the gold standard for BV diagnosis; this method is often labor-intensive and time-consuming, and results vary from person to person. We developed and optimized a convolutional neural network (CNN) model and evaluated its ability to automatically identify and classify three categories of Nugent scores from microscope images. The CNN model was first established with a panel of microscopic images with Nugent scores determined by experts. The model was trained by minimizing the cross-entropy loss function and optimized by using a momentum optimizer. The separate test sets of images collected from three hospitals were evaluated by the CNN model. The CNN model consisted of 25 convolutional layers, 2 pooling layers, and a fully connected layer. The model obtained 82.4% sensitivity and 96.6% specificity with the 5,815 validation images when altered vaginal flora and BV were considered the positive samples, which was better than the rates achieved by top-level technologists and obstetricians in China. The capability of our model for generalization was so strong that it exhibited 75.1% accuracy in three categories of Nugent scores on the independent test set of 1,082 images, which was 6.6% higher than the average of three technologists, who are hold bachelor's degrees in medicine and are qualified to make diagnostic decisions. When three technologists ran one specimen in triplicate, the precision of three categories of Nugent scores was 54.0%. One hundred three samples diagnosed by two technologists on different days showed a repeatability of 90.3%. The CNN model outperformed human health care practitioners in terms of accuracy and stability for three categories of Nugent score diagnosis. The deep learning model may offer translational applications in automating diagnosis of bacterial vaginosis with proper supporting hardware.

Stress testing reveals gaps in clinic readiness of image-based diagnostic artificial intelligence models

Artificial intelligence models match or exceed dermatologists in melanoma image classification. Less is known about their robustness against real-world variations, and clinicians may incorrectly assume that a model with an acceptable area under the receiver operating characteristic curve or related performance metric is ready for clinical use. Here, we systematically assessed the performance of dermatologist-level convolutional neural networks (CNNs) on real-world non-curated images by applying computational "stress tests". Our goal was to create a proxy environment in which to comprehensively test the generalizability of off-the-shelf CNNs developed without training or evaluation protocols specific to individual clinics. We found inconsistent predictions on images captured repeatedly in the same setting or subjected to simple transformations (e.g., rotation). Such transformations resulted in false positive or negative predictions for 6.5-22% of skin lesions across test datasets. Our findings indicate that models meeting conventionally reported metrics need further validation with computational stress tests to assess clinic readiness.

Deep learning-enabled medical computer vision

A decade of unprecedented progress in artificial intelligence (AI) has demonstrated the potential for many fields-including medicine-to benefit from the insights that AI techniques can extract from data. Here we survey recent progress in the development of modern computer vision techniques-powered by deep learning-for medical applications, focusing on medical imaging, medical video, and clinical deployment. We start by briefly summarizing a decade of progress in convolutional neural networks, including the vision tasks they enable, in the context of healthcare. Next, we discuss several example medical imaging applications that stand to benefit-including cardiology, pathology, dermatology, ophthalmology-and propose new avenues for continued work. We then expand into general medical video, highlighting ways in which clinical workflows can integrate computer vision to enhance care. Finally, we discuss the challenges and hurdles required for real-world clinical deployment of these technologies.

Computer vision applied to dual-energy computed tomography images for precise calcinosis cutis quantification in patients with systemic sclerosis

Background

Although treatments have been proposed for calcinosis cutis (CC) in patients with systemic sclerosis (SSc), a standardized and validated method for CC burden quantification is necessary to enable valid clinical trials. We tested the hypothesis that computer vision applied to dual-energy computed tomography (DECT) finger images is a useful approach for precise and accurate CC quantification in SSc patients.

Methods

De-identified 2-dimensional (2D) DECT images from SSc patients with clinically evident lesser finger CC lesions were obtained. An expert musculoskeletal radiologist confirmed accurate manual segmentation (subtraction) of the phalanges for each image as a gold standard, and a U-Net Convolutional Neural Network (CNN) computer vision model for segmentation of healthy phalanges was developed and tested. A validation study was performed in an independent dataset whereby two independent radiologists manually measured the longest length and perpendicular short axis of each lesion and then calculated an estimated area by assuming the lesion was elliptical using the formula long axis/2 × short axis/2 × π, and a computer scientist used a region growing technique to calculate the area of CC lesions. Spearman’s correlation coefficient, Lin’s concordance correlation coefficient with 95% confidence intervals (CI), and a Bland-Altman plot (Stata V 15.1, College Station, TX) were used to test for equivalence between the radiologists’ and the CNN algorithm-generated area estimates.

Results

Forty de-identified 2D DECT images from SSc patients with clinically evident finger CC lesions were obtained and divided into training (N = 30 with image rotation × 3 to expand the set to N = 120) and test sets (N = 10). In the training set, five hundred epochs (iterations) were required to train the CNN algorithm to segment phalanges from adjacent CC, and accurate segmentation was evaluated using the ten held-out images. To test model performance, CC lesional area estimates calculated by two independent radiologists and a computer scientist were compared (radiologist 1 vs. radiologist 2 and radiologist 1 vs. computer vision approach) using an independent test dataset comprised of 31 images (8 index finger and 23 other fingers). For the two radiologists’, and the radiologist vs. computer vision measurements, Spearman’s rho was 0.91 and 0.94, respectively, both p < 0.0001; Lin’s concordance correlation coefficient was 0.91 (95% CI 0.85–0.98, p < 0.001) and 0.95 (95% CI 0.91–0.99, p < 0.001); and Bland-Altman plots demonstrated a mean difference between radiologist vs. radiologist, and radiologist vs. computer vision area estimates of − 0.5 mm² (95% limits of agreement − 10.0–9.0 mm²) and 1.7 mm² (95% limits of agreement − 6.0–9.5 mm², respectively.

Conclusions

We demonstrate that CNN quantification has a high degree of correlation with expert radiologist measurement of finger CC area measurements. Future work will include segmentation of 3-dimensional (3D) images for volumetric and density quantification, as well as validation in larger, independent cohorts.

Touchable cell biophysics property recognition platforms enable multifunctional blood smart health care

As a crucial biophysical property, red blood cell (RBC) deformability is pathologically altered in numerous disease states, and biochemical and structural changes occur over time in stored samples of otherwise normal RBCs. However, there is still a gap in applying it further to point-of-care blood devices due to the large external equipment (high-resolution microscope and microfluidic pump), associated operational difficulties, and professional analysis. Herein, we revolutionarily propose a smart optofluidic system to provide a differential diagnosis for blood testing via precise cell biophysics property recognition both mechanically and morphologically. Deformation of the RBC population is caused by pressing the hydrogel via an integrated mechanical transfer device. The biophysical properties of the cell population are obtained by the designed smartphone algorithm. Artificial intelligence-based modeling of cell biophysics properties related to blood diseases and quality was developed for online testing. We currently achieve 100% diagnostic accuracy for five typical clinical blood diseases (90 megaloblastic anemia, 78 myelofibrosis, 84 iron deficiency anemia, 48 thrombotic thrombocytopenic purpura, and 48 thalassemias) via real-world prospective implementation; furthermore, personalized blood quality (for transfusion in cardiac surgery) monitoring is achieved with an accuracy of 96.9%. This work suggests a potential basis for next-generation blood smart health care devices.

Performance of a deep neural network in teledermatology: a single-centre prospective diagnostic study

BACKGROUND

The use of artificial intelligence (AI) algorithms for the diagnosis of skin diseases has shown promise in experimental settings but has not been yet tested in real-life conditions.

OBJECTIVE

To assess the diagnostic performance and potential clinical utility of a 174-multiclass AI algorithm in a real-life telemedicine setting.

METHODS

Prospective, diagnostic accuracy study including consecutive patients who submitted images for teledermatology evaluation. The treating dermatologist chose a single image to upload to a web application during teleconsultation. A follow-up reader study including nine healthcare providers (3 dermatologists, 3 dermatology residents and 3 general practitioners) was performed.

RESULTS

A total of 340 cases from 281 patients met study inclusion criteria. The mean (SD) age of patients was 33.7 (17.5) years; 63% (n = 177) were female. Exposure to the AI algorithm results was considered useful in 11.8% of visits (n = 40) and the teledermatologist correctly modified the real-time diagnosis in 0.6% (n = 2) of cases. The overall top-1 accuracy of the algorithm (41.2%) was lower than that of the dermatologists (60.1%), residents (57.8%) and general practitioners (49.3%) (all comparisons P < 0.05, in the reader study). When the analysis was limited to the diagnoses on which the algorithm had been explicitly trained, the balanced top-1 accuracy of the algorithm (47.6%) was comparable to the dermatologists (49.7%) and residents (47.7%) but superior to the general practitioners (39.7%; P = 0.049). Algorithm performance was associated with patient skin type and image quality.

CONCLUSIONS

A 174-disease class AI algorithm appears to be a promising tool in the triage and evaluation of lesions with patient-taken photographs via telemedicine.

Deep learning based classification of facial dermatological disorders

Common properties of dermatological diseases are mostly lesions with abnormal pattern and skin color (usually redness). Therefore, dermatology is one of the most appropriate areas in medicine for automated diagnosis from images using pattern recognition techniques to provide accurate, objective, early diagnosis and interventions. Also, automated techniques provide diagnosis without depending on location and time. In addition, the number of patients in dermatology departments and costs of dermatologist visits can be reduced. Therefore, in this work, an automated method is proposed to classify dermatological diseases from color digital photographs. Efficiency of the proposed approach is provided by 2 stages. In the 1st stage, lesions are detected and extracted by using a variational level set technique after noise reduction and intensity normalization steps. In the 2nd stage, lesions are classified using a pre-trained DenseNet201 architecture with an efficient loss function. In this study, five common facial dermatological diseases are handled since they also cause anxiety, depression and even suicide death. The main contributions provided by this work can be identified as follows: (i) A comprehensive survey about the state-of-the-art works on classifications of dermatological diseases using deep learning; (ii) A new fully automated lesion detection and segmentation based on level sets; (iii) A new adaptive, hybrid and non-symmetric loss function; (iv) Using a pre-trained DenseNet201 structure with the new loss function to classify skin lesions; (v) Comparative evaluations of ten convolutional networks for skin lesion classification. Experimental results indicate that the proposed approach can classify lesions with high performance (95.24% accuracy).

Deep Learning for Diagnostic Binary Classification of Multiple-Lesion Skin Diseases

Background: Diagnosis of skin diseases is often challenging and computer-aided diagnostic tools are urgently needed to underpin decision making.

Objective: To develop a convolutional neural network model to classify clinically relevant selected multiple-lesion skin diseases, this in accordance to the STARD guidelines.

Methods: This was an image-based retrospective study using multi-task learning for binary classification. A VGG-16 model was trained on 16,543 non-standardized images. Image data was distributed in training set (80%), validation set (10%), and test set (10%). All images were collected from a clinical database of a Danish population attending one dermatological department. Included was patients categorized with ICD-10 codes related to acne, rosacea, psoriasis, eczema, and cutaneous t-cell lymphoma.

Results: Acne was distinguished from rosacea with a sensitivity of 85.42% CI 72.24–93.93% and a specificity of 89.53% CI 83.97–93.68%, cutaneous t-cell lymphoma was distinguished from eczema with a sensitivity of 74.29% CI 67.82–80.05% and a specificity of 84.09% CI 80.83–86.99%, and psoriasis from eczema with a sensitivity of 81.79% CI 78.51–84.76% and a specificity of 73.57% CI 69.76–77.13%. All results were based on the test set.

Conclusion: The performance rates reported were equal or superior to those reported for general practitioners with dermatological training, indicating that computer-aided diagnostic models based on convolutional neural network may potentially be employed for diagnosing multiple-lesion skin diseases.

[Artificial intelligence in image analysis-fundamentals and new developments]

BACKGROUND

Since 2017, there have been several reports of artificial intelligence (AI) achieving comparable performance to human experts on medical image analysis tasks. With the first ratification of a computer vision algorithm as a medical device in 2018, the way was paved for these methods to eventually become an integral part of modern clinical practice.

OBJECTIVES

The purpose of this article is to review the main developments that have occurred over the last few years in AI for image analysis, in relation to clinical applications and dermatology.

MATERIALS AND METHODS

Following the annual ImageNet challenge, we review classical methods of machine learning for image analysis and demonstrate how these methods incorporated human expertise but failed to meet industrial requirements regarding performance and scalability. With the rise of deep learning based on artificial neural networks, these limitations could be overcome. We discuss important aspects of this technology including transfer learning and report on recent developments such as explainable AI and generative models.

RESULTS

Deep learning models achieved performance on a par with human experts in a broad variety of diagnostic tasks and were shown to be suitable for industrialization. Therefore, current developments focus less on further improving accuracy but rather address open issues such as interpretability and applicability under clinical conditions. Upcoming generative models allow for entirely new applications.

CONCLUSIONS

Deep learning has a history of remarkable success and has become the new technical standard for image analysis. The dramatic improvement these models brought over classical approaches enables applications in a rapidly increasing number of clinical fields. In dermatology, as in many other domains, artificial intelligence still faces considerable challenges but is undoubtedly developing into an essential tool of modern medicine.

Diagnosing skin diseases using an AI-based dermatology consult

Deep learning outperforms general practitioners in diagnosing 26 common skin conditions.