Prospective validation of dermoscopy-based open-source artificial intelligence for melanoma diagnosis (PROVE-AI study)

Prospective multicenter study using artificial intelligence to improve dermoscopic melanoma diagnosis in patient care

BACKGROUND

Early detection of melanoma, a potentially lethal type of skin cancer with high prevalence worldwide, improves patient prognosis. In retrospective studies, artificial intelligence (AI) has proven to be helpful for enhancing melanoma detection. However, there are few prospective studies confirming these promising results. Existing studies are limited by low sample sizes, too homogenous datasets, or lack of inclusion of rare melanoma subtypes, preventing a fair and thorough evaluation of AI and its generalizability, a crucial aspect for its application in the clinical setting.

METHODS

Therefore, we assessed "All Data are Ext" (ADAE), an established open-source ensemble algorithm for detecting melanomas, by comparing its diagnostic accuracy to that of dermatologists on a prospectively collected, external, heterogeneous test set comprising eight distinct hospitals, four different camera setups, rare melanoma subtypes, and special anatomical sites. We advanced the algorithm with real test-time augmentation (R-TTA, i.e., providing real photographs of lesions taken from multiple angles and averaging the predictions), and evaluated its generalization capabilities.

RESULTS

Overall, the AI shows higher balanced accuracy than dermatologists (0.798, 95% confidence interval (CI) 0.779-0.814 vs. 0.781, 95% CI 0.760-0.802; p = 4.0e-145), obtaining a higher sensitivity (0.921, 95% CI 0.900-0.942 vs. 0.734, 95% CI 0.701-0.770; p = 3.3e-165) at the cost of a lower specificity (0.673, 95% CI 0.641-0.702 vs. 0.828, 95% CI 0.804-0.852; p = 3.3e-165).

CONCLUSION

As the algorithm exhibits a significant performance advantage on our heterogeneous dataset exclusively comprising melanoma-suspicious lesions, AI may offer the potential to support dermatologists, particularly in diagnosing challenging cases.

The SLICE-3D dataset: 400,000 skin lesion image crops extracted from 3D TBP for skin cancer detection

AI image classification algorithms have shown promising results when applied to skin cancer detection. Most public skin cancer image datasets are comprised of dermoscopic photos and are limited by selection bias, lack of standardization, and lend themselves to development of algorithms that can only be used by skilled clinicians. The SLICE-3D ("Skin Lesion Image Crops Extracted from 3D TBP") dataset described here addresses those concerns and contains images of over 400,000 distinct skin lesions from seven dermatologic centers from around the world. De-identified images were systematically extracted from sensitive 3D Total Body Photographs and are comparable in optical resolution to smartphone images. Algorithms trained on lower quality images could improve clinical workflows and detect skin cancers earlier if deployed in primary care or non-clinical settings, where photos are captured by non-expert physicians or patients. Such a tool could prompt individuals to visit a specialized dermatologist. This dataset circumvents many inherent limitations of prior datasets and may be used to build upon previous applications of skin imaging for cancer detection.

Knowledge-based, computerized, patient clinical decision support system for perioperative pain, nausea and constipation management: a clinical feasibility study

Opioid administration is particularly challenging in the perioperative period. Computerized-based Clinical Decision Support Systems (CDSS) are a promising innovation that might improve perioperative pain control. We report the development and feasibility validation of a knowledge-based CDSS aiming at optimizing the management of perioperative pain, postoperative nausea and vomiting (PONV), and laxative medications. This novel CDSS uses patient adaptive testing through a smartphone display, literature-based rules, and individual medical prescriptions to produce direct medical advice for the patient user. Our objective was to test the feasibility of the clinical use of our CDSS in the perioperative setting. This was a prospective single arm, single center, cohort study conducted in Strasbourg University Hospital. The primary outcome was the agreement between the recommendation provided by the experimental device and the recommendation provided by study personnel who interpreted the same care algorithm (control). Thirty-seven patients were included in the study of which 30 (81%) used the experimental device. Agreement between these two care recommendations (computer driven vs. clinician driven) was observed in 51 out 54 uses of the device (94.2% [95% CI 85.9-98.4%]). The agreement level had a probability of 86.6% to exceed the 90% clinically relevant agreement threshold. The knowledge-based, patient CDSS we developed was feasible at providing recommendations for the treatment of pain, PONV and constipation in a perioperative clinical setting.Trial registration number & date The study protocol was registered in ClinicalTrial.gov before enrollment began (NCT05707247 on January 26th, 2023).

A survey of skin tone assessment in prospective research

Increasing evidence supports reduced accuracy of noninvasive assessment tools, such as pulse oximetry, temperature probes, and AI skin diagnosis benchmarks, in patients with darker skin tones. The FDA is exploring potential strategies for device regulation to improve performance across diverse skin tones by including skin tone criteria. However, there is no consensus about how prospective studies should perform skin tone assessment in order to take this bias into account. There are several tools available to conduct skin tone assessments including administered visual scales (e.g., Fitzpatrick Skin Type, Pantone, Monk Skin Tone) and color measurement tools (e.g., reflectance colorimeters, reflectance spectrophotometers, cameras), although none are consistently used or validated across multiple medical domains. Accurate and consistent skin tone measurement depends on many factors including standardized environments, lighting, body parts assessed, patient conditions, and choice of skin tone assessment tool(s). As race and ethnicity are inadequate proxies for skin tone, these considerations can be helpful in standardizing the effect of skin tone on studies such as AI dermatology diagnoses, pulse oximetry, and temporal thermometers. Skin tone bias in medical devices is likely due to systemic factors that lead to inadequate validation across diverse skin tones. There is an opportunity for researchers to use skin tone assessment methods with standardized considerations in prospective studies of noninvasive tools that may be affected by skin tone. We propose considerations that researchers must take in order to improve device robustness to skin tone bias.

Revolutionizing teledermatology: Exploring the integration of artificial intelligence, including Generative Pre-trained Transformer chatbots for artificial intelligence-driven anamnesis, diagnosis, and treatment plans

The integration of teledermatology and artificial intelligence (AI) marks a significant advancement in dermatologic care. This study examines the synergistic interplay between these two domains, highlighting their collective impact on enhancing the accuracy, accessibility, and efficiency of teledermatologic services. Teledermatology expands dermatologic care to remote and underserved areas, and AI technologies show considerable potential in analyzing dermatologic images and performing various tasks involved in teledermatology consultations. Such integration facilitates rapid, precise diagnoses, personalized treatment plans, and data-driven insights. Our explorative study involved designing a GPT-based chatbot named "Dr. DermBot" and exploring its performance in a teledermatologic consultation process. The design phase focused on the chatbot's ability to conduct consultations autonomously. The subsequent testing phase assessed its performance against the backdrop of current teledermatologic practices, exploring the potential of AI and chatbots to simulate and potentially enhance teledermatologic health care. Our study demonstrates the promising future of combining teledermatology with AI. It also brings to light ethical and legal concerns, including the protection of patient data privacy and adherence to regulatory standards. The union of teledermatology and AI not only aims to enhance the precision of teledermatologic diagnoses but also broadens the accessibility of dermatologic services to previously underserved populations, benefiting patients, health care providers, and the overall health care system.

Artificial intelligence in dermatopathology: Updates, strengths, and challenges

Artificial intelligence (AI) has evolved to become a significant force in various domains, including medicine. We explore the role of AI in pathology, with a specific focus on dermatopathology and neoplastic dermatopathology. AI, encompassing machine learning and deep learning, has demonstrated its potential in tasks ranging from diagnostic applications on whole slide imaging to predictive and prognostic functions in skin pathology. In dermatopathology, studies have assessed AI's ability to identify skin lesions, classify melanomas, and improve diagnostic accuracy. Results indicate that AI, particularly convolutional neural networks, can outperform human pathologists in terms of sensitivity and specificity. AI aids in predicting disease outcomes, identifying aggressive tumors, and differentiating between various skin conditions. Neoplastic dermatopathology showcases AI's prowess in classifying melanocytic lesions, discriminating between melanomas and nevi, and aids dermatopathologists in making accurate diagnoses. Studies emphasize the reproducibility and diagnostic aid that AI provides, especially in challenging cases. In inflammatory and lymphoproliferative dermatopathology, limited research exists, but studies show attempts to use AI to differentiate conditions such as mycosis fungoides and eczema. Although some results are promising, further exploration is needed in these areas. We highlight the extraordinary interest AI has garnered in the scientific community and its potential to assist clinicians and pathologists. Despite the advancements, we have stressed the importance of collaboration between medical professionals, computer scientists, bioinformaticians, and engineers to harness AI's benefits and acknowledging its limitations and risks. The integration of AI into dermatopathology holds great promise, positioning it as a valuable tool rather than as a replacement for human expertise.

Evaluation of an artificial intelligence-based decision support for the detection of cutaneous melanoma in primary care: a prospective real-life clinical trial

BACKGROUND

Use of artificial intelligence (AI), or machine learning, to assess dermoscopic images of skin lesions to detect melanoma has, in several retrospective studies, shown high levels of diagnostic accuracy on par with - or even outperforming - experienced dermatologists. However, the enthusiasm around these algorithms has not yet been matched by prospective clinical trials performed in authentic clinical settings. In several European countries, including Sweden, the initial clinical assessment of suspected skin cancer is principally conducted in the primary healthcare setting by primary care physicians, with or without access to teledermoscopic support from dermatology clinics.

OBJECTIVES

To determine the diagnostic performance of an AI-based clinical decision support tool for cutaneous melanoma detection, operated by a smartphone application (app), when used prospectively by primary care physicians to assess skin lesions of concern due to some degree of melanoma suspicion.

METHODS

This prospective multicentre clinical trial was conducted at 36 primary care centres in Sweden. Physicians used the smartphone app on skin lesions of concern by photographing them dermoscopically, which resulted in a dichotomous decision support text regarding evidence for melanoma. Regardless of the app outcome, all lesions underwent standard diagnostic procedures (surgical excision or referral to a dermatologist). After investigations were complete, lesion diagnoses were collected from the patients' medical records and compared with the app's outcome and other lesion data.

RESULTS

In total, 253 lesions of concern in 228 patients were included, of which 21 proved to be melanomas, with 11 thin invasive melanomas and 10 melanomas in situ. The app's accuracy in identifying melanomas was reflected in an area under the receiver operating characteristic (AUROC) curve of 0.960 [95% confidence interval (CI) 0.928-0.980], corresponding to a maximum sensitivity and specificity of 95.2% and 84.5%, respectively. For invasive melanomas alone, the AUROC was 0.988 (95% CI 0.965-0.997), corresponding to a maximum sensitivity and specificity of 100% and 92.6%, respectively.

CONCLUSIONS

The clinical decision support tool evaluated in this investigation showed high diagnostic accuracy when used prospectively in primary care patients, which could add significant clinical value for primary care physicians assessing skin lesions for melanoma.

Development of Personalized Strategies for Precisely Battling Malignant Melanoma

Melanoma is the most severe and fatal form of skin cancer, resulting from multiple gene mutations with high intra-tumor and inter-tumor molecular heterogeneity. Treatment options for patients whose disease has progressed beyond the ability for surgical resection rely on currently accepted standard therapies, notably immune checkpoint inhibitors and targeted therapies. Acquired resistance to these therapies and treatment-associated toxicity necessitate exploring novel strategies, especially those that can be personalized for specific patients and/or populations. Here, we review the current landscape and progress of standard therapies and explore what personalized oncology techniques may entail in the scope of melanoma. Our purpose is to provide an up-to-date summary of the tools at our disposal that work to circumvent the common barriers faced when battling melanoma.

Performance of Commercial Dermatoscopic Systems That Incorporate Artificial Intelligence for the Identification of Melanoma in General Practice: A Systematic Review

BACKGROUND

Cutaneous melanoma remains an increasing global public health burden, particularly in fair-skinned populations. Advancing technologies, particularly artificial intelligence (AI), may provide an additional tool for clinicians to help detect malignancies with a more accurate success rate. This systematic review aimed to report the performance metrics of commercially available convolutional neural networks (CNNs) tasked with detecting MM.

METHODS

A systematic literature search was performed using CINAHL, Medline, Scopus, ScienceDirect and Web of Science databases.

RESULTS

A total of 16 articles reporting MM were included in this review. The combined number of melanomas detected was 1160, and non-melanoma lesions were 33,010. The performance of market-approved technology and clinician performance for classifying melanoma was highly heterogeneous, with sensitivity ranging from 16.4 to 100.0%, specificity between 40.0 and 98.3% and accuracy between 44.0 and 92.0%. Less heterogeneity was observed when clinicians worked in unison with AI, with sensitivity ranging between 83.3 and 100.0%, specificity between 83.7 and 87.3%, and accuracy between 86.4 and 86.9%.

CONCLUSION

Instead of focusing on the performance of AI versus clinicians for classifying melanoma, more consistent performance has been obtained when clinicians' work is supported by AI, facilitating management decisions and improving health outcomes.

Transparent medical image AI via an image-text foundation model grounded in medical literature

Building trustworthy and transparent image-based medical artificial intelligence (AI) systems requires the ability to interrogate data and models at all stages of the development pipeline, from training models to post-deployment monitoring. Ideally, the data and associated AI systems could be described using terms already familiar to physicians, but this requires medical datasets densely annotated with semantically meaningful concepts. In the present study, we present a foundation model approach, named MONET (medical concept retriever), which learns how to connect medical images with text and densely scores images on concept presence to enable important tasks in medical AI development and deployment such as data auditing, model auditing and model interpretation. Dermatology provides a demanding use case for the versatility of MONET, due to the heterogeneity in diseases, skin tones and imaging modalities. We trained MONET based on 105,550 dermatological images paired with natural language descriptions from a large collection of medical literature. MONET can accurately annotate concepts across dermatology images as verified by board-certified dermatologists, competitively with supervised models built on previously concept-annotated dermatology datasets of clinical images. We demonstrate how MONET enables AI transparency across the entire AI system development pipeline, from building inherently interpretable models to dataset and model auditing, including a case study dissecting the results of an AI clinical trial.

Deep learning-aided decision support for diagnosis of skin disease across skin tones

Although advances in deep learning systems for image-based medical diagnosis demonstrate their potential to augment clinical decision-making, the effectiveness of physician-machine partnerships remains an open question, in part because physicians and algorithms are both susceptible to systematic errors, especially for diagnosis of underrepresented populations. Here we present results from a large-scale digital experiment involving board-certified dermatologists (n = 389) and primary-care physicians (n = 459) from 39 countries to evaluate the accuracy of diagnoses submitted by physicians in a store-and-forward teledermatology simulation. In this experiment, physicians were presented with 364 images spanning 46 skin diseases and asked to submit up to four differential diagnoses. Specialists and generalists achieved diagnostic accuracies of 38% and 19%, respectively, but both specialists and generalists were four percentage points less accurate for the diagnosis of images of dark skin as compared to light skin. Fair deep learning system decision support improved the diagnostic accuracy of both specialists and generalists by more than 33%, but exacerbated the gap in the diagnostic accuracy of generalists across skin tones. These results demonstrate that well-designed physician-machine partnerships can enhance the diagnostic accuracy of physicians, illustrating that success in improving overall diagnostic accuracy does not necessarily address bias.

Principles, applications, and future of artificial intelligence in dermatology

This paper provides an overview of artificial-intelligence (AI), as applied to dermatology. We focus our discussion on methodology, AI applications for various skin diseases, limitations, and future opportunities. We review how the current image-based models are being implemented in dermatology across disease subsets, and highlight the challenges facing widespread adoption. Additionally, we discuss how the future of AI in dermatology might evolve and the emerging paradigm of large language, and multi-modal models to emphasize the importance of developing responsible, fair, and equitable models in dermatology.

SkinNet-INIO: Multiclass Skin Lesion Localization and Classification Using Fusion-Assisted Deep Neural Networks and Improved Nature-Inspired Optimization Algorithm

Background: Using artificial intelligence (AI) with the concept of a deep learning-based automated computer-aided diagnosis (CAD) system has shown improved performance for skin lesion classification. Although deep convolutional neural networks (DCNNs) have significantly improved many image classification tasks, it is still difficult to accurately classify skin lesions because of a lack of training data, inter-class similarity, intra-class variation, and the inability to concentrate on semantically significant lesion parts. Innovations: To address these issues, we proposed an automated deep learning and best feature selection framework for multiclass skin lesion classification in dermoscopy images. The proposed framework performs a preprocessing step at the initial step for contrast enhancement using a new technique that is based on dark channel haze and top-bottom filtering. Three pre-trained deep learning models are fine-tuned in the next step and trained using the transfer learning concept. In the fine-tuning process, we added and removed a few additional layers to lessen the parameters and later selected the hyperparameters using a genetic algorithm (GA) instead of manual assignment. The purpose of hyperparameter selection using GA is to improve the learning performance. After that, the deeper layer is selected for each network and deep features are extracted. The extracted deep features are fused using a novel serial correlation-based approach. This technique reduces the feature vector length to the serial-based approach, but there is little redundant information. We proposed an improved anti-Lion optimization algorithm for the best feature selection to address this issue. The selected features are finally classified using machine learning algorithms. Main Results: The experimental process was conducted using two publicly available datasets, ISIC2018 and ISIC2019. Employing these datasets, we obtained an accuracy of 96.1 and 99.9%, respectively. Comparison was also conducted with state-of-the-art techniques and shows the proposed framework improved accuracy. Conclusions: The proposed framework successfully enhances the contrast of the cancer region. Moreover, the selection of hyperparameters using the automated techniques improved the learning process of the proposed framework. The proposed fusion and improved version of the selection process maintains the best accuracy and shorten the computational time.