A deep learning system for differential diagnosis of skin diseases

Semisupervised Learning with Report-guided Pseudo Labels for Deep Learning-based Prostate Cancer Detection Using Biparametric MRI

Purpose

To evaluate a novel method of semisupervised learning (SSL) guided by automated sparse information from diagnostic reports to leverage additional data for deep learning-based malignancy detection in patients with clinically significant prostate cancer.

Materials and Methods

This retrospective study included 7756 prostate MRI examinations (6380 patients) performed between January 2014 and December 2020 for model development. An SSL method, report-guided SSL (RG-SSL), was developed for detection of clinically significant prostate cancer using biparametric MRI. RG-SSL, supervised learning (SL), and state-of-the-art SSL methods were trained using 100, 300, 1000, or 3050 manually annotated examinations. Performance on detection of clinically significant prostate cancer by RG-SSL, SL, and SSL was compared on 300 unseen examinations from an external center with a histopathologically confirmed reference standard. Performance was evaluated using receiver operating characteristic (ROC) and free-response ROC analysis. P values for performance differences were generated with a permutation test.

Results

At 100 manually annotated examinations, mean examination-based diagnostic area under the ROC curve (AUC) values for RG-SSL, SL, and the best SSL were 0.86 ± 0.01 (SD), 0.78 ± 0.03, and 0.81 ± 0.02, respectively. Lesion-based detection partial AUCs were 0.62 ± 0.02, 0.44 ± 0.04, and 0.48 ± 0.09, respectively. Examination-based performance of SL with 3050 examinations was matched by RG-SSL with 169 manually annotated examinations, thus requiring 14 times fewer annotations. Lesion-based performance was matched with 431 manually annotated examinations, requiring six times fewer annotations.

Conclusion

RG-SSL outperformed SSL in clinically significant prostate cancer detection and achieved performance similar to SL even at very low annotation budgets.Keywords: Annotation Efficiency, Computer-aided Detection and Diagnosis, MRI, Prostate Cancer, Semisupervised Deep Learning Supplemental material is available for this article. Published under a CC BY 4.0 license.

Explainability agreement between dermatologists and five visual explanations techniques in deep neural networks for melanoma AI classification

Introduction

The use of deep convolutional neural networks for analyzing skin lesion images has shown promising results. The identification of skin cancer by faster and less expensive means can lead to an early diagnosis, saving lives and avoiding treatment costs. However, to implement this technology in a clinical context, it is important for specialists to understand why a certain model makes a prediction; it must be explainable. Explainability techniques can be used to highlight the patterns of interest for a prediction.

Methods

Our goal was to test five different techniques: Grad-CAM, Grad-CAM++, Score-CAM, Eigen-CAM, and LIME, to analyze the agreement rate between features highlighted by the visual explanation maps to 3 important clinical criteria for melanoma classification: asymmetry, border irregularity, and color heterogeneity (ABC rule) in 100 melanoma images. Two dermatologists scored the visual maps and the clinical images using a semi-quantitative scale, and the results were compared. They also ranked their preferable techniques.

Results

We found that the techniques had different agreement rates and acceptance. In the overall analysis, Grad-CAM showed the best total+partial agreement rate (93.6%), followed by LIME (89.8%), Grad-CAM++ (88.0%), Eigen-CAM (86.4%), and Score-CAM (84.6%). Dermatologists ranked their favorite options: Grad-CAM and Grad-CAM++, followed by Score-CAM, LIME, and Eigen-CAM.

Discussion

Saliency maps are one of the few methods that can be used for visual explanations. The evaluation of explainability with humans is ideal to assess the understanding and applicability of these methods. Our results demonstrated that there is a significant agreement between clinical features used by dermatologists to diagnose melanomas and visual explanation techniques, especially Grad-Cam.

Deep Transfer Learning with Enhanced Feature Fusion for Detection of Abnormalities in X-ray Images

Medical image classification poses significant challenges in real-world scenarios. One major obstacle is the scarcity of labelled training data, which hampers the performance of image-classification algorithms and generalisation. Gathering sufficient labelled data is often difficult and time-consuming in the medical domain, but deep learning (DL) has shown remarkable performance, although it typically requires a large amount of labelled data to achieve optimal results. Transfer learning (TL) has played a pivotal role in reducing the time, cost, and need for a large number of labelled images. This paper presents a novel TL approach that aims to overcome the limitations and disadvantages of TL that are characteristic of an ImageNet dataset, which belongs to a different domain. Our proposed TL approach involves training DL models on numerous medical images that are similar to the target dataset. These models were then fine-tuned using a small set of annotated medical images to leverage the knowledge gained from the pre-training phase. We specifically focused on medical X-ray imaging scenarios that involve the humerus and wrist from the musculoskeletal radiographs (MURA) dataset. Both of these tasks face significant challenges regarding accurate classification. The models trained with the proposed TL were used to extract features and were subsequently fused to train several machine learning (ML) classifiers. We combined these diverse features to represent various relevant characteristics in a comprehensive way. Through extensive evaluation, our proposed TL and feature-fusion approach using ML classifiers achieved remarkable results. For the classification of the humerus, we achieved an accuracy of 87.85%, an F1-score of 87.63%, and a Cohen's Kappa coefficient of 75.69%. For wrist classification, our approach achieved an accuracy of 85.58%, an F1-score of 82.70%, and a Cohen's Kappa coefficient of 70.46%. The results demonstrated that the models trained using our proposed TL approach outperformed those trained with ImageNet TL. We employed visualisation techniques to further validate these findings, including a gradient-based class activation heat map (Grad-CAM) and locally interpretable model-independent explanations (LIME). These visualisation tools provided additional evidence to support the superior accuracy of models trained with our proposed TL approach compared to those trained with ImageNet TL. Furthermore, our proposed TL approach exhibited greater robustness in various experiments compared to ImageNet TL. Importantly, the proposed TL approach and the feature-fusion technique are not limited to specific tasks. They can be applied to various medical image applications, thus extending their utility and potential impact. To demonstrate the concept of reusability, a computed tomography (CT) case was adopted. The results obtained from the proposed method showed improvements.

Electric Double Layer Based Epidermal Electronics for Healthcare and Human-Machine Interface

Epidermal electronics, an emerging interdisciplinary field, is advancing the development of flexible devices that can seamlessly integrate with the skin. These devices, especially Electric Double Layer (EDL)-based sensors, overcome the limitations of conventional electronic devices, offering high sensitivity, rapid response, and excellent stability. Especially, Electric Double Layer (EDL)-based epidermal sensors show great potential in the application of wearable electronics to detect biological signals due to their high sensitivity, fast response, and excellent stability. The advantages can be attributed to the biocompatibility of the materials, the flexibility of the devices, and the large capacitance due to the EDL effect. Furthermore, we discuss the potential of EDL epidermal electronics as wearable sensors for health monitoring and wound healing. These devices can analyze various biofluids, offering real-time feedback on parameters like pH, temperature, glucose, lactate, and oxygen levels, which aids in accurate diagnosis and effective treatment. Beyond healthcare, we explore the role of EDL epidermal electronics in human-machine interaction, particularly their application in prosthetics and pressure-sensing robots. By mimicking the flexibility and sensitivity of human skin, these devices enhance the functionality and user experience of these systems. This review summarizes the latest advancements in EDL-based epidermal electronic devices, offering a perspective for future research in this rapidly evolving field.

Automatische Körperteil-Identifikation in dermatologischen klinischen Bildern durch maschinelles Lernen

Hintergrund

Dermatologische Erkrankungen sind in allen Bevölkerungsgruppen weit verbreitet. Das betroffene Körperteil ist für ihre Diagnose, Therapie und Forschung von Bedeutung. Die automatische Identifizierung der abgebildeten Körperteile in dermatologischen Krankheitsbildern könnte daher die klinische Versorgung verbessern, indem sie zusätzliche Informationen für klinische Entscheidungsalgorithmen liefert, schwer zu behandelnde Bereiche aufdeckt und die Forschung durch die Identifizierung neuer Krankheitsmuster unterstützt.

Patienten und Methodik

In dieser Studie wurden 6219 annotierte dermatologische Bilder aus unserer klinischen Datenbank verwendet, womit ein neuronales Netz trainiert und validiert wurde. Als Anwendung wurden mit diesem System qualitative Heatmaps für die Verteilung von Körperteilen bei häufigen dermatologischen Erkrankungen erstellt.

Ergebnisse

Der Algorithmus erreichte eine mittlere balancierte Genauigkeit (Accuracy) von 89% (74,8%–96,5%). Die Fotos von nichtmelanozytärem Hautkrebs betrafen vor allem das Gesicht und den Oberkörper, während die größte Häufigkeit der Ekzem‐ und Psoriasis‐Bildverteilung den Oberkörper, die Beine und die Hände umfassten.

Schlussfolgerungen

Die Genauigkeit dieses Systems ist vergleichbar mit den besten bisher veröffentlichten Algorithmen für Bildklassifizierungsaufgaben, was darauf hindeutet, dass dieser Algorithmus die Diagnose, Therapie und Forschung bei dermatologischen Erkrankungen verbessern könnte.

Automatic body part identification in real-world clinical dermatological images using machine learning

BACKGROUND

Dermatological conditions are prevalent across all population sub-groups. The affected body part is of importance to their diagnosis, therapy, and research. The automatic identification of body parts in dermatological clinical pictures could therefore improve clinical care by providing additional information for clinical decision-making algorithms, discovering hard-to-treat areas, and research by identifying new patterns of disease.

PATIENTS AND METHODS

In this study, we used 6,219 labelled dermatological images from our clinical database, which were used to train and validate a convolutional neural network. As a use case, qualitative heatmaps for the body part distribution in common dermatological conditions was generated using this system.

RESULTS

The algorithm reached a mean balanced accuracy of 89% (range 74.8%-96.5%). Non-melanoma skin cancer photos were mostly of the face and torso, while hotspots of eczema and psoriasis image distribution included the torso, legs, and hands.

CONCLUSIONS

The accuracy of this system is comparable to the best to-date published algorithms for image classification challenges, suggesting this algorithm could boost diagnosis, therapy, and research of dermatological conditions.

Assessing the Performance of GPT-3.5 and GPT-4 on the 2023 Japanese Nursing Examination

Purpose The purpose of this study was to evaluate the changes in capabilities between the Generative Pre-trained Transformer (GPT)-3.5 and GPT-4 versions of the large-scale language model ChatGPT within a Japanese medical context. Methods The study involved ChatGPT versions 3.5 and 4 responding to questions from the 112th Japanese National Nursing Examination (JNNE). The study comprised three analyses: correct answer rate and score rate calculations, comparisons between GPT-3.5 and GPT-4, and comparisons of correct answer rates for conversation questions. Results ChatGPT versions 3.5 and 4 responded to 237 out of 238 Japanese questions from the 112th JNNE. While GPT-3.5 achieved an overall accuracy rate of 59.9%, failing to meet the passing standards in compulsory and general/scenario-based questions, scoring 58.0% and 58.3%, respectively, GPT-4 had an accuracy rate of 79.7%, satisfying the passing standards by scoring 90.0% and 77.7%, respectively. For each problem type, GPT-4 showed a higher accuracy rate than GPT-3.5. Specifically, the accuracy rates for compulsory questions improved from 58.0% with GPT-3.5 to 90.0% with GPT-4. For general questions, the rates went from 64.6% with GPT-3.5 to 75.6% with GPT-4. In scenario-based questions, the accuracy rates improved substantially from 51.7% with GPT-3.5 to 80.0% with GPT-4. For conversation questions, GPT-3.5 had an accuracy rate of 73.3% and GPT-4 had an accuracy rate of 93.3%. Conclusions The GPT-4 version of ChatGPT displayed performance sufficient to pass the JNNE, significantly improving from GPT-3.5. This suggests specialized medical training could make such models beneficial in Japanese clinical settings, aiding decision-making. However, user awareness and training are crucial, given potential inaccuracies in ChatGPT's responses. Hence, responsible usage with an understanding of its capabilities and limitations is vital to best support healthcare professionals and patients.

Deep Learning Based Classification of Dermatological Disorders

Automated medical diagnosis has become crucial and significantly supports medical doctors. Thus, there is a demand for inventing deep learning (DL) and convolutional networks for analyzing medical images. Dermatology, in particular, is one of the domains that was recently targeted by AI specialists to introduce new DL algorithms or enhance convolutional neural network (CNN) architectures. A significantly high proportion of studies in the field are concerned with skin cancer, whereas other dermatological disorders are still limited. In this work, we examined the performance of 6 CNN architectures named VGG16, EfficientNet, InceptionV3, MobileNet, NasNet, and ResNet50 for the top 3 dermatological disorders that frequently appear in the Middle East. An Image filtering and denoising were imposed in this work to enhance image quality and increase architecture performance. Experimental results revealed that MobileNet achieved the highest performance and accuracy among the CNN architectures and can classify disorder with high performance (95.7% accuracy). Future scope will focus more on proposing a new methodology for deep-based classification. In addition, we will expand the dataset for more images that consider new disorders and variations.

Detecting shortcut learning for fair medical AI using shortcut testing

Machine learning (ML) holds great promise for improving healthcare, but it is critical to ensure that its use will not propagate or amplify health disparities. An important step is to characterize the (un)fairness of ML models-their tendency to perform differently across subgroups of the population-and to understand its underlying mechanisms. One potential driver of algorithmic unfairness, shortcut learning, arises when ML models base predictions on improper correlations in the training data. Diagnosing this phenomenon is difficult as sensitive attributes may be causally linked with disease. Using multitask learning, we propose a method to directly test for the presence of shortcut learning in clinical ML systems and demonstrate its application to clinical tasks in radiology and dermatology. Finally, our approach reveals instances when shortcutting is not responsible for unfairness, highlighting the need for a holistic approach to fairness mitigation in medical AI.

Multi-center study on predicting breast cancer lymph node status from core needle biopsy specimens using multi-modal and multi-instance deep learning

The objective of our study is to develop a deep learning model based on clinicopathological data and digital pathological image of core needle biopsy specimens for predicting breast cancer lymph node metastasis. We collected 3701 patients from the Fourth Hospital of Hebei Medical University and 190 patients from four medical centers in Hebei Province. Integrating clinicopathological data and image features build multi-modal and multi-instance (MMMI) deep learning model to obtain the final prediction. For predicting with or without lymph node metastasis, the AUC was 0.770, 0.709, 0.809 based on the clinicopathological features, WSI and MMMI, respectively. For predicting four classification of lymph node status (no metastasis, isolated tumor cells (ITCs), micrometastasis, and macrometastasis), the prediction based on clinicopathological features, WSI and MMMI were compared. The AUC for no metastasis was 0.770, 0.709, 0.809, respectively; ITCs were 0.619, 0.531, 0.634, respectively; micrometastasis were 0.636, 0.617, 0.691, respectively; and macrometastasis were 0.748, 0.691, 0.758, respectively. The MMMI model achieved the highest prediction accuracy. For prediction of different molecular types of breast cancer, MMMI demonstrated a better prediction accuracy for any type of lymph node status, especially in the molecular type of triple negative breast cancer (TNBC). In the external validation sets, MMMI also showed better prediction accuracy in the four classification, with AUC of 0.725, 0.757, 0.525, and 0.708, respectively. Finally, we developed a breast cancer lymph node metastasis prediction model based on a MMMI model. Through all cases tests, the results showed that the overall prediction ability was high.

ComBat Harmonization: Empirical Bayes versus fully Bayes approaches

Studying small effects or subtle neuroanatomical variation requires large-scale sample size data. As a result, combining neuroimaging data from multiple datasets is necessary. Variation in acquisition protocols, magnetic field strength, scanner build, and many other non-biologically related factors can introduce undesirable bias into studies. Hence, harmonization is required to remove the bias-inducing factors from the data. ComBat is one of the most common methods applied to features from structural images. ComBat models the data using a hierarchical Bayesian model and uses the empirical Bayes approach to infer the distribution of the unknown factors. The empirical Bayes harmonization method is computationally efficient and provides valid point estimates. However, it tends to underestimate uncertainty. This paper investigates a new approach, fully Bayesian ComBat, where Monte Carlo sampling is used for statistical inference. When comparing fully Bayesian and empirical Bayesian ComBat, we found Empirical Bayesian ComBat more effectively removed scanner strength information and was much more computationally efficient. Conversely, fully Bayesian ComBat better preserved biological disease and age-related information while performing more accurate harmonization on traveling subjects. The fully Bayesian approach generates a rich posterior distribution, which is useful for generating simulated imaging features for improving classifier performance in a limited data setting. We show the generative capacity of our model for augmenting and improving the detection of patients with Alzheimer's disease. Posterior distributions for harmonized imaging measures can also be used for brain-wide uncertainty comparison and more principled downstream statistical analysis.Code for our new fully Bayesian ComBat extension is available at https://github.com/batmanlab/BayesComBat.

Smart e-Skin Cancer Care in Europe During and after the Covid-19 Pandemic: a Multidisciplinary Expert Consensus

INTRODUCTION

Melanoma is the deadliest of all the skin cancers and its incidence is increasing every year in Europe. Patients with melanoma often present late to the specialist and treatment is delayed for many reasons (delay in patient consultation, misdiagnosis by general practitioners, and/or limited access to dermatologists). Beyond this, there are significant inequalities in skin cancer between population groups within the same country and between countries across Europe. The emergence of the COVID-19 pandemic only aggravated these health deficiencies.

OBJECTIVES

The aim was to create an expert opinion about the challenges in skin cancer management in Europe during the post COVID-19 acute pandemic and to identify and discuss the implementation of new technologies (including e-health and artificial intelligence defined as "Smart Skin Cancer Care") to overcome them.

METHODS

For this purpose, an ad-hoc questionnaire with items addressing topics of skin cancer care was developed, answered independently and discussed by a multidisciplinary European panel of experts comprising dermatologists, dermato-oncologists, patient advocacy representatives, digital health technology experts, and health technology assessment experts.

RESULTS

After all panel of experts discussions, a multidisciplinary expert opinion was created.

CONCLUSIONS

As a conclusion, the access to dermatologists is difficult and will be aggravated in the near future. This fact, together with important differences in Skin Cancer Care in Europe, suggest the need of a new approach to skin health, prevention and disease management paradigm (focused on integration of new technologies) to minimize the impact of skin cancer and to ensure optimal quality and equity.

A moment kernel machine for clinical data mining to inform medical decision making

Machine learning-aided medical decision making presents three major challenges: achieving model parsimony, ensuring credible predictions, and providing real-time recommendations with high computational efficiency. In this paper, we formulate medical decision making as a classification problem and develop a moment kernel machine (MKM) to tackle these challenges. The main idea of our approach is to treat the clinical data of each patient as a probability distribution and leverage moment representations of these distributions to build the MKM, which transforms the high-dimensional clinical data to low-dimensional representations while retaining essential information. We then apply this machine to various pre-surgical clinical datasets to predict surgical outcomes and inform medical decision making, which requires significantly less computational power and time for classification while yielding favorable performance compared to existing methods. Moreover, we utilize synthetic datasets to demonstrate that the developed moment-based data mining framework is robust to noise and missing data, and achieves model parsimony giving an efficient way to generate satisfactory predictions to aid personalized medical decision making.

Annotation-Efficient Deep Learning Model for Pancreatic Cancer Diagnosis and Classification Using CT Images: A Retrospective Diagnostic Study

The aim of this study was to develop a novel deep learning (DL) model without requiring large-annotated training datasets for detecting pancreatic cancer (PC) using computed tomography (CT) images. This retrospective diagnostic study was conducted using CT images collected from 2004 and 2019 from 4287 patients diagnosed with PC. We proposed a self-supervised learning algorithm (pseudo-lesion segmentation (PS)) for PC classification, which was trained with and without PS and validated on randomly divided training and validation sets. We further performed cross-racial external validation using open-access CT images from 361 patients. For internal validation, the accuracy and sensitivity for PC classification were 94.3% (92.8-95.4%) and 92.5% (90.0-94.4%), and 95.7% (94.5-96.7%) and 99.3 (98.4-99.7%) for the convolutional neural network (CNN) and transformer-based DL models (both with PS), respectively. Implementing PS on a small-sized training dataset (randomly sampled 10%) increased accuracy by 20.5% and sensitivity by 37.0%. For external validation, the accuracy and sensitivity were 82.5% (78.3-86.1%) and 81.7% (77.3-85.4%) and 87.8% (84.0-90.8%) and 86.5% (82.3-89.8%) for the CNN and transformer-based DL models (both with PS), respectively. PS self-supervised learning can increase DL-based PC classification performance, reliability, and robustness of the model for unseen, and even small, datasets. The proposed DL model is potentially useful for PC diagnosis.

Robust and data-efficient generalization of self-supervised machine learning for diagnostic imaging

Machine-learning models for medical tasks can match or surpass the performance of clinical experts. However, in settings differing from those of the training dataset, the performance of a model can deteriorate substantially. Here we report a representation-learning strategy for machine-learning models applied to medical-imaging tasks that mitigates such 'out of distribution' performance problem and that improves model robustness and training efficiency. The strategy, which we named REMEDIS (for 'Robust and Efficient Medical Imaging with Self-supervision'), combines large-scale supervised transfer learning on natural images and intermediate contrastive self-supervised learning on medical images and requires minimal task-specific customization. We show the utility of REMEDIS in a range of diagnostic-imaging tasks covering six imaging domains and 15 test datasets, and by simulating three realistic out-of-distribution scenarios. REMEDIS improved in-distribution diagnostic accuracies up to 11.5% with respect to strong supervised baseline models, and in out-of-distribution settings required only 1-33% of the data for retraining to match the performance of supervised models retrained using all available data. REMEDIS may accelerate the development lifecycle of machine-learning models for medical imaging.

Non-Melanoma Skin Cancer Detection in the Age of Advanced Technology: A Review

Skin cancer is the most common cancer diagnosis in the United States, with approximately one in five Americans expected to be diagnosed within their lifetime. Non-melanoma skin cancer is the most prevalent type of skin cancer, and as cases rise globally, physicians need reliable tools for early detection. Artificial intelligence has gained substantial interest as a decision support tool in medicine, particularly in image analysis, where deep learning has proven to be an effective tool. Because specialties such as dermatology rely primarily on visual diagnoses, deep learning could have many diagnostic applications, including the diagnosis of skin cancer. Furthermore, with the advancement of mobile smartphones and their increasingly powerful cameras, deep learning technology could also be utilized in remote skin cancer screening applications. Ultimately, the available data for the detection and diagnosis of skin cancer using deep learning technology are promising, revealing sensitivity and specificity that are not inferior to those of trained dermatologists. Work is still needed to increase the clinical use of AI-based tools, but based on the current data and the attitudes of patients and physicians, deep learning technology could be used effectively as a clinical decision-making tool in collaboration with physicians to improve diagnostic efficiency and accuracy.

Use of Deep Neural Networks in the Detection and Automated Classification of Lesions Using Clinical Images in Ophthalmology, Dermatology, and Oral Medicine-A Systematic Review

Artificial neural networks (ANN) are artificial intelligence (AI) techniques used in the automated recognition and classification of pathological changes from clinical images in areas such as ophthalmology, dermatology, and oral medicine. The combination of enterprise imaging and AI is gaining notoriety for its potential benefits in healthcare areas such as cardiology, dermatology, ophthalmology, pathology, physiatry, radiation oncology, radiology, and endoscopic. The present study aimed to analyze, through a systematic literature review, the application of performance of ANN and deep learning in the recognition and automated classification of lesions from clinical images, when comparing to the human performance. The PRISMA 2020 approach (Preferred Reporting Items for Systematic Reviews and Meta-analyses) was used by searching four databases of studies that reference the use of IA to define the diagnosis of lesions in ophthalmology, dermatology, and oral medicine areas. A quantitative and qualitative analyses of the articles that met the inclusion criteria were performed. The search yielded the inclusion of 60 studies. It was found that the interest in the topic has increased, especially in the last 3 years. We observed that the performance of IA models is promising, with high accuracy, sensitivity, and specificity, most of them had outcomes equivalent to human comparators. The reproducibility of the performance of models in real-life practice has been reported as a critical point. Study designs and results have been progressively improved. IA resources have the potential to contribute to several areas of health. In the coming years, it is likely to be incorporated into everyday life, contributing to the precision and reducing the time required by the diagnostic process.

Artificial intelligence in mobile health for skin cancer diagnostics at home (AIM HIGH): a pilot feasibility study

Background

Artificial intelligence (AI)-based mobile phone apps (mHealth) have the potential to streamline care for suspicious skin lesions in primary care. This study aims to investigate the conditions and feasibility of a study that incorporates an AI-based app in primary care and evaluates its potential impact.

Methods

We conducted a pilot feasibility study from November 22nd, 2021 to June 9th, 2022 with a mixed-methods design on implementation of an AI-based mHealth app for skin cancer detection in three primary care practices in the Netherlands (Rotterdam, Leiden and Katwijk). The primary outcome was the inclusion and successful participation rate of patients and general practitioners (GPs). Secondary outcomes were the reasons, facilitators and barriers for successful participation and the potential impact in both pathways for future sample size calculations. Patients were offered use of an AI-based mHealth app before consulting their GP. GPs assessed the patients blinded and then unblinded to the app. Qualitative data included observations and audio-diaries from patients and GPs and focus-groups and interviews with GPs and GP assistants.

Findings

Fifty patients were included with a median age of 52 years (IQR 33.5-60.3), 64% were female, and 90% had a light skin type. The average patient inclusion rate was 4-6 per GP practice per month and 84% (n = 42) successfully participated. Similarly, in 90% (n = 45 patients) the GPs also successfully completed the study. GPs never changed their working diagnosis, but did change their treatment plan (n = 5) based on the app's assessments. Notably, 54% of patients with a benign skin lesion and low risk rating, indicated that they would be reassured and cancel their GP visit with these results (p < 0.001).

Interpretation

Our findings suggest that studying implementation of an AI-based mHealth app for detection of skin cancer in the hands of patients or as a diagnostic tool used by GPs in primary care appears feasible. Preliminary results indicate potential to further investigate both intended use settings.

Funding

SkinVision B.V.

Diagnosis of Parkinson's Disease via the Metabolic Fingerprint in Saliva by Deep Learning

Parkinson's disease (PD) is the second cause of the neurodegenerative disorder, affecting over 6 million people worldwide. The World Health Organization estimated that population aging will cause global PD prevalence to double in the coming 30 years. Optimal management of PD shall start at diagnosis and requires both a timely and accurate method. Conventional PD diagnosis needs observations and clinical signs assessment, which are time-consuming and low-throughput. A lack of body fluid diagnostic biomarkers for PD has been a significant challenge, although substantial progress has been made in genetic and imaging marker development. Herein, a platform that noninvasively collects saliva metabolic fingerprinting (SMF) by nanoparticle-enhanced laser desorption-ionization mass spectrometry with high-reproducibility and high-throughput, using ultra-small sample volume (down to 10 nL), is developed. Further, excellent diagnostic performance is achieved with an area-under-the-curve of 0.8496 (95% CI: 0.7393-0.8625) by constructing deep learning model from 312 participants. In conclusion, an alternative solution is provided for the molecular diagnostics of PD with SMF and metabolic biomarker screening for therapeutic intervention.

Dissection of medical AI reasoning processes via physician and generative-AI collaboration

Despite the proliferation and clinical deployment of artificial intelligence (AI)-based medical software devices, most remain black boxes that are uninterpretable to key stakeholders including patients, physicians, and even the developers of the devices. Here, we present a general model auditing framework that combines insights from medical experts with a highly expressive form of explainable AI that leverages generative models, to understand the reasoning processes of AI devices. We then apply this framework to generate the first thorough, medically interpretable picture of the reasoning processes of machine-learning-based medical image AI. In our synergistic framework, a generative model first renders "counterfactual" medical images, which in essence visually represent the reasoning process of a medical AI device, and then physicians translate these counterfactual images to medically meaningful features. As our use case, we audit five high-profile AI devices in dermatology, an area of particular interest since dermatology AI devices are beginning to achieve deployment globally. We reveal how dermatology AI devices rely both on features used by human dermatologists, such as lesional pigmentation patterns, as well as multiple, previously unreported, potentially undesirable features, such as background skin texture and image color balance. Our study also sets a precedent for the rigorous application of explainable AI to understand AI in any specialized domain and provides a means for practitioners, clinicians, and regulators to uncloak AI's powerful but previously enigmatic reasoning processes in a medically understandable way.

Artificial Intelligence: Exploring the Future of Innovation in Allergy Immunology

PURPOSE OF REVIEW

Artificial intelligence (AI) has increasingly been used in healthcare. Given the capacity of AI to handle large data and complex relationships between variables, AI is well suited for applications in healthcare. Recently, AI has been applied to allergy research.

RECENT FINDINGS

In this article, we review how AI technologies have been utilized in basic science and clinical allergy research for asthma, atopic dermatitis, rhinology, adverse reactions to drugs and vaccines, food allergy, anaphylaxis, urticaria, and eosinophilic gastrointestinal disorders. We discuss barriers for AI adoption to improve the care of patients with atopic diseases. These studies demonstrate the utility of applying AI to the field of allergy to help investigators expand their understanding of disease pathogenesis, improve diagnostic accuracy, enable prediction for treatments and outcomes, and for drug discovery.

Automated Facial Acne Lesion Detecting and Counting Algorithm for Acne Severity Evaluation and Its Utility in Assisting Dermatologists

BACKGROUND

Although lesion counting is an evaluation method that effectively analyzes facial acne severity, its usage is limited because of difficult implementation.

OBJECTIVES

We aimed to develop and validate an automated algorithm that detects and counts acne lesions by type, and to evaluate its clinical applicability as an assistance tool through a reader test.

METHODS

A total of 20,699 lesions (closed and open comedones, papules, nodules/cysts, and pustules) were manually labeled on 1213 facial images of 398 facial acne photography sets (frontal and both lateral views) acquired from 258 patients and used for training and validating algorithms based on a convolutional neural network for classifying five classes of acne lesions or for binary classification into noninflammatory and inflammatory lesions.

RESULTS

In the validation dataset, the highest mean average precision was 28.48 for the binary classification algorithm. Pearson's correlation of lesion counts between algorithm and ground-truth was 0.72 (noninflammatory) and 0.90 (inflammatory), respectively. In the reader test, eight readers (100.0%) detected and counted lesions more accurately using the algorithm compared with the reader-alone evaluation.

CONCLUSIONS

Overall, our algorithm demonstrated clinically applicable performance in detecting and counting facial acne lesions by type and its utility as an assistance tool for evaluating acne severity.

Stomach tissue classification using autofluorescence spectroscopy and machine learning

BACKGROUND AND OBJECTIVES

Determination of stomach tumor location and invasion depth requires delineation of gastric histological structure, which has hitherto been widely accomplished by histochemical staining. In recent years, alternative histochemical evaluation methods have been pursued to accelerate intraoperative diagnosis, often by bypassing the time-consuming step of dyeing. Owing to strong endogenous signals from coenzymes, metabolites, and proteins, autofluorescence spectroscopy is a favorable candidate technique to achieve this aim.

MATERIALS AND METHODS

We investigated stomach tissue slices and block specimens using a fast fluorescence imaging scanner. To obtain histological information from broad and structureless fluorescence spectra, we analyzed tens of thousands of spectra with multiple machine-learning algorithms and built a tissue classification model trained with dissected gastric tissues.

RESULTS

A machine-learning-based spectro-histological model was built based on the autofluorescence spectra measured from stomach tissue samples with delineated and validated histological structures. The scores from a principal components analysis were employed as input features, and prediction accuracy was confirmed to be 92.0%, 90.1%, and 91.4% for mucosa, submucosa, and muscularis propria, respectively. We investigated the tissue samples in both sliced and block forms using a fast fluorescence imaging scanner.

CONCLUSION

We successfully demonstrated differentiation of multiple tissue layers of well-defined specimens with the guidance of a histologist. Our spectro-histology classification model is applicable to histological prediction for both tissue blocks and slices, even though only sliced samples were trained.

Labeling confidence for uncertainty-aware histology image classification

Deep learning-based models applied to digital pathology require large, curated datasets with high-quality (HQ) annotations to perform correctly. In many cases, recruiting expert pathologists to annotate large databases is not feasible, and it is necessary to collect additional labeled data with varying label qualities, e.g., pathologists-in-training (henceforth, non-expert annotators). Learning from datasets with noisy labels is more challenging in medical applications since medical imaging datasets tend to have instance-dependent noise and suffer from high inter/intra-observer variability. In this paper, we design an uncertainty-driven labeling strategy with which we generate soft labels from 10 non-expert annotators for multi-class skin cancer classification. Based on this soft annotation, we propose an uncertainty estimation-based framework to handle these noisy labels. This framework is based on a novel formulation using a dual-branch min-max entropy calibration to penalize inexact labels during the training. Comprehensive experiments demonstrate the promising performance of our labeling strategy. Results show a consistent improvement by using soft labels with standard cross-entropy loss during training (∼4.0% F1-score) and increases when calibrating the model with the proposed min-max entropy calibration (∼6.6% F1-score). These improvements are produced at negligible cost, both in terms of annotation and calculation.

Melanoma Clinical Decision Support System: An Artificial Intelligence-Based Tool to Diagnose and Predict Disease Outcome in Early-Stage Melanoma Patients

This study set out to assess the performance of an artificial intelligence (AI) algorithm based on clinical data and dermatoscopic imaging for the early diagnosis of melanoma, and its capacity to define the metastatic progression of melanoma through serological and histopathological biomarkers, enabling dermatologists to make more informed decisions about patient management. Integrated analysis of demographic data, images of the skin lesions, and serum and histopathological markers were analyzed in a group of 196 patients with melanoma. The interleukins (ILs) IL-4, IL-6, IL-10, and IL-17A as well as IFNγ (interferon), GM-CSF (granulocyte and macrophage colony-stimulating factor), TGFβ (transforming growth factor), and the protein DCD (dermcidin) were quantified in the serum of melanoma patients at the time of diagnosis, and the expression of the RKIP, PIRIN, BCL2, BCL3, MITF, and ANXA5 proteins was detected by immunohistochemistry (IHC) in melanoma biopsies. An AI algorithm was used to improve the early diagnosis of melanoma and to predict the risk of metastasis and of disease-free survival. Two models were obtained to predict metastasis (including “all patients” or only patients “at early stages of melanoma”), and a series of attributes were seen to predict the progression of metastasis: Breslow thickness, infiltrating BCL-2 expressing lymphocytes, and IL-4 and IL-6 serum levels. Importantly, a decrease in serum GM-CSF seems to be a marker of poor prognosis in patients with early-stage melanomas.

Determining the clinical applicability of machine learning models through assessment of reporting across skin phototypes and rarer skin cancer types: A systematic review

Machine learning (ML) models for skin cancer recognition may have variable performance across different skin phototypes and skin cancer types. Overall performance metrics alone are insufficient to detect poor subgroup performance. We aimed (1) to assess whether studies of ML models reported results separately for different skin phototypes and rarer skin cancers, and (2) to graphically represent the skin cancer training datasets used by current ML models. In this systematic review, we searched PubMed, Embase and CENTRAL. We included all studies in medical journals assessing an ML technique for skin cancer diagnosis that used clinical or dermoscopic images from 1 January 2012 to 22 September 2021. No language restrictions were applied. We considered rarer skin cancers to be skin cancers other than pigmented melanoma, basal cell carcinoma and squamous cell carcinoma. We identified 114 studies for inclusion. Rarer skin cancers were included by 8/114 studies (7.0%), and results for a rarer skin cancer were reported separately in 1/114 studies (0.9%). Performance was reported across all skin phototypes in 1/114 studies (0.9%), but performance was uncertain in skin phototypes I and VI from minimal representation of the skin phototypes in the test dataset (9/3756 and 1/3756, respectively). For training datasets, although public datasets were most frequently used, with the most widely used being the International Skin Imaging Collaboration (ISIC) archive (65/114 studies, 57.0%), the largest datasets were private. Our review identified that most ML models did not report performance separately for rarer skin cancers and different skin phototypes. A degree of variability in ML model performance across subgroups is expected, but the current lack of transparency is not justifiable and risks models being used inappropriately in populations in whom accuracy is low.

Objective and automatic grading system of facial signs from smartphones' pictures in South African men: Validation versus dermatologists and characterization of changes with age

OBJECTIVE

To evaluate the capacity of the automatic detection system to accurately grade, from selfie pictures, the severity of eight facial signs in South African men.

METHODS

Selfie pictures (obtained from frontal and back cameras) of 281 South African men differently aged (20-70 years) were obtained and analyzed by an automatic artificial intelligence (AI)-based automatic grading system. Data were compared with the clinical gradings made by experts and dermatologists.

RESULTS

In all facial signs, both series of gradings were found highly correlated with, however, different coefficients (0.59-0.95), those of marionette lines and cheek pores being of lower values. No differences were observed between data obtained by frontal and back cameras. With age, in most cases, gradings show up to the 50-59 year age-class, linear-like changes. When compared to men of other ancestries, South African men present lower wrinkles/texture, pigmentation, and ptosis/sagging scores till 50-59 years, albeit not much different in the cheek pores sign. The early onset (mean age) of visibility of wrinkles/texture for South African men were (i.e., reaching grade >1) 39 and 45 years for ptosis/sagging.

CONCLUSION

This study completes and enlarges the previous works conducted on men of other ancestries by showing some South African specificities and slight differences with men of comparable phototypes (Afro American).

Convolutional neural network models for automatic diagnosis and graduation in skin frostbite

The study aimed to develop and validate a convolutional neural network (CNN)-based deep learning method for automatic diagnosis and graduation of skin frostbite. A dataset of 71 annotated images was used for the training, the validation, and the testing based on ResNet-50 model. The performances were evaluated with the test set. The diagnosis and graduation performance of our approach was compared with two residents from burns department. The approach correctly identified all the frostbite of IV (18/18, 100%), but with respectively 1 mistake in the diagnosis of degree I (29/30, 96.67%), II (28/29, 96.55%) and III (37/38, 97.37%). The accuracy of the approach on the whole test set was 97.39% (112/115). The accuracy of the two residents were respectively 77.39% and 73.04%. Weighted Kappa of 0.583 indicates good reliability between the two residents (P = .445). Kendall's coefficient of concordance is 0.326 (P = .548), indicating differences in accuracy between the approach and the two residents. Our approach based on CNNs demonstrated an encouraging performance for the automatic diagnosis and graduation of skin frostbite, with higher accuracy and efficiency.

Predicting Hypoperfusion Lesion and Target Mismatch in Stroke from Diffusion-weighted MRI Using Deep Learning

Background Perfusion imaging is important to identify a target mismatch in stroke but requires contrast agents and postprocessing software. Purpose To use a deep learning model to predict the hypoperfusion lesion in stroke and identify patients with a target mismatch profile from diffusion-weighted imaging (DWI) and clinical information alone, using perfusion MRI as the reference standard. Materials and Methods Imaging data sets of patients with acute ischemic stroke with baseline perfusion MRI and DWI were retrospectively reviewed from multicenter data available from 2008 to 2019 (Imaging Collaterals in Acute Stroke, Diffusion and Perfusion Imaging Evaluation for Understanding Stroke Evolution 2, and University of California, Los Angeles stroke registry). For perfusion MRI, rapid processing of perfusion and diffusion software automatically segmented the hypoperfusion lesion (time to maximum, ≥6 seconds) and ischemic core (apparent diffusion coefficient [ADC], ≤620 × 10-6 mm2/sec). A three-dimensional U-Net deep learning model was trained using baseline DWI, ADC, National Institutes of Health Stroke Scale score, and stroke symptom sidedness as inputs, with the union of hypoperfusion and ischemic core segmentation serving as the ground truth. Model performance was evaluated using the Dice score coefficient (DSC). Target mismatch classification based on the model was compared with that of the clinical-DWI mismatch approach defined by the DAWN trial by using the McNemar test. Results Overall, 413 patients (mean age, 67 years ± 15 [SD]; 207 men) were included for model development and primary analysis using fivefold cross-validation (247, 83, and 83 patients in the training, validation, and test sets, respectively, for each fold). The model predicted the hypoperfusion lesion with a median DSC of 0.61 (IQR, 0.45-0.71). The model identified patients with target mismatch with a sensitivity of 90% (254 of 283; 95% CI: 86, 93) and specificity of 77% (100 of 130; 95% CI: 69, 83) compared with the clinical-DWI mismatch sensitivity of 50% (140 of 281; 95% CI: 44, 56) and specificity of 89% (116 of 130; 95% CI: 83, 94) (P < .001 for all). Conclusion A three-dimensional U-Net deep learning model predicted the hypoperfusion lesion from diffusion-weighted imaging (DWI) and clinical information and identified patients with a target mismatch profile with higher sensitivity than the clinical-DWI mismatch approach. ClinicalTrials.gov registration nos. NCT02225730, NCT01349946, NCT02586415 © RSNA, 2022 Supplemental material is available for this article. See also the editorial by Kallmes and Rabinstein in this issue.

Exploring the potential of artificial intelligence in improving skin lesion diagnosis in primary care

Dermatological conditions are a relevant health problem. Machine learning (ML) models are increasingly being applied to dermatology as a diagnostic decision support tool using image analysis, especially for skin cancer detection and disease classification. The objective of this study was to perform a prospective validation of an image analysis ML model, which is capable of screening 44 skin diseases, comparing its diagnostic accuracy with that of General Practitioners (GPs) and teledermatology (TD) dermatologists in a real-life setting. Prospective, diagnostic accuracy study including 100 consecutive patients with a skin problem who visited a participating GP in central Catalonia, Spain, between June 2021 and October 2021. The skin issue was first assessed by the GPs. Then an anonymised skin disease picture was taken and uploaded to the ML application, which returned a list with the Top-5 possible diagnosis in order of probability. The same image was then sent to a dermatologist via TD for diagnosis, as per clinical practice. The GPs Top-3, ML model's Top-5 and dermatologist's Top-3 assessments were compared to calculate the accuracy, sensitivity, specificity and diagnostic accuracy of the ML models. The overall Top-1 accuracy of the ML model (39%) was lower than that of GPs (64%) and dermatologists (72%). When the analysis was limited to the diagnoses on which the algorithm had been explicitly trained (n = 82), the balanced Top-1 accuracy of the ML model increased (48%) and in the Top-3 (75%) was comparable to the GPs Top-3 accuracy (76%). The Top-5 accuracy of the ML model (89%) was comparable to the dermatologist Top-3 accuracy (90%). For the different diseases, the sensitivity of the model (Top-3 87% and Top-5 96%) is higher than that of the clinicians (Top-3 GPs 76% and Top-3 dermatologists 84%) only in the benign tumour pathology group, being on the other hand the most prevalent category (n = 53). About the satisfaction of professionals, 92% of the GPs considered it as a useful diagnostic support tool (DST) for the differential diagnosis and in 60% of the cases as an aid in the final diagnosis of the skin lesion. The overall diagnostic accuracy of the model in this study, under real-life conditions, is lower than that of both GPs and dermatologists. This result aligns with the findings of few existing prospective studies conducted under real-life conditions. The outcomes emphasize the significance of involving clinicians in the training of the model and the capability of ML models to assist GPs, particularly in differential diagnosis. Nevertheless, external testing in real-life conditions is crucial for data validation and regulation of these AI diagnostic models before they can be used in primary care.

Differentiating malignant and benign eyelid lesions using deep learning

Artificial intelligence as a screening tool for eyelid lesions will be helpful for early diagnosis of eyelid malignancies and proper decision-making. This study aimed to evaluate the performance of a deep learning model in differentiating eyelid lesions using clinical eyelid photographs in comparison with human ophthalmologists. We included 4954 photographs from 928 patients in this retrospective cross-sectional study. Images were classified into three categories: malignant lesion, benign lesion, and no lesion. Two pre-trained convolutional neural network (CNN) models, DenseNet-161 and EfficientNetV2-M architectures, were fine-tuned to classify images into three or two (malignant versus benign) categories. For a ternary classification, the mean diagnostic accuracies of the CNNs were 82.1% and 83.0% using DenseNet-161 and EfficientNetV2-M, respectively, which were inferior to those of the nine clinicians (87.0-89.5%). For the binary classification, the mean accuracies were 87.5% and 92.5% using DenseNet-161 and EfficientNetV2-M models, which was similar to that of the clinicians (85.8-90.0%). The mean AUC of the two CNN models was 0.908 and 0.950, respectively. Gradient-weighted class activation map successfully highlighted the eyelid tumors on clinical photographs. Deep learning models showed a promising performance in discriminating malignant versus benign eyelid lesions on clinical photographs, reaching the level of human observers.

Anomaly Detection in Chest X-rays Based on Dual-Attention Mechanism and Multi-Scale Feature Fusion

The efficient and automatic detection of chest abnormalities is vital for the auxiliary diagnosis of medical images. Many studies utilize computer vision and deep learning approaches involving symmetry and asymmetry concepts to detect chest abnormalities, and achieve promising findings. However, an accurate instance-level and multi-label detection of abnormalities in chest X-rays remains a significant challenge. Here, a novel anomaly detection method for symmetric chest X-rays using dual-attention and multi-scale feature fusion is proposed. Three aspects of our method should be noted in comparison with the previous approaches. We improved the deep neural network with channel-dimensional and spatial-dimensional attention to capture the abundant contextual features. We then used an optimized multi-scale learning framework for feature fusion to adapt to the scale variation in the abnormalities. Considering the influence of the data imbalance and other factors, we introduced a seesaw loss function to flexibly adjust the sample weights and enhance the model learning efficiency. The rigorous experimental evaluation of a public chest X-ray dataset with fourteen different types of abnormalities demonstrates that our model has a mean average precision of 0.362 and outperforms existing methods.

Relationship between the deep features of the full-scan pathological map of mucinous gastric carcinoma and related genes based on deep learning

Background

Long-term differential expression of disease-associated genes is a crucial driver of pathological changes in mucinous gastric carcinoma. Therefore, there should be a correlation between depth features extracted from pathology-based full-scan images using deep learning and disease-associated gene expression. This study tried to provides preliminary evidence that long-term differentially expressed (disease-associated) genes lead to subtle changes in disease pathology by exploring their correlation, and offer a new ideas for precise analysis of pathomics and combined analysis of pathomics and genomics.

Methods

Full pathological scans, gene sequencing data, and clinical data of patients with mucinous gastric carcinoma were downloaded from TCGA data. The VGG-16 network architecture was used to construct a binary classification model to explore the potential of VGG-16 applications and extract the deep features of the pathology-based full-scan map. Differential gene expression analysis was performed and a protein-protein interaction network was constructed to screen disease-related core genes. Differential, Lasso regression, and extensive correlation analyses were used to screen for valuable deep features. Finally, a correlation analysis was used to determine whether there was a correlation between valuable deep features and disease-related core genes.

Result

The accuracy of the binary classification model was 0.775 ± 0.129. A total of 24 disease-related core genes were screened, including ASPM, AURKA, AURKB, BUB1, BUB1B, CCNA2, CCNB1, CCNB2, CDCA8, CDK1, CENPF, DLGAP5, KIF11, KIF20A, KIF2C, KIF4A, MELK, PBK, RRM2, TOP2A, TPX2, TTK, UBE2C, and ZWINT. In addition, differential, Lasso regression, and extensive correlation analyses were used to screen eight valuable deep features, including features 51, 106, 109, 118, 257, 282, 326, and 487. Finally, the results of the correlation analysis suggested that valuable deep features were either positively or negatively correlated with core gene expression.

Conclusion

The preliminary results of this study support our hypotheses. Deep learning may be an important bridge for the joint analysis of pathomics and genomics and provides preliminary evidence for long-term abnormal expression of genes leading to subtle changes in pathology.

A deep-learning algorithm to classify skin lesions from mpox virus infection

Undetected infection and delayed isolation of infected individuals are key factors driving the monkeypox virus (now termed mpox virus or MPXV) outbreak. To enable earlier detection of MPXV infection, we developed an image-based deep convolutional neural network (named MPXV-CNN) for the identification of the characteristic skin lesions caused by MPXV. We assembled a dataset of 139,198 skin lesion images, split into training/validation and testing cohorts, comprising non-MPXV images (n = 138,522) from eight dermatological repositories and MPXV images (n = 676) from the scientific literature, news articles, social media and a prospective cohort of the Stanford University Medical Center (n = 63 images from 12 patients, all male). In the validation and testing cohorts, the sensitivity of the MPXV-CNN was 0.83 and 0.91, the specificity was 0.965 and 0.898 and the area under the curve was 0.967 and 0.966, respectively. In the prospective cohort, the sensitivity was 0.89. The classification performance of the MPXV-CNN was robust across various skin tones and body regions. To facilitate the usage of the algorithm, we developed a web-based app by which the MPXV-CNN can be accessed for patient guidance. The capability of the MPXV-CNN for identifying MPXV lesions has the potential to aid in MPXV outbreak mitigation.

Interpretable Skin Cancer Classification based on Incremental Domain Knowledge Learning

The recent advances in artificial intelligence have led to the rapid development of computer-aided skin cancer diagnosis applications that perform on par with dermatologists. However, the black-box nature of such applications makes it difficult for physicians to trust the predicted decisions, subsequently preventing the proliferation of such applications in the clinical workflow. In this work, we aim to address this challenge by developing an interpretable skin cancer diagnosis approach using clinical images. Accordingly, a skin cancer diagnosis model consolidated with two interpretability methods is developed. The first interpretability method integrates skin cancer diagnosis domain knowledge, characterized by a skin lesion taxonomy, into model development, whereas the other method focuses on visualizing the decision-making process by highlighting the dominant of interest regions of skin lesion images. The proposed model is trained and validated on clinical images since the latter are easily obtainable by non-specialist healthcare providers. The results demonstrate the effectiveness of incorporating lesion taxonomy in improving model classification accuracy, where our model can predict the skin lesion origin as melanocytic or non-melanocytic with an accuracy of 87%, predict lesion malignancy with 77% accuracy, and provide disease diagnosis with an accuracy of 71%. In addition, the implemented interpretability methods assist understand the model's decision-making process and detecting misdiagnoses. This work is a step toward achieving interpretability in skin cancer diagnosis using clinical images. The developed approach can assist general practitioners to make an early diagnosis, thus reducing the redundant referrals that expert dermatologists receive for further investigations.

PlexusNet: A neural network architectural concept for medical image classification

State-of-the-art (SOTA) convolutional neural network models have been widely adapted in medical imaging and applied to address different clinical problems. However, the complexity and scale of such models may not be justified in medical imaging and subject to the available resource budget. Further increasing the number of representative feature maps for the classification task decreases the model explainability. The current data normalization practice is fixed prior to model development and discounting the specification of the data domain. Acknowledging these issues, the current work proposed a new scalable model family called PlexusNet; the block architecture and model scaling by the network's depth, width, and branch regulate PlexusNet's architecture. The efficient computation costs outlined the dimensions of PlexusNet scaling and design. PlexusNet includes a new learnable data normalization algorithm for better data generalization. We applied a simple yet effective neural architecture search to design PlexusNet tailored to five clinical classification problems that achieve a performance noninferior to the SOTA models ResNet-18 and EfficientNet B0/1. It also does so with lower parameter capacity and representative feature maps in ten-fold ranges than the smallest SOTA models with comparable performance. The visualization of representative features revealed distinguishable clusters associated with categories based on latent features generated by PlexusNet. The package and source code are at https://github.com/oeminaga/PlexusNet.git.

Performance of ChatGPT on USMLE: Potential for AI-assisted medical education using large language models

We evaluated the performance of a large language model called ChatGPT on the United States Medical Licensing Exam (USMLE), which consists of three exams: Step 1, Step 2CK, and Step 3. ChatGPT performed at or near the passing threshold for all three exams without any specialized training or reinforcement. Additionally, ChatGPT demonstrated a high level of concordance and insight in its explanations. These results suggest that large language models may have the potential to assist with medical education, and potentially, clinical decision-making.

The dominant logic of Big Tech in healthcare and pharma

Digital health and digital pharma are considered supportive tools for patients and healthcare providers (HCPs), making the market highly attractive for industry players. Not surprisingly, Tech Giants have started to move into this area. We utilized established management models and publicly available information sources, such as annual company reports, and performed a thorough analysis to uncover the underlying business models of Alphabet, Amazon, Apple, IBM, and Microsoft in order to better understand their intention and course of entering the healthcare and pharma industries. Our results indicate that Big Tech or Tech Giants do address the needs of patients and physicians, while having built clear value propositions, value chains, and revenue models to sustainably revolutionize the healthcare and pharma industries.

AAPM task group report 273: Recommendations on best practices for AI and machine learning for computer-aided diagnosis in medical imaging

Rapid advances in artificial intelligence (AI) and machine learning, and specifically in deep learning (DL) techniques, have enabled broad application of these methods in health care. The promise of the DL approach has spurred further interest in computer-aided diagnosis (CAD) development and applications using both "traditional" machine learning methods and newer DL-based methods. We use the term CAD-AI to refer to this expanded clinical decision support environment that uses traditional and DL-based AI methods. Numerous studies have been published to date on the development of machine learning tools for computer-aided, or AI-assisted, clinical tasks. However, most of these machine learning models are not ready for clinical deployment. It is of paramount importance to ensure that a clinical decision support tool undergoes proper training and rigorous validation of its generalizability and robustness before adoption for patient care in the clinic. To address these important issues, the American Association of Physicists in Medicine (AAPM) Computer-Aided Image Analysis Subcommittee (CADSC) is charged, in part, to develop recommendations on practices and standards for the development and performance assessment of computer-aided decision support systems. The committee has previously published two opinion papers on the evaluation of CAD systems and issues associated with user training and quality assurance of these systems in the clinic. With machine learning techniques continuing to evolve and CAD applications expanding to new stages of the patient care process, the current task group report considers the broader issues common to the development of most, if not all, CAD-AI applications and their translation from the bench to the clinic. The goal is to bring attention to the proper training and validation of machine learning algorithms that may improve their generalizability and reliability and accelerate the adoption of CAD-AI systems for clinical decision support.

Convolutional neural network-based skin image segmentation model to improve classification of skin diseases in conventional and non-standardized picture images

BACKGROUND

For dermatological practices, non-standardized conventional photo images are taken and collected as a mixture of variable fields of the image view, including close-up images focusing on designated lesions and long-shot images including normal skin and background of the body surface. Computer-aided detection/diagnosis (CAD) models trained using non-standardized conventional photo images exhibit lower performance rates than CAD models that detect lesions in a localized small area, such as dermoscopic images.

OBJECTIVE

We aimed to develop a convolutional neural network (CNN) model for skin image segmentation to generate a skin disease image dataset suitable for CAD of multiple skin disease classification.

METHODS

We trained a DeepLabv3 + -based CNN segmentation model to detect skin and lesion areas and segmented out areas that satisfy the following conditions: more than 80% of the image will be the skin area, and more than 10% of the image will be the lesion area.

RESULTS

The generated CNN-segmented image database was examined using CAD of skin disease classification and achieved approximately 90% sensitivity and specificity to differentiate atopic dermatitis from malignant diseases and complications, such as mycosis fungoides, impetigo, and herpesvirus infection. The accuracy of skin disease classification in the CNN-segmented image dataset was almost equal to that of the manually cropped image dataset and higher than that of the original image dataset.

CONCLUSION

Our CNN segmentation model, which automatically extracts lesions and segmented images of the skin area regardless of image fields, will reduce the burden of physician annotation and improve CAD performance.

A cell phone app for facial acne severity assessment

Acne vulgaris, the most common skin disease, can cause substantial economic and psychological impacts to the people it affects, and its accurate grading plays a crucial role in the treatment of patients. In this paper, we firstly proposed an acne grading criterion that considers lesion classifications and a metric for producing accurate severity ratings. Due to similar appearance of acne lesions with comparable severities and difficult-to-count lesions, severity assessment is a challenging task. We cropped facial skin images of several lesion patches and then addressed the acne lesion with a lightweight acne regular network (Acne-RegNet). Acne-RegNet was built by using a median filter and histogram equalization to improve image quality, a channel attention mechanism to boost the representational power of network, a region-based focal loss to handle classification imbalances and a model pruning and feature-based knowledge distillation to reduce model size. After the application of Acne-RegNet, the severity score is calculated, and the acne grading is further optimized by the metadata of the patients. The entire acne assessment procedure was deployed to a mobile device, and a phone app was designed. Compared with state-of-the-art lightweight models, the proposed Acne-RegNet significantly improves the accuracy of lesion classifications. The acne app demonstrated promising results in severity assessments (accuracy: 94.56%) and showed a dermatologist-level diagnosis on the internal clinical dataset.The proposed acne app could be a useful adjunct to assess acne severity in clinical practice and it enables anyone with a smartphone to immediately assess acne, anywhere and anytime.

Artificial intelligence-based point-of-care lung ultrasound for screening COVID-19 pneumoniae: Comparison with CT scans

BACKGROUND

Although lung ultrasound has been reported to be a portable, cost-effective, and accurate method to detect pneumonia, it has not been widely used because of the difficulty in its interpretation. Here, we aimed to investigate the effectiveness of a novel artificial intelligence-based automated pneumonia detection method using point-of-care lung ultrasound (AI-POCUS) for the coronavirus disease 2019 (COVID-19).

METHODS

We enrolled consecutive patients admitted with COVID-19 who underwent computed tomography (CT) in August and September 2021. A 12-zone AI-POCUS was performed by a novice observer using a pocket-size device within 24 h of the CT scan. Fifteen control subjects were also scanned. Additionally, the accuracy of the simplified 8-zone scan excluding the dorsal chest, was assessed. More than three B-lines detected in one lung zone were considered zone-level positive, and the presence of positive AI-POCUS in any lung zone was considered patient-level positive. The sample size calculation was not performed given the retrospective all-comer nature of the study.

RESULTS

A total of 577 lung zones from 56 subjects (59.4 ± 14.8 years, 23% female) were evaluated using AI-POCUS. The mean number of days from disease onset was 9, and 14% of patients were under mechanical ventilation. The CT-validated pneumonia was seen in 71.4% of patients at total 577 lung zones (53.3%). The 12-zone AI-POCUS for detecting CT-validated pneumonia in the patient-level showed the accuracy of 94.5% (85.1%- 98.1%), sensitivity of 92.3% (79.7%- 97.3%), specificity of 100% (80.6%- 100%), positive predictive value of 95.0% (89.6% - 97.7%), and Kappa of 0.33 (0.27-0.40). When simplified with 8-zone scan, the accuracy, sensitivity, and sensitivity were 83.9% (72.2%- 91.3%), 77.5% (62.5%- 87.7%), and 100% (80.6%- 100%), respectively. The zone-level accuracy, sensitivity, and specificity of AI-POCUS were 65.3% (61.4%- 69.1%), 37.2% (32.0%- 42.7%), and 97.8% (95.2%- 99.0%), respectively.

INTERPRETATION

AI-POCUS using the novel pocket-size ultrasound system showed excellent agreement with CT-validated COVID-19 pneumonia, even when used by a novice observer.

AFFIRM: Affinity Fusion-Based Framework for Iteratively Random Motion Correction of Multi-Slice Fetal Brain MRI

Multi-slice magnetic resonance images of the fetal brain are usually contaminated by severe and arbitrary fetal and maternal motion. Hence, stable and robust motion correction is necessary to reconstruct high-resolution 3D fetal brain volume for clinical diagnosis and quantitative analysis. However, the conventional registration-based correction has a limited capture range and is insufficient for detecting relatively large motions. Here, we present a novel Affinity Fusion-based Framework for Iteratively Random Motion (AFFIRM) correction of the multi-slice fetal brain MRI. It learns the sequential motion from multiple stacks of slices and integrates the features between 2D slices and reconstructed 3D volume using affinity fusion, which resembles the iterations between slice-to-volume registration and volumetric reconstruction in the regular pipeline. The method accurately estimates the motion regardless of brain orientations and outperforms other state-of-the-art learning-based methods on the simulated motion-corrupted data, with a 48.4% reduction of mean absolute error for rotation and 61.3% for displacement. We then incorporated AFFIRM into the multi-resolution slice-to-volume registration and tested it on the real-world fetal MRI scans at different gestation stages. The results indicated that adding AFFIRM to the conventional pipeline improved the success rate of fetal brain super-resolution reconstruction from 77.2% to 91.9%.

Accuracy and clinical relevance of an automated, algorithm-based analysis of facial signs from selfie images of women in the United States of various ages, ancestries and phototypes: A cross-sectional observational study

BACKGROUND

Real-life validation is necessary to ensure our artificial intelligence (AI) skin diagnostic tool is inclusive across a diverse and representative US population of various ages, ancestries and skin phototypes.

OBJECTIVES

To explore the relevance and accuracy of an automated, algorithm-based analysis of facial signs in representative women of different ancestries, ages and phototypes, living in the same country.

METHODS

In a cross-sectional study of selfie images of 1041 US women, algorithm-based analyses of seven facial signs were automatically graded by an AI-based algorithm and by 50 US dermatologists of various profiles (age, gender, ancestry, geographical location). For automated analysis and dermatologist assessment, the same referential skin atlas was used to standardize the grading scales. The average values and their variability were compared with respect to age, ancestry and phototype.

RESULTS

For five signs, the grading obtained by the automated system were strongly correlated with dermatologists' assessments (r ≥ 0.75); cheek skin pores were moderately correlated (r = 0.63) and pigmentation signs, especially for the darkest skin tones, were weakly correlated (r = 0.40) to the dermatologist assessments. Age and ancestry had no effect on the correlations. In many cases, the automated system performed better than the dermatologist-assessed clinical grading due to 0.3-0.5 grading unit differences among the dermatologist panel that were not related to any individual characteristic (e.g. gender, age, ancestry, location). The use of phototypes, as discontinuous categorical variables, is likely a limiting factor in the assessments of grading, whether obtained by automated analysis or clinical assessment of the images.

CONCLUSIONS

The AI-based automatic procedure is accurate and clinically relevant for analysing facial signs in a diverse and inclusive population of US women, as confirmed by a diverse panel of dermatologists, although skin tone requires further improvement.

A study on skin tumor classification based on dense convolutional networks with fused metadata

Skin cancer is the most common cause of death in humans. Statistics show that competent dermatologists have a diagnostic accuracy rate of less than 80%, while inexperienced dermatologists have a diagnostic accuracy rate of less than 60%. The higher rate of misdiagnosis will cause many patients to miss the most effective treatment window, risking the patients' life safety. However, the majority of the current study of neural network-based skin cancer diagnosis remains at the image level without patient clinical data. A deep convolutional network incorporating clinical patient metadata of skin cancer is presented to realize the classification model of skin cancer in order to further increase the accuracy of skin cancer diagnosis. There are three basic steps in the approach. First, the high-level features (edge features, color features, texture features, form features, etc.). Implied by the image were retrieved using the pre-trained DenseNet-169 model on the ImageNet dataset. Second, the MetaNet module is introduced, which uses metadata to control a certain portion of each feature channel in the DenseNet-169 network in order to produce weighted features. The MetaBlock module was added at the same time to improve the features retrieved from photos using metadata, choosing the most pertinent characteristics in accordance with the metadata data. The features of the MetaNet and MetaBlock modules were finally combined to create the MD-Net module, which was then used as input into the classifier to get the classification results for skin cancers. On the PAD-UFES-20 and ISIC 2019 datasets, the suggested methodology was assessed. The DenseNet-169 network model combined with this module, according to experimental data, obtains 81.4% in the balancing accuracy index, and its diagnostic accuracy is up between 8% and 15.6% compared to earlier efforts. Additionally, it solves the problem of actinic keratosis and poorly classified skin fibromas.

Shifting machine learning for healthcare from development to deployment and from models to data

In the past decade, the application of machine learning (ML) to healthcare has helped drive the automation of physician tasks as well as enhancements in clinical capabilities and access to care. This progress has emphasized that, from model development to model deployment, data play central roles. In this Review, we provide a data-centric view of the innovations and challenges that are defining ML for healthcare. We discuss deep generative models and federated learning as strategies to augment datasets for improved model performance, as well as the use of the more recent transformer models for handling larger datasets and enhancing the modelling of clinical text. We also discuss data-focused problems in the deployment of ML, emphasizing the need to efficiently deliver data to ML models for timely clinical predictions and to account for natural data shifts that can deteriorate model performance.

Attention Cost-Sensitive Deep Learning-Based Approach for Skin Cancer Detection and Classification

Deep learning-based models have been employed for the detection and classification of skin diseases through medical imaging. However, deep learning-based models are not effective for rare skin disease detection and classification. This is mainly due to the reason that rare skin disease has very a smaller number of data samples. Thus, the dataset will be highly imbalanced, and due to the bias in learning, most of the models give better performances. The deep learning models are not effective in detecting the affected tiny portions of skin disease in the overall regions of the image. This paper presents an attention-cost-sensitive deep learning-based feature fusion ensemble meta-classifier approach for skin cancer detection and classification. Cost weights are included in the deep learning models to handle the data imbalance during training. To effectively learn the optimal features from the affected tiny portions of skin image samples, attention is integrated into the deep learning models. The features from the finetuned models are extracted and the dimensionality of the features was further reduced by using a kernel-based principal component (KPCA) analysis. The reduced features of the deep learning-based finetuned models are fused and passed into ensemble meta-classifiers for skin disease detection and classification. The ensemble meta-classifier is a two-stage model. The first stage performs the prediction of skin disease and the second stage performs the classification by considering the prediction of the first stage as features. Detailed analysis of the proposed approach is demonstrated for both skin disease detection and skin disease classification. The proposed approach demonstrated an accuracy of 99% on skin disease detection and 99% on skin disease classification. In all the experimental settings, the proposed approach outperformed the existing methods and demonstrated a performance improvement of 4% accuracy for skin disease detection and 9% accuracy for skin disease classification. The proposed approach can be used as a computer-aided diagnosis (CAD) tool for the early diagnosis of skin cancer detection and classification in healthcare and medical environments. The tool can accurately detect skin diseases and classify the skin disease into their skin disease family.

Deep learning for the detection of anatomical tissue structures and neoplasms of the skin on scanned histopathological tissue sections

Basal cell carcinoma (BCC), squamous cell carcinoma (SqCC) and melanoma are among the most common cancer types. Correct diagnosis based on histological evaluation after biopsy or excision is paramount for adequate therapy stratification. Deep learning on histological slides has been suggested to complement and improve routine diagnostics, but publicly available curated and annotated data and usable models trained to distinguish common skin tumors are rare and often lack heterogeneous non-tumor categories. A total of 16 classes from 386 cases were manually annotated on scanned histological slides, 129,364 100 x 100 µm (~395 x 395 px) image tiles were extracted and split into a training, validation and test set. An EfficientV2 neuronal network was trained and optimized to classify image categories. Cross entropy loss, balanced accuracy and Matthews correlation coefficient were used for model evaluation. Image and patient data were assessed with confusion matrices. Application of the model to an external set of whole slides facilitated localization of melanoma and non-tumor tissue. Automated differentiation of BCC, SqCC, melanoma, naevi and non-tumor tissue structures was possible, and a high diagnostic accuracy was achieved in the validation (98%) and test (97%) set. In summary, we provide a curated dataset including the most common neoplasms of the skin and various anatomical compartments to enable researchers to train, validate and improve deep learning models. Automated classification of skin tumors by deep learning techniques is possible with high accuracy, facilitates tumor localization and has the potential to support and improve routine diagnostics.

Artificial Intelligence in Dermatology Image Analysis: Current Developments and Future Trends

BACKGROUND

Thanks to the rapid development of computer-based systems and deep-learning-based algorithms, artificial intelligence (AI) has long been integrated into the healthcare field. AI is also particularly helpful in image recognition, surgical assistance and basic research. Due to the unique nature of dermatology, AI-aided dermatological diagnosis based on image recognition has become a modern focus and future trend. Key scientific concepts of review: The use of 3D imaging systems allows clinicians to screen and label skin pigmented lesions and distributed disorders, which can provide an objective assessment and image documentation of lesion sites. Dermatoscopes combined with intelligent software help the dermatologist to easily correlate each close-up image with the corresponding marked lesion in the 3D body map. In addition, AI in the field of prosthetics can assist in the rehabilitation of patients and help to restore limb function after amputation in patients with skin tumors.

THE AIM OF THE STUDY

For the benefit of patients, dermatologists have an obligation to explore the opportunities, risks and limitations of AI applications. This study focuses on the application of emerging AI in dermatology to aid clinical diagnosis and treatment, analyzes the current state of the field and summarizes its future trends and prospects so as to help dermatologists realize the impact of new technological innovations on traditional practices so that they can embrace and use AI-based medical approaches more quickly.