Agreement between dermatological diagnoses made by live examination compared to analysis of digital images

Diagnostic reliability in teledermatology: a systematic review and a meta-analysis

OBJECTIVES

To compare teledermatology and face-to-face (F2F) agreement in primary diagnoses of dermatological conditions.

DESIGN

Systematic review and meta-analysis METHODS: MEDLINE, Embase, Cochrane Library (Wiley), CINAHL and medRxiv were searched between January 2010 and May 2022. Observational studies and randomised clinical trials that reported percentage agreement or kappa concordance for primary diagnoses between teledermatology and F2F physicians were included. Titles, abstracts and full-text articles were screened in duplicate. From 7173 citations, 44 articles were included. A random-effects meta-analysis was conducted to estimate pooled estimates. Primary outcome measures were mean percentage and kappa concordance for assessing diagnostic matches between teledermatology and F2F physicians. Secondary outcome measures included the agreement between teledermatologists, F2F dermatologists, and teledermatology and histopathology results.

RESULTS

44 studies were extracted and reviewed. The pooled agreement rate was 68.9%, and kappa concordance was 0.67. When dermatologists conducted F2F and teledermatology consults, the overall diagnostic agreement was significantly higher at 71% compared with 44% for non-specialists. Kappa concordance was 0.69 for teledermatologist versus specialist and 0.52 for non-specialists. Higher diagnostic agreements were also noted with image acquisition training and digital photography. The agreement rate was 76.4% between teledermatologists, 82.4% between F2F physicians and 55.7% between teledermatology and histopathology.

CONCLUSIONS AND RELEVANCE

Teledermatology can be an attractive option particularly in resource-poor settings. Future efforts should be placed on incorporating image acquisition training and access to high-quality imaging technologies.

TRIAL REGISTRATION NUMBER

10.17605/OSF.IO/FJDVG.

Diagnostic Reliability of In-Person Versus Remote Dermatology: A Meta-Analysis

Background:Studies comparing teledermatology with in-person dermatologists report wide variations in diagnostic agreement. Teledermatology studies should have two independent in-person consultations establishing a baseline for comparing diagnoses made face-to-face and those made remotely.Objective:To perform a meta-analysis of comparison studies having two in-person dermatologists and at least one remote dermatologist examining the same patients to determine the overall preponderance of agreement.Method:Studies having two in-person diagnosticians were identified from previous teledermatology research reviews and independent searches of PubMed and other databases. Data from six studies identified were meta-analyzed.Results:Some studies showed high levels of diagnostic concordance, while others did not. Meta-analysis revealed that concordance rates reported in the teledermatology and clinical (in-person) consultations were significantly different (odds ratio = 0.55 [Mantel-Haenszel, fixed effect model, 95% confidence interval = 0.42-0.72], χ2 = 11.87, p < 0.05, I2 = 58%). Overall results showed that in-person primary diagnoses are significantly more concordant than remote. The results also suggest that diagnoses made in-person and teledermatology were marginally but significantly different than remote.Conclusion:Although the results of this study suggest teledermatology diagnoses are less reliable than those in-person, there are still valid reasons for using teledermatology to improve access, reduce costs, and triage patients to determine those warranting further in-person consultation and/or laboratory tests. More caution should be exercised in teledermatology when diagnoses involve risky skin conditions. There is evidence that this happens in practice.

Evidence- and consensus-based recommendations for the use of pegvaliase in adults with phenylketonuria

Purpose

Phenylketonuria (PKU) is a rare metabolic disorder that requires life-long management to reduce phenylalanine (Phe) concentrations within the recommended range. The availability of pegvaliase (PALYNZIQ™, an enzyme that can metabolize Phe) as a new therapy necessitates the provision of guidance for its use.

Methods

A Steering Committee comprising 17 health-care professionals with experience in using pegvaliase through the clinical development program drafted guidance statements during a series of face-to-face meetings. A modified Delphi methodology was used to demonstrate consensus among a wider group of health-care professionals with experience in using pegvaliase.

Results

Guidance statements were developed for four categories: (1) treatment goals and considerations prior to initiating therapy, (2) dosing considerations, (3) considerations for dietary management, and (4) best approaches to optimize medical management. A total of 34 guidance statements were included in the modified Delphi voting and consensus was reached on all after two rounds of voting.

Conclusion

Here we describe evidence- and consensus-based recommendations for the use of pegvaliase in adults with PKU. The manuscript was evaluated against the Appraisal of Guidelines for Research and Evaluation (AGREE II) instrument and is intended for use by health-care professionals who will prescribe pegvaliase and those who will treat patients receiving pegvaliase.

Teledermatology for diagnosing skin cancer in adults

BACKGROUND

Early accurate detection of all skin cancer types is essential to guide appropriate management and to improve morbidity and survival. Melanoma and squamous cell carcinoma (SCC) are high-risk skin cancers which have the potential to metastasise and ultimately lead to death, whereas basal cell carcinoma (BCC) is usually localised with potential to infiltrate and damage surrounding tissue. Anxiety around missing early curable cases needs to be balanced against inappropriate referral and unnecessary excision of benign lesions. Teledermatology provides a way for generalist clinicians to access the opinion of a specialist dermatologist for skin lesions that they consider to be suspicious without referring the patients through the normal referral pathway. Teledermatology consultations can be 'store-and-forward' with electronic digital images of a lesion sent to a dermatologist for review at a later time, or can be live and interactive consultations using videoconferencing to connect the patient, referrer and dermatologist in real time.

OBJECTIVES

To determine the diagnostic accuracy of teledermatology for the detection of any skin cancer (melanoma, BCC or cutaneous squamous cell carcinoma (cSCC)) in adults, and to compare its accuracy with that of in-person diagnosis.

SEARCH METHODS

We undertook a comprehensive search of the following databases from inception up to August 2016: Cochrane Central Register of Controlled Trials, MEDLINE, Embase, CINAHL, CPCI, Zetoc, Science Citation Index, US National Institutes of Health Ongoing Trials Register, NIHR Clinical Research Network Portfolio Database and the World Health Organization International Clinical Trials Registry Platform. We studied reference lists and published systematic review articles.

SELECTION CRITERIA

Studies evaluating skin cancer diagnosis for teledermatology alone, or in comparison with face-to-face diagnosis by a specialist clinician, compared with a reference standard of histological confirmation or clinical follow-up and expert opinion. We also included studies evaluating the referral accuracy of teledermatology compared with a reference standard of face-to-face diagnosis by a specialist clinician.

DATA COLLECTION AND ANALYSIS

Two review authors independently extracted all data using a standardised data extraction and quality assessment form (based on QUADAS-2). We contacted authors of included studies where there were information related to the target condition of any skin cancer missing. Data permitting, we estimated summary sensitivities and specificities using the bivariate hierarchical model. Due to the scarcity of data, we undertook no covariate investigations for this review. For illustrative purposes, we plotted estimates of sensitivity and specificity on coupled forest plots for diagnostic threshold and target condition under consideration.

MAIN RESULTS

The review included 22 studies reporting diagnostic accuracy data for 4057 lesions and 879 malignant cases (16 studies) and referral accuracy data for reported data for 1449 lesions and 270 'positive' cases as determined by the reference standard face-to-face decision (six studies). Methodological quality was variable with poor reporting hindering assessment. The overall risk of bias was high or unclear for participant selection, reference standard, and participant flow and timing in at least half of all studies; the majority were at low risk of bias for the index test. The applicability of study findings were of high or unclear concern for most studies in all domains assessed due to the recruitment of participants from secondary care settings or specialist clinics rather than from primary or community-based settings in which teledermatology is more likely to be used and due to the acquisition of lesion images by dermatologists or in specialist imaging units rather than by primary care clinicians.Seven studies provided data for the primary target condition of any skin cancer (1588 lesions and 638 malignancies). For the correct diagnosis of lesions as malignant using photographic images, summary sensitivity was 94.9% (95% confidence interval (CI) 90.1% to 97.4%) and summary specificity was 84.3% (95% CI 48.5% to 96.8%) (from four studies). Individual study estimates using dermoscopic images or a combination of photographic and dermoscopic images generally suggested similarly high sensitivities with highly variable specificities. Limited comparative data suggested similar diagnostic accuracy between teledermatology assessment and in-person diagnosis by a dermatologist; however, data were too scarce to draw firm conclusions. For the detection of invasive melanoma or atypical intraepidermal melanocytic variants both sensitivities and specificities were more variable. Sensitivities ranged from 59% (95% CI 42% to 74%) to 100% (95% CI 48% to 100%) and specificities from 30% (95% CI 22% to 40%) to 100% (95% CI 93% to 100%), with reported diagnostic thresholds including the correct diagnosis of melanoma, classification of lesions as 'atypical' or 'typical, and the decision to refer or to excise a lesion.Referral accuracy data comparing teledermatology against a face-to-face reference standard suggested good agreement for lesions considered to require some positive action by face-to-face assessment (sensitivities of over 90%). For lesions considered of less concern when assessed face-to-face (e.g. for lesions not recommended for excision or referral), agreement was more variable with teledermatology specificities ranging from 57% (95% CI 39% to 73%) to 100% (95% CI 86% to 100%), suggesting that remote assessment is more likely recommend excision, referral or follow-up compared to in-person decisions.

AUTHORS' CONCLUSIONS

Studies were generally small and heterogeneous and methodological quality was difficult to judge due to poor reporting. Bearing in mind concerns regarding the applicability of study participants and of lesion image acquisition in specialist settings, our results suggest that teledermatology can correctly identify the majority of malignant lesions. Using a more widely defined threshold to identify 'possibly' malignant cases or lesions that should be considered for excision is likely to appropriately triage those lesions requiring face-to-face assessment by a specialist. Despite the increasing use of teledermatology on an international level, the evidence base to support its ability to accurately diagnose lesions and to triage lesions from primary to secondary care is lacking and further prospective and pragmatic evaluation is needed.

Telemedicine in dermatology: findings and experiences worldwide - a systematic literature review

Telemedicine has become an important element of health care in many countries and profited from the technological progress of the last two decades. Due to the visual character of the dermatological specialty, teledermatology in particular participated in that development and is becoming a major tool in dermatological consultation. The objective of this article was to identify the use of teledermatology across the world based on published original articles. A systematic literature search of the MEDLINE and Embase databases for eligible publications (predefined inclusion and exclusion criteria) and a cross-validation search were conducted. Search results were reviewed systematically. The search resulted in 204 publications meeting the inclusion criteria for analysis. The highest number of published studies on teledermatology was performed in the United States, followed by the United Kingdom, Spain, the Netherlands, Italy and Austria. The majority of dermatological indications for telemedical consultations were not specified or included various kinds of skin diseases, followed by skin cancer and wounds. Research questions predominantly focused on concordance, effectiveness and cost-effectiveness to determine the value. Teledermatology proved to be a reliable consultation tool in the majority of studies. If specified, telemedicine was used in daily dermatological routine for patient management purposes, to consult patients in peripheral locations, or for medical support in nursing homes or home care settings. The application of teledermatology worldwide is highest in North American and European countries, while countries with poor geographical distribution of physicians seem to be under-represented in teledermatological use, as concluded from publication output. Regarding indications, comparison with classic consultation and area of application, most studies were of general nature. For precise determination of the value, systematic studies would be needed. However, teledermatology is already accepted as a valid tool.

Comparing High Definition Live Interactive and Store-and-Forward Consultations to In-Person Examinations

BACKGROUND

There is little teledermatology research directly comparing remote methods, even less research with two in-person dermatologist agreement providing a baseline for comparing remote methods, and no research using high definition video as a live interactive method.

OBJECTIVE

To compare in-person consultations with store-and-forward and live interactive methods, the latter having two levels of image quality.

METHODS

A controlled study was conducted where patients were examined in-person, by high definition video, and by store-and-forward methods. The order patients experienced methods and residents assigned methods rotated, although an attending always saw patients in-person. The type of high definition video employed, lower resolution compressed or higher resolution uncompressed, was alternated between clinics. Primary and differential diagnoses, biopsy recommendations, and diagnostic and biopsy confidence ratings were recorded.

RESULTS

Concordance and confidence were significantly better for in-person versus remote methods and biopsy recommendations were lower. Store-and-forward and higher resolution uncompressed video results were similar and better than those for lower resolution compressed video.

LIMITATIONS

Dermatology residents took store-and-forward photos and their quality was likely superior to those normally taken in practice. There were variations in expertise between the attending and second and third year residents.

CONCLUSION

The superiority of in-person consultations suggests the tendencies to order more biopsies or still see patients in-person are often justified in teledermatology and that high resolution uncompressed video can close the resolution gap between store-and-forward and live interactive methods.

Practice Guidelines for Teledermatology

Previous American Telemedicine Association (ATA) Teledermatology Practice Guidelines were issued in 2007. This updated version reflects new knowledge in the field, new technologies, and the need to incorporate teledermatology practice in a variety of settings, including hospitals, urgent care centers, Federally Qualified Health Centers, school-based clinics, public health facilities, and patient homes.

A Prospective Study of Mobile Phones for Dermatology in a Clinical Setting

We evaluated the accuracy of diagnoses made from pictures taken with the built-in cameras of mobile phones in a ‘real-life’ clinical setting. A total of 263 patients took part, who photographed their own lesions where possible, and provided clinical information via a questionnaire. After the teledermatology procedure, each patient was examined face-to-face and a gold standard diagnosis was made. The telemedicine data and pictures were diagnosed by 15 dermatologists. The 299 cases contained 1–22 clinical images each (median 3). Nine dermatologists finished all the cases and the remaining six completed some of them, thus providing 2893 decisions. Overall, 61% of all cases were rated as possible to diagnose and of those, 80% were correct in comparison with the face-to-face diagnosis. Image quality was evaluated and the median was 5 on a 10-point scale. There was a significant correlation between the correct diagnosis and the quality of the photographs taken ( P & 0.001). In nearly two-thirds of all cases, a teledermatology diagnosis was possible; however, there was insufficient information to make a telemedicine diagnosis in about one-third of the cases. If applied carefully, mobile phones could be a powerful tool for people to optimize their health care status.

The Empirical Foundations of Teledermatology: A Review of the Research Evidence

INTRODUCTION

This article presents the scientific evidence for the merit of telemedicine interventions in the diagnosis and management of skin disorders (teledermatology) in the published literature. The impetus for this work derives from the high prevalence of skin disorders, the high cost, the limited availability of dermatologists in certain areas, and the promise of teledermatology to address unmet needs in this area.

MATERIALS AND METHODS

The findings are based on a targeted review of scientific studies published from January 2005 through April 2015. The initial search yielded some 5,020 articles in Google Scholar and 428 in PubMed. A review of the abstracts yielded 71 publications that met the inclusion criteria for this analysis. Evidence is organized according to the following: feasibility and acceptance; intermediate outcomes (use of service, compliance, and diagnostic and treatment concordance and accuracy); outcomes (health improvement and problem resolution); and cost savings. A special section is devoted to studies conducted at the Veterans Health Administration.

RESULTS

Definitions of teledermatology varied across a wide spectrum of skin disorders, technologies, diagnostic tools, provider types, settings, and patient populations. Outcome measures included diagnostic concordance, treatment plans, and health.

CONCLUSIONS

Despite these complexities, sufficient evidence was observed consistently supporting the effectiveness of teledermatology in improving accessibility to specialty care, diagnostic and treatment concordance, and skin care provided by primary care physicians, while also reducing cost. One study reported suboptimal clinical results from teledermatology for patients with pigmented skin lesions. On the other hand, confocal microscopy and advanced dermoscopy improved diagnostic accuracy, especially when rendered by experienced teledermatologists.

Teledermatology

This article provides an overview of teledermatology with an emphasis on the evidence most relevant to referring clinicians, who are often primary care clinicians. Discussion includes the different modalities used for teledermatology and their diagnostic reliability, diagnostic accuracy, impact on in-person dermatology visits, clinical outcomes, and user satisfaction.

Teledermatology protocol for screening of skin cancer

BACKGROUND

Telemedicine refers to the use of technology as improvement of healthcare delivery to places where distance becomes an obstacle. Its use represents a great potential for dermatology, a specialty whose visual analysis phase is essential in diagnosis.

OBJECTIVES

To analyze the compatibility index of skin cancer diagnoses between primary care and teledermatology, and to validate a protocol for standardization of digital imaging to obtain the reports in teledermatology.

METHODS

An observational cross-sectional study developed through the census of 333 examination requests, received between January/2012 and July/2012, in the Center for Telemedicine and Telehealth of SES-SC. We used a protocol for photographic lesion standardization, consisting of three steps (panoramic photo, close-up with ruler and dermoscopy). After collection, the data were sent to a virtual site on the Internet, and recorded with the use of an electronic health record containing the images, the skin phototype and demographic characteristics.

RESULTS

The level of compatibility between the diagnosis of skin cancer in Santa Catarina's primary care and the diagnosis proposed by teledermatology was 19.02%. Proportionally, it was 21.21% for BCC, 44.44% for SCC and 6.98% for MM. The protocol was statistically significant (p <0.05), with an OR of 38.77.

CONCLUSION

The rate of diagnostic compatibility of skin cancer was low and the use of the protocol optimized the chance of validating requests for examination.