Diagnostic accuracy of diabetic retinopathy grading by an artificial intelligence-enabled algorithm compared with a human standard for wide-field true-colour confocal scanning and standard digital retinal images

Comparison of 21 artificial intelligence algorithms in automated diabetic retinopathy screening using handheld fundus camera

BACKGROUND

Diabetic retinopathy (DR) is a common complication of diabetes and may lead to irreversible visual loss. Efficient screening and improved treatment of both diabetes and DR have amended visual prognosis for DR. The number of patients with diabetes is increasing and telemedicine, mobile handheld devices and automated solutions may alleviate the burden for healthcare. We compared the performance of 21 artificial intelligence (AI) algorithms for referable DR screening in datasets taken by handheld Optomed Aurora fundus camera in a real-world setting.

PATIENTS AND METHODS

Prospective study of 156 patients (312 eyes) attending DR screening and follow-up. Both papilla- and macula-centred 50° fundus images were taken from each eye. DR was graded by experienced ophthalmologists and 21 AI algorithms.

RESULTS

Most eyes, 183 out of 312 (58.7%), had no DR and mild NPDR was noted in 21 (6.7%) of the eyes. Moderate NPDR was detected in 66 (21.2%) of the eyes, severe NPDR in 1 (0.3%), and PDR in 41 (13.1%) composing a group of 34.6% of eyes with referable DR. The AI algorithms achieved a mean agreement of 79.4% for referable DR, but the results varied from 49.4% to 92.3%. The mean sensitivity for referable DR was 77.5% (95% CI 69.1-85.8) and specificity 80.6% (95% CI 72.1-89.2). The rate for images ungradable by AI varied from 0% to 28.2% (mean 1.9%). Nineteen out of 21 (90.5%) AI algorithms resulted in grading for DR at least in 98% of the images.

CONCLUSIONS

Fundus images captured with Optomed Aurora were suitable for DR screening. The performance of the AI algorithms varied considerably emphasizing the need for external validation of screening algorithms in real-world settings before their clinical application.

Transparency in Artificial Intelligence Reporting in Ophthalmology-A Scoping Review

Topic

This scoping review summarizes artificial intelligence (AI) reporting in ophthalmology literature in respect to model development and validation. We characterize the state of transparency in reporting of studies prospectively validating models for disease classification.

Clinical Relevance

Understanding what elements authors currently describe regarding their AI models may aid in the future standardization of reporting. This review highlights the need for transparency to facilitate the critical appraisal of models prior to clinical implementation, to minimize bias and inappropriate use. Transparent reporting can improve effective and equitable use in clinical settings.

Methods

Eligible articles (as of January 2022) from PubMed, Embase, Web of Science, and CINAHL were independently screened by 2 reviewers. All observational and clinical trial studies evaluating the performance of an AI model for disease classification of ophthalmic conditions were included. Studies were evaluated for reporting of parameters derived from reporting guidelines (CONSORT-AI, MI-CLAIM) and our previously published editorial on model cards. The reporting of these factors, which included basic model and dataset details (source, demographics), and prospective validation outcomes, were summarized.

Results

Thirty-seven prospective validation studies were included in the scoping review. Eleven additional associated training and/or retrospective validation studies were included if this information could not be determined from the primary articles. These 37 studies validated 27 unique AI models; multiple studies evaluated the same algorithms (EyeArt, IDx-DR, and Medios AI). Details of model development were variably reported; 18 of 27 models described training dataset annotation and 10 of 27 studies reported training data distribution. Demographic information of training data was rarely reported; 7 of the 27 unique models reported age and gender and only 2 reported race and/or ethnicity. At the level of prospective clinical validation, age and gender of populations was more consistently reported (29 and 28 of 37 studies, respectively), but only 9 studies reported race and/or ethnicity data. Scope of use was difficult to discern for the majority of models. Fifteen studies did not state or imply primary users.

Conclusion

Our scoping review demonstrates variable reporting of information related to both model development and validation. The intention of our study was not to assess the quality of the factors we examined, but to characterize what information is, and is not, regularly reported. Our results suggest the need for greater transparency in the reporting of information necessary to determine the appropriateness and fairness of these tools prior to clinical use.

Financial Disclosures

Proprietary or commercial disclosure may be found in the Footnotes and Disclosures at the end of this article.

Rate and Predictors of Misclassification of Active Diabetic Macular Edema as Detected by an Automated Retinal Image Analysis System

INTRODUCTION

The aim of this work is to estimate the sensitivity, specificity, and misclassification rate of an automated retinal image analysis system (ARIAS) in diagnosing active diabetic macular edema (DME) and to identify factors associated with true and false positives.

METHODS

We conducted a cross-sectional study of prospectively enrolled patients with diabetes mellitus (DM) referred to a tertiary medical retina center for screening or management of DME. All patients underwent two-field fundus photography (macula- and disc-centered) with a true-color confocal camera; images were processed by EyeArt V.2.1.0 (Woodland Hills, CA, USA). Active DME was defined as the presence of intraretinal or subretinal fluid on spectral-domain optical coherence tomography (SD-OCT). Sensitivity and specificity and their 95% confidence intervals (CIs) were calculated. Variables associated with true (i.e., DME labeled as present by ARIAS + fluid on SD-OCT) and false positives (i.e., DME labeled as present by ARIAS + no fluid on SD-OCT) of active DME were explored.

RESULTS

A total of 298 eyes were included; 92 eyes (31%) had active DME. ARIAS sensitivity and specificity were 82.61% (95% CI 72.37-89.60) and 84.47% (95% CI 78.34-89.10). The misclassification rate was 16%. Factors associated with true positives included younger age (p = 0.01), shorter DM duration (p = 0.006), presence of hard exudates (p = 0.005), and microaneurysms (p = 0.002). Factors associated with false positives included longer DM duration (p = 0.01), worse diabetic retinopathy severity (p = 0.008), history of inactivated DME (p < 0.001), and presence of hard exudates (p < 0.001), microaneurysms (p < 0.001), or epiretinal membrane (p = 0.06).

CONCLUSIONS

The sensitivity of ARIAS was diminished in older patients and those without DME-related fundus lesions, while the specificity was reduced in cases with a history of inactivated DME. ARIAS performed well in screening for naïve DME but is not effective in surveillance inactivated DME.

Ocular biomarkers: useful incidental findings by deep learning algorithms in fundus photographs

BACKGROUND/OBJECTIVES

Artificial intelligence can assist with ocular image analysis for screening and diagnosis, but it is not yet capable of autonomous full-spectrum screening. Hypothetically, false-positive results may have unrealized screening potential arising from signals persisting despite training and/or ambiguous signals such as from biomarker overlap or high comorbidity. The study aimed to explore the potential to detect clinically useful incidental ocular biomarkers by screening fundus photographs of hypertensive adults using diabetic deep learning algorithms.

SUBJECTS/METHODS

Patients referred for treatment-resistant hypertension were imaged at a hospital unit in Perth, Australia, between 2016 and 2022. The same 45° colour fundus photograph selected for each of the 433 participants imaged was processed by three deep learning algorithms. Two expert retinal specialists graded all false-positive results for diabetic retinopathy in non-diabetic participants.

RESULTS

Of the 29 non-diabetic participants misclassified as positive for diabetic retinopathy, 28 (97%) had clinically useful retinal biomarkers. The models designed to screen for fewer diseases captured more incidental disease. All three algorithms showed a positive correlation between severity of hypertensive retinopathy and misclassified diabetic retinopathy.

CONCLUSIONS

The results suggest that diabetic deep learning models may be responsive to hypertensive and other clinically useful retinal biomarkers within an at-risk, hypertensive cohort. Observing that models trained for fewer diseases captured more incidental pathology increases confidence in signalling hypotheses aligned with using self-supervised learning to develop autonomous comprehensive screening. Meanwhile, non-referable and false-positive outputs of other deep learning screening models could be explored for immediate clinical use in other populations.

Preliminary Evidence of the Use of Generative AI in Health Care Clinical Services: Systematic Narrative Review

BACKGROUND

Generative artificial intelligence tools and applications (GenAI) are being increasingly used in health care. Physicians, specialists, and other providers have started primarily using GenAI as an aid or tool to gather knowledge, provide information, train, or generate suggestive dialogue between physicians and patients or between physicians and patients' families or friends. However, unless the use of GenAI is oriented to be helpful in clinical service encounters that can improve the accuracy of diagnosis, treatment, and patient outcomes, the expected potential will not be achieved. As adoption continues, it is essential to validate the effectiveness of the infusion of GenAI as an intelligent technology in service encounters to understand the gap in actual clinical service use of GenAI.

OBJECTIVE

This study synthesizes preliminary evidence on how GenAI assists, guides, and automates clinical service rendering and encounters in health care The review scope was limited to articles published in peer-reviewed medical journals.

METHODS

We screened and selected 0.38% (161/42,459) of articles published between January 1, 2020, and May 31, 2023, identified from PubMed. We followed the protocols outlined in the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) guidelines to select highly relevant studies with at least 1 element on clinical use, evaluation, and validation to provide evidence of GenAI use in clinical services. The articles were classified based on their relevance to clinical service functions or activities using the descriptive and analytical information presented in the articles.

RESULTS

Of 161 articles, 141 (87.6%) reported using GenAI to assist services through knowledge access, collation, and filtering. GenAI was used for disease detection (19/161, 11.8%), diagnosis (14/161, 8.7%), and screening processes (12/161, 7.5%) in the areas of radiology (17/161, 10.6%), cardiology (12/161, 7.5%), gastrointestinal medicine (4/161, 2.5%), and diabetes (6/161, 3.7%). The literature synthesis in this study suggests that GenAI is mainly used for diagnostic processes, improvement of diagnosis accuracy, and screening and diagnostic purposes using knowledge access. Although this solves the problem of knowledge access and may improve diagnostic accuracy, it is oriented toward higher value creation in health care.

CONCLUSIONS

GenAI informs rather than assisting or automating clinical service functions in health care. There is potential in clinical service, but it has yet to be actualized for GenAI. More clinical service-level evidence that GenAI is used to streamline some functions or provides more automated help than only information retrieval is needed. To transform health care as purported, more studies related to GenAI applications must automate and guide human-performed services and keep up with the optimism that forward-thinking health care organizations will take advantage of GenAI.

Assessing Diabetic Retinopathy Staging With AI: A Comparative Analysis Between Pseudocolor and LED Imaging

Purpose

To compare the diagnostic performance of artificial intelligence (AI)-based diabetic retinopathy (DR) staging system across pseudocolor, simulated white light (SWL), and light-emitting diode (LED) camera imaging modalities.

Methods

A cross-sectional investigation involved patients with diabetes undergoing imaging with an iCare DRSplus confocal LED camera and an Optos confocal, ultra-widefield pseudocolor camera, with and without SWL. Macula-centered and optic nerve-centered 45 × 45-degree photographs were processed using EyeArt v2.1. Human graders established the ground truth (GT) for DR severity on dilated fundus exams. Sensitivity and weighted Cohen's weighted kappa (wκ) were calculated. An ordinal generalized linear mixed model identified factors influencing accurate DR staging.

Results

The study included 362 eyes from 189 patients. The LED camera excelled in identifying sight-threatening DR stages (sensitivity = 0.83, specificity = 0.95 for proliferative DR) and had the highest agreement with the GT (wκ = 0.71). The addition of SWL to pseudocolor imaging resulted in decreased performance (sensitivity = 0.33, specificity = 0.98 for proliferative DR; wκ = 0.55). Peripheral lesions reduced the likelihood of being staged in the same or higher DR category by 80% (P < 0.001).

Conclusions

Pseudocolor and LED cameras, although proficient, demonstrated non-interchangeable performance, with the LED camera exhibiting superior accuracy in identifying advanced DR stages. These findings underscore the importance of implementing AI systems trained for ultra-widefield imaging, considering the impact of peripheral lesions on correct DR staging.

Translational Relevance

This study underscores the need for artificial intelligence-based systems specifically trained for ultra-widefield imaging in diabetic retinopathy assessment.

Diabetic Retinopathy Screening Using Smartphone-Based Fundus Photography and Deep-Learning Artificial Intelligence in the Yucatan Peninsula: A Field Study

BACKGROUND

To compare the performance of Medios (offline) and EyeArt (online) artificial intelligence (AI) algorithms for detecting diabetic retinopathy (DR) on images captured using fundus-on-smartphone photography in a remote outreach field setting.

METHODS

In June, 2019 in the Yucatan Peninsula, 248 patients, many of whom had chronic visual impairment, were screened for DR using two portable Remidio fundus-on-phone cameras, and 2130 images obtained were analyzed, retrospectively, by Medios and EyeArt. Screening performance metrics also were determined retrospectively using masked image analysis combined with clinical examination results as the reference standard.

RESULTS

A total of 129 patients were determined to have some level of DR; 119 patients had no DR. Medios was capable of evaluating every patient with a sensitivity (95% confidence intervals [CIs]) of 94% (88%-97%) and specificity of 94% (88%-98%). Owing primarily to photographer error, EyeArt evaluated 156 patients with a sensitivity of 94% (86%-98%) and specificity of 86% (77%-93%). In a head-to-head comparison of 110 patients, the sensitivities of Medios and EyeArt were 99% (93%-100%) and 95% (87%-99%). The specificities for both were 88% (73%-97%).

CONCLUSIONS

Medios and EyeArt AI algorithms demonstrated high levels of sensitivity and specificity for detecting DR when applied in this real-world field setting. Both programs should be considered in remote, large-scale DR screening campaigns where immediate results are desirable, and in the case of EyeArt, online access is possible.

Multitask Learning for Activity Detection in Neovascular Age-Related Macular Degeneration

Purpose

The purpose of this study was to provide a comparison of performance and explainability of a multitask convolutional deep neuronal network to single-task networks for activity detection in neovascular age-related macular degeneration (nAMD).

Methods

From 70 patients (46 women and 24 men) who attended the University Eye Hospital Tübingen, 3762 optical coherence tomography B-scans (right eye = 2011 and left eye = 1751) were acquired with Heidelberg Spectralis, Heidelberg, Germany. B-scans were graded by a retina specialist and an ophthalmology resident, and then used to develop a multitask deep learning model to predict disease activity in neovascular age-related macular degeneration along with the presence of sub- and intraretinal fluid. We used performance metrics for comparison to single-task networks and visualized the deep neural network (DNN)-based decision with t-distributed stochastic neighbor embedding and clinically validated saliency mapping techniques.

Results

The multitask model surpassed single-task networks in accuracy for activity detection (94.2% vs. 91.2%). The area under the curve of the receiver operating curve was 0.984 for the multitask model versus 0.974 for the single-task model. Furthermore, compared to single-task networks, visualizations via t-distributed stochastic neighbor embedding and saliency maps highlighted that multitask networks' decisions for activity detection in neovascular age-related macular degeneration were highly consistent with the presence of both sub- and intraretinal fluid.

Conclusions

Multitask learning increases the performance of neuronal networks for predicting disease activity, while providing clinicians with an easily accessible decision control, which resembles human reasoning.

Translational Relevance

By improving nAMD activity detection performance and transparency of automated decisions, multitask DNNs can support the translation of machine learning research into clinical decision support systems for nAMD activity detection.

Artificial Intelligence for Diabetic Retinopathy Screening Using Color Retinal Photographs: From Development to Deployment

Diabetic retinopathy (DR), a leading cause of preventable blindness, is expected to remain a growing health burden worldwide. Screening to detect early sight-threatening lesions of DR can reduce the burden of vision loss; nevertheless, the process requires intensive manual labor and extensive resources to accommodate the increasing number of patients with diabetes. Artificial intelligence (AI) has been shown to be an effective tool which can potentially lower the burden of screening DR and vision loss. In this article, we review the use of AI for DR screening on color retinal photographs in different phases of application, ranging from development to deployment. Early studies of machine learning (ML)-based algorithms using feature extraction to detect DR achieved a high sensitivity but relatively lower specificity. Robust sensitivity and specificity were achieved with the application of deep learning (DL), although ML is still used in some tasks. Public datasets were utilized in retrospective validations of the developmental phases in most algorithms, which require a large number of photographs. Large prospective clinical validation studies led to the approval of DL for autonomous screening of DR although the semi-autonomous approach may be preferable in some real-world settings. There have been few reports on real-world implementations of DL for DR screening. It is possible that AI may improve some real-world indicators for eye care in DR, such as increased screening uptake and referral adherence, but this has not been proven. The challenges in deployment may include workflow issues, such as mydriasis to lower ungradable cases; technical issues, such as integration into electronic health record systems and integration into existing camera systems; ethical issues, such as data privacy and security; acceptance of personnel and patients; and health-economic issues, such as the need to conduct health economic evaluations of using AI in the context of the country. The deployment of AI for DR screening should follow the governance model for AI in healthcare which outlines four main components: fairness, transparency, trustworthiness, and accountability.

Quantitative approaches in multimodal fundus imaging: State of the art and future perspectives

When it first appeared, multimodal fundus imaging revolutionized the diagnostic workup and provided extremely useful new insights into the pathogenesis of fundus diseases. The recent addition of quantitative approaches has further expanded the amount of information that can be obtained. In spite of the growing interest in advanced quantitative metrics, the scientific community has not reached a stable consensus on repeatable, standardized quantitative techniques to process and analyze the images. Furthermore, imaging artifacts may considerably affect the processing and interpretation of quantitative data, potentially affecting their reliability. The aim of this survey is to provide a comprehensive summary of the main multimodal imaging techniques, covering their limitations as well as their strengths. We also offer a thorough analysis of current quantitative imaging metrics, looking into their technical features, limitations, and interpretation. In addition, we describe the main imaging artifacts and their potential impact on imaging quality and reliability. The prospect of increasing reliance on artificial intelligence-based analyses suggests there is a need to develop more sophisticated quantitative metrics and to improve imaging technologies, incorporating clear, standardized, post-processing procedures. These measures are becoming urgent if these analyses are to cross the threshold from a research context to real-life clinical practice.

Comparison of peripheral extension, acquisition time, and image chromaticity of Optos, Clarus, and EIDON systems

PURPOSE

To evaluate differences in acquisition time, peripheral extension, and chromaticity between 3 different commercialized ultra-wide-field (UWF) fundus cameras.

METHODS

Patients were prospectively enrolled from 07/2021 to 11/2021. Patients underwent fundus photography with the following scanning protocols: (1) single shot with Silverstone (Optos, California), two-shot montage with Clarus 500 (Carl Zeiss, Dublin, CA), and three-shot montage with iCare EIDON FA with UWF module (CenterVue Spa, a company of iCare Finland Oy; Vantaa, Finland). Acquisition time was calculated as the interval between the beginning and the end of the acquisition. Peripheral extension was quantified as the average ratio between the total retinal pixel area and the optic nerve head (ONH) pixel area. The average chromaticity of all pixels in the red-green-blue (RGB) space was calculated.

RESULTS

Twenty-three eyes of 13 prospectively enrolled healthy controls were included in the study. Optos Silverstone had a higher total retina area/ONH area ratio (509.1 [480.9;559.3]) compared to Zeiss Clarus (442.0 [431.9;510.5], p = 0.02) and iCare EIDON (369.7 [345.3;387.8], p < 0.0001). Silverstone demonstrated the shortest acquisition time (median [interquartile range]: 32 [20;58.5] s) compared to Zeiss Clarus (42 [28.5;53.5] s, p = 0.6733) and iCare EIDON (72 [68.5;78] s, p = 0.0003). iCare EIDON demonstrated the lowest variability of acquisition time (9.5 s), compared to Zeiss Clarus (25 s) and Optos Silverstone (38.5 s). A statistically significant difference was found in the RGB distribution between each of the 3 devices (p < 0.001). iCare EIDON demonstrated an average barycenter position (RGB = [0.412, 0.314, 0.275]) that represented the best color balance of the image. Zeiss Clarus had a noticeable red shift at the expense of the blue and green channels (RGB = [0.515, 0.294, 0.191]). Optos Silverstone showed an absence of the blue channel (RGB = [0.621, 0.372, 0.007]) which results in a distortion of the color of the image.

CONCLUSION

Optos Silverstone and Zeiss Clarus required less time than iCare EIDON to acquire a comparable size image and captured larger areas of the retina than iCare EIDON. iCare EIDON provided more color-balanced retinal images with greater richness of color content than the other two devices.

Protocol for a systematic review and meta-analysis of the diagnostic accuracy of artificial intelligence for grading of ophthalmology imaging modalities

BACKGROUND

With the rise of artificial intelligence (AI) in ophthalmology, the need to define its diagnostic accuracy is increasingly important. The review aims to elucidate the diagnostic accuracy of AI algorithms in screening for all ophthalmic conditions in patient care settings that involve digital imaging modalities, using the reference standard of human graders.

METHODS

This is a systematic review and meta-analysis. A literature search will be conducted on Ovid MEDLINE, Ovid EMBASE, and Wiley Cochrane CENTRAL from January 1, 2000, to December 20, 2021. Studies will be selected via screening the titles and abstracts, followed by full-text screening. Articles that compare the results of AI-graded ophthalmic images with results from human graders as a reference standard will be included; articles that do not will be excluded. The systematic review software DistillerSR will be used to automate part of the screening process as an adjunct to human reviewers. After the full-text screening, data will be extracted from each study via the categories of study characteristics, patient information, AI methods, intervention, and outcomes. Risk of bias will be scored using Quality Assessment of Diagnostic Accuracy Studies (QUADAS-2) by two trained independent reviewers. Disagreements at any step will be addressed by a third adjudicator. The study results will include summary receiver operating characteristic (sROC) curve plots as well as pooled sensitivity and specificity of artificial intelligence for detection of any ophthalmic conditions based on imaging modalities compared to the reference standard. Statistics will be calculated in the R statistical software.

DISCUSSION

This study will provide novel insights into the diagnostic accuracy of AI in new domains of ophthalmology that have not been previously studied. The protocol also outlines the use of an AI-based software to assist in article screening, which may serve as a reference for improving the efficiency and accuracy of future large systematic reviews.

TRIAL REGISTRATION

PROSPERO, CRD42021274441.

Comparison between Deep-Learning-Based Ultra-Wide-Field Fundus Imaging and True-Colour Confocal Scanning for Diagnosing Glaucoma

In this retrospective, comparative study, we evaluated and compared the performance of two confocal imaging modalities in detecting glaucoma based on a deep learning (DL) classifier: ultra-wide-field (UWF) fundus imaging and true-colour confocal scanning. A total of 777 eyes, including 273 normal control eyes and 504 glaucomatous eyes, were tested. A convolutional neural network was used for each true-colour confocal scan (Eidon AF™, CenterVue, Padova, Italy) and UWF fundus image (Optomap™, Optos PLC, Dunfermline, UK) to detect glaucoma. The diagnostic model was trained using 545 training and 232 test images. The presence of glaucoma was determined, and the accuracy and area under the receiver operating characteristic curve (AUC) metrics were assessed for diagnostic power comparison. DL-based UWF fundus imaging achieved an AUC of 0.904 (95% confidence interval (CI): 0.861−0.937) and accuracy of 83.62%. In contrast, DL-based true-colour confocal scanning achieved an AUC of 0.868 (95% CI: 0.824−0.912) and accuracy of 81.46%. Both DL-based confocal imaging modalities showed no significant differences in their ability to diagnose glaucoma (p = 0.135) and were comparable to the traditional optical coherence tomography parameter-based methods (all p > 0.005). Therefore, using a DL-based algorithm on true-colour confocal scanning and UWF fundus imaging, we confirmed that both confocal fundus imaging techniques had high value in diagnosing glaucoma.

Cost-effectiveness of Artificial Intelligence as a Decision-Support System Applied to the Detection and Grading of Melanoma, Dental Caries, and Diabetic Retinopathy

OBJECTIVE

To assess the cost-effectiveness of artificial intelligence (AI) for supporting clinicians in detecting and grading diseases in dermatology, dentistry, and ophthalmology.

IMPORTANCE

AI has been referred to as a facilitator for more precise, personalized, and safer health care, and AI algorithms have been reported to have diagnostic accuracies at or above the average physician in dermatology, dentistry, and ophthalmology.

DESIGN, SETTING, AND PARTICIPANTS

This economic evaluation analyzed data from 3 Markov models used in previous cost-effectiveness studies that were adapted to compare AI vs standard of care to detect melanoma on skin photographs, dental caries on radiographs, and diabetic retinopathy on retina fundus imaging. The general US and German population aged 50 and 12 years, respectively, as well as individuals with diabetes in Brazil aged 40 years were modeled over their lifetime. Monte Carlo microsimulations and sensitivity analyses were used to capture lifetime efficacy and costs. An annual cycle length was chosen. Data were analyzed between February 2021 and August 2021.

EXPOSURE

AI vs standard of care.

MAIN OUTCOMES AND MEASURES

Association of AI with tooth retention-years for dentistry and quality-adjusted life-years (QALYs) for individuals in dermatology and ophthalmology; diagnostic costs.

RESULTS

In 1000 microsimulations with 1000 random samples, AI as a diagnostic-support system showed limited cost-savings and gains in tooth retention-years and QALYs. In dermatology, AI showed mean costs of $750 (95% CI, $608-$970) and was associated with 86.5 QALYs (95% CI, 84.9-87.9 QALYs), while the control showed higher costs $759 (95% CI, $618-$970) with similar QALY outcome. In dentistry, AI accumulated costs of €320 (95% CI, €299-€341) (purchasing power parity [PPP] conversion, $429 [95% CI, $400-$458]) with 62.4 years per tooth retention (95% CI, 60.7-65.1 years). The control was associated with higher cost, €342 (95% CI, €318-€368) (PPP, $458; 95% CI, $426-$493) and fewer tooth retention-years (60.9 years; 95% CI, 60.5-63.1 years). In ophthalmology, AI accrued costs of R $1321 (95% CI, R $1283-R $1364) (PPP, $559; 95% CI, $543-$577) at 8.4 QALYs (95% CI, 8.0-8.7 QALYs), while the control was less expensive (R $1260; 95% CI, R $1222-R $1303) (PPP, $533; 95% CI, $517-$551) and associated with similar QALYs. Dominance in favor of AI was dependent on small differences in the fee paid for the service and the treatment assumed after diagnosis. The fee paid for AI was a factor in patient preferences in cost-effectiveness between strategies.

CONCLUSIONS AND RELEVANCE

The findings of this study suggest that marginal improvements in diagnostic accuracy when using AI may translate into a marginal improvement in outcomes. The current evidence supporting AI as decision support from a cost-effectiveness perspective is limited; AI should be evaluated on a case-specific basis to capture not only differences in costs and payment mechanisms but also treatment after diagnosis.

Imaging diabetic retinal disease: clinical imaging requirements

Diabetic retinopathy (DR) is a sight-threatening complication of diabetes mellitus (DM) and it contributes substantially to the burden of disease globally. During the last decades, the development of multiple imaging modalities to evaluate DR, combined with emerging treatment possibilities, has led to the implementation of large-scale screening programmes resulting in improved prevention of vision loss. However, not all patients are able to participate in such programmes and not all are at equal risk of DR development and progression. In this review, we discuss the relevance of the currently available imaging modalities for the evaluation of DR: colour fundus photography (CFP), ultrawide-field photography (UWFP), fundus fluorescein angiography (FFA), optical coherence tomography (OCT), OCT angiography (OCTA) and functional testing. Furthermore, we suggest where a particular imaging technique of DR may aid the evaluation of the disease in different clinical settings. Combining information from various imaging modalities may enable the design of more personalized care including the initiation of treatment and understanding the progression of disease more adequately.

In-Person Verification of Deep Learning Algorithm for Diabetic Retinopathy Screening Using Different Techniques Across Fundus Image Devices

Purpose

To evaluate the clinical performance of an automated diabetic retinopathy (DR) screening model to detect referable cases at Siriraj Hospital, Bangkok, Thailand.

Methods

A retrospective review of two sets of fundus photographs (Eidon and Nidek) was undertaken. The images were classified by DR staging prior to the development of a DR screening model. In a prospective cross-sectional enrollment of patients with diabetes, automated detection of referable DR was compared with the results of the gold standard, a dilated fundus examination.

Results

The study analyzed 2533 Nidek fundus images and 1989 Eidon images. The sensitivities calculated for the Nidek and Eidon images were 0.93 and 0.88 and the specificities were 0.91 and 0.85, respectively. In a clinical verification phase using 982 Nidek and 674 Eidon photographs, the calculated sensitivities and specificities were 0.86 and 0.92 for Nidek along with 0.92 and 0.84 for Eidon, respectively. The 60°-field images from the Eidon yielded a more desirable performance in differentiating referable DR than did the corresponding images from the Nidek.

Conclusions

A conventional fundus examination requires intense healthcare resources. It is time consuming and possibly leads to unavoidable human errors. The deep learning algorithm for the detection of referable DR exhibited a favorable performance and is a promising alternative for DR screening. However, variations in the color and pixels of photographs can cause differences in sensitivity and specificity. The image angle and poor quality of fundus photographs were the main limitations of the automated method.

Translational Relevance

The deep learning algorithm, developed from basic research of image processing, was applied to detect referable DR in a real-word clinical care setting.

Recent Advances and Clinical Application of Color Scanning Laser Ophthalmoscope

Scanning laser ophthalmoscopes (SLOs) have been available since the early 1990s, but they were not commonly used because their advantages were not enough to replace conventional color fundus photography. In recent years, color SLOs have improved significantly, and the colored SLO images are obtained by combining multiple SLO images taken by lasers of different wavelengths. A combination of these images of different lasers can create an image that is close to that of the real ocular fundus. One advantage of the advanced SLOs is that they can obtain images with a wider view of the ocular fundus while maintaining a high resolution even through non-dilated eyes. The current SLOs are superior to the conventional fundus photography in their ability to image abnormal alterations of the retina and choroid. Thus, the purpose of this review was to present the characteristics of the current color SLOs and to show how that can help in the diagnosis and the following of changes after treatments. To accomplish these goals, we will present our findings in patients with different types of retinochoroidal disorders.

Detection of Diabetic Retinopathy from Ultra-Widefield Scanning Laser Ophthalmoscope Images: A Multicenter Deep Learning Analysis

PURPOSE

To develop a deep learning (DL) system that can detect referable diabetic retinopathy (RDR) and vision-threatening diabetic retinopathy (VTDR) from images obtained on ultra-widefield scanning laser ophthalmoscope (UWF-SLO).

DESIGN

Observational, cross-sectional study.

PARTICIPANTS

A total of 9392 UWF-SLO images of 1903 eyes from 1022 subjects with diabetes from Hong Kong, the United Kingdom, India, and Argentina.

METHODS

All images were labeled according to the presence or absence of RDR and the presence or absence of VTDR. Labeling was performed by retina specialists from fundus examination, according to the International Clinical Diabetic Retinopathy Disease Severity Scale. Three convolutional neural networks (ResNet50) were trained with a transfer-learning procedure for assessing gradability and identifying VTDR and RDR. External validation was performed on 4 datasets spanning different geographical regions.

MAIN OUTCOME MEASURES

Area under the receiver operating characteristic curve (AUROC); area under the precision-recall curve (AUPRC); sensitivity, specificity, and accuracy of the DL system in gradability assessment; and detection of RDR and VTDR.

RESULTS

For gradability assessment, the system achieved an AUROC of 0.923 (95% confidence interval [CI], 0.892-0.947), sensitivity of 86.5% (95% CI, 77.6-92.8), and specificity of 82.1% (95% CI, 77.3-86.2) for the primary validation dataset, and >0.82 AUROCs, >79.6% sensitivity, and >70.4% specificity for the geographical external validation datasets. For detecting RDR and VTDR, the AUROCs were 0.981 (95% CI, 0.977-0.984) and 0.966 (95% CI, 0.961-0.971), with sensitivities of 94.9% (95% CI, 92.3-97.9) and 87.2% (95% CI, 81.5-91.6), specificities of 95.1% (95% CI, 90.6-97.9) and 95.8% (95% CI, 93.3-97.6), and positive predictive values (PPVs) of 98.0% (95% CI, 96.1-99.0) and 91.1% (95% CI, 86.3-94.3) for the primary validation dataset, respectively. The AUROCs and accuracies for detecting both RDR and VTDR were >0.9% and >80%, respectively, for the geographical external validation datasets. The AUPRCs were >0.9, and sensitivities, specificities, and PPVs were >80% for the geographical external validation datasets for RDR and VTDR detection.

CONCLUSIONS

The excellent performance achieved with this DL system for image quality assessment and detection of RDR and VTDR in UWF-SLO images highlights its potential as an efficient and effective diabetic retinopathy screening tool.

Digital Ophthalmology in the UK - Diabetic Retinopathy Screening and Virtual Glaucoma Clinics in the National Health Service

The customary doctor and patient interactions are currently undergoing significant changes through technological advances in imaging and data processing and the need for reducing person-to person contacts during the COVID-19 crisis. There is a trend away from face-to-face examinations to virtual assessments and decision making. Ophthalmology is particularly amenable to such changes, as a high proportion of clinical decisions are based on routine tests and imaging results, which can be assessed remotely. The uptake of digital ophthalmology varies significantly between countries. Due to financial constraints within the National Health Service, specialized ophthalmology units in the UK have been early adopters of digital technology. For more than a decade, patients have been managed remotely in the diabetic retinopathy screening service and virtual glaucoma clinics. We describe the day-to-day running of such services and the doctor and patient experiences with digital ophthalmology in daily practice.

Evidence Based Prediction and Progression Monitoring on Retinal Images from Three Nations

Purpose

The aim of this work is to demonstrate how a retinal image analysis system, DAPHNE, supports the optimization of diabetic retinopathy (DR) screening programs for grading color fundus photography.

Method

Retinal image sets, graded by trained and certified human graders, were acquired from Saudi Arabia, China, and Kenya. Each image was subsequently analyzed by the DAPHNE automated software. The sensitivity, specificity, and positive and negative predictive values for the detection of referable DR or diabetic macular edema were evaluated, taking human grading or clinical assessment outcomes to be the gold standard. The automated software's ability to identify co-pathology and to correctly label DR lesions was also assessed.

Results

In all three datasets the agreement between the automated software and human grading was between 0.84 to 0.88. Sensitivity did not vary significantly between populations (94.28%–97.1%) with specificity ranging between 90.33% to 92.12%. There were excellent negative predictive values above 93% in all image sets. The software was able to monitor DR progression between baseline and follow-up images with the changes visualized. No cases of proliferative DR or DME were missed in the referable recommendations.

Conclusions

The DAPHNE automated software demonstrated its ability not only to grade images but also to reliably monitor and visualize progression. Therefore it has the potential to assist timely image analysis in patients with diabetes in varied populations and also help to discover subtle signs of sight-threatening disease onset.

Translational Relevance

This article takes research on machine vision and evaluates its readiness for clinical use.