Automated Diabetic Retinopathy Image Assessment Software: Diagnostic Accuracy and Cost-Effectiveness Compared with Human Graders

Cost-effectiveness and cost-utility of community-based blinding fundus diseases screening with artificial intelligence: A modelling study from Shanghai, China

BACKGROUND

With application of artificial intelligence (AI) in the disease screening, process reengineering occurred simultaneously. Whether process reengineering deserves special emphasis in AI implementation in the community-based blinding fundus diseases screening is not clear.

METHOD

Cost-effectiveness and cost-utility analyses were performed employing decision-analytic Markov models. A hypothetical cohort of community residents was followed in the model over a period of 30 1-year Markov cycles, starting from the age of 60. The simulated cohort was based on work data of the Shanghai Digital Eye Disease Screening program (SDEDS). Three scenarios were compared: centralized screening with manual grading-based telemedicine systems (Scenario 1), centralized screening with an AI-assisted screening system (Scenario 2), and process reengineered screening with an AI-assisted screening system (Scenario 3). The main outcomes were incremental cost-effectiveness ratio (ICER) and incremental cost-utility ratio (ICUR).

RESULTS

Compared with Scenario 1, Scenario 2 results in incremental 187.03 years of blindness avoided and incremental 106.78 QALYs at an additional cost of $ 490010.62 per 10,000 people screened, with an ICER of $2619.98 per year of blindness avoided and an ICUR of $4589.13 per QALY. Compared with Scenario 1, Scenario 3 results in incremental 187.03 years of blindness avoided and incremental 106.78 QALYs at an additional cost of $242313.23 per 10,000 people screened, with an ICER of $1295.60 per year of blindness avoided and an ICUR of $2269.35 per QALY. Although Scenario 2 and 3 could be considered cost-effective, the screening cost of Scenario 3 was 27.6 % and the total cost was 1.1 % lower, with the same expected effectiveness and utility. The probabilistic sensitivity analyses show that Scenario 3 dominated 69.1 % and 70.3 % of simulations under one and three times the local GDP per capita thresholds.

CONCLUSIONS

AI can improve the cost-effectiveness and cost-utility of screenings, especially when process reengineering is performed. Therefore, process reengineering is strongly recommended when AI is implemented.

The Use of Artificial Intelligence for Estimating Anterior Chamber Depth from Slit-Lamp Images Developed Using Anterior-Segment Optical Coherence Tomography

Primary angle closure glaucoma (PACG) is a major cause of visual impairment, particularly in Asia. Although effective screening tools are necessary, the current gold standard is complex and time-consuming, requiring extensive expertise. Artificial intelligence has introduced new opportunities for innovation in ophthalmic imaging. Anterior chamber depth (ACD) is a key risk factor for angle closure and has been suggested as a quick screening parameter for PACG. This study aims to develop an AI algorithm to quantitatively predict ACD from anterior segment photographs captured using a portable smartphone slit-lamp microscope. We retrospectively collected 204,639 frames from 1586 eyes, with ACD values obtained by anterior-segment OCT. We developed two models, (Model 1) diagnosable frame extraction and (Model 2) ACD estimation, using SWSL ResNet as the machine learning model. Model 1 achieved an accuracy of 0.994. Model 2 achieved an MAE of 0.093 ± 0.082 mm, an MSE of 0.123 ± 0.170 mm, and a correlation of R = 0.953. Furthermore, our model's estimation of the risk for angle closure showed a sensitivity of 0.943, specificity of 0.902, and an area under the curve (AUC) of 0.923 (95%CI: 0.878-0.968). We successfully developed a high-performance ACD estimation model, laying the groundwork for predicting other quantitative measurements relevant to PACG screening.

Artificial Intelligence-Based Screening System for Diabetic Retinopathy in Primary Care

BACKGROUND

This study aimed to test an artificial intelligence-based reading system (AIRS) capable of reading retinographies of type 2 diabetic (T2DM) patients and a predictive algorithm (DRPA) that predicts the risk of each patient with T2DM of developing diabetic retinopathy (DR).

METHODS

We tested the ability of the AIRS to read and classify 15,297 retinal photographs from our database of diabetics and 1200 retinal images taken with Messidor-2 into the different DR categories. We tested the DRPA in a sample of 40,129 T2DM patients. The results obtained by the AIRS and the DRPA were then compared with those provided by four retina specialists regarding sensitivity (S), specificity (SP), positive predictive value (PPV), negative predictive value (NPV), accuracy (ACC), and area under the curve (AUC).

RESULTS

The results of testing the AIRS for identifying referral DR (RDR) in our database were ACC = 98.6, S = 96.7, SP = 99.8, PPV = 99.0, NPV = 98.0, and AUC = 0.958, and in Messidor-2 were ACC = 96.78%, S = 94.64%, SP = 99.14%, PPV = 90.54%, NPV = 99.53%, and AUC = 0.918. The results of our DRPA when predicting the presence of any type of DR were ACC = 0.97, S = 0.89, SP = 0.98, PPV = 0.79, NPV = 0.98, and AUC = 0.92.

CONCLUSIONS

The AIRS performed well when reading and classifying the retinographies of T2DM patients with RDR. The DRPA performed well in predicting the absence of DR based on some clinical variables.

Ocular biomarkers: useful incidental findings by deep learning algorithms in fundus photographs

BACKGROUND/OBJECTIVES

Artificial intelligence can assist with ocular image analysis for screening and diagnosis, but it is not yet capable of autonomous full-spectrum screening. Hypothetically, false-positive results may have unrealized screening potential arising from signals persisting despite training and/or ambiguous signals such as from biomarker overlap or high comorbidity. The study aimed to explore the potential to detect clinically useful incidental ocular biomarkers by screening fundus photographs of hypertensive adults using diabetic deep learning algorithms.

SUBJECTS/METHODS

Patients referred for treatment-resistant hypertension were imaged at a hospital unit in Perth, Australia, between 2016 and 2022. The same 45° colour fundus photograph selected for each of the 433 participants imaged was processed by three deep learning algorithms. Two expert retinal specialists graded all false-positive results for diabetic retinopathy in non-diabetic participants.

RESULTS

Of the 29 non-diabetic participants misclassified as positive for diabetic retinopathy, 28 (97%) had clinically useful retinal biomarkers. The models designed to screen for fewer diseases captured more incidental disease. All three algorithms showed a positive correlation between severity of hypertensive retinopathy and misclassified diabetic retinopathy.

CONCLUSIONS

The results suggest that diabetic deep learning models may be responsive to hypertensive and other clinically useful retinal biomarkers within an at-risk, hypertensive cohort. Observing that models trained for fewer diseases captured more incidental pathology increases confidence in signalling hypotheses aligned with using self-supervised learning to develop autonomous comprehensive screening. Meanwhile, non-referable and false-positive outputs of other deep learning screening models could be explored for immediate clinical use in other populations.

Advancing healthcare with artificial intelligence: diagnostic accuracy of machine learning algorithm in diagnosis of diabetic retinopathy in the Brazilian population

BACKGROUND

In healthcare systems in general, access to diabetic retinopathy (DR) screening is limited. Artificial intelligence has the potential to increase care delivery. Therefore, we trained and evaluated the diagnostic accuracy of a machine learning algorithm for automated detection of DR.

METHODS

We included color fundus photographs from individuals from 4 databases (primary and specialized care settings), excluding uninterpretable images. The datasets consist of images from Brazilian patients, which differs from previous work. This modification allows for a more tailored application of the model to Brazilian patients, ensuring that the nuances and characteristics of this specific population are adequately captured. The sample was fractionated in training (70%) and testing (30%) samples. A convolutional neural network was trained for image classification. The reference test was the combined decision from three ophthalmologists. The sensitivity, specificity, and area under the ROC curve of the algorithm for detecting referable DR (moderate non-proliferative DR; severe non-proliferative DR; proliferative DR and/or clinically significant macular edema) were estimated.

RESULTS

A total of 15,816 images (4590 patients) were included. The overall prevalence of any degree of DR was 26.5%. Compared with human evaluators (manual method of diagnosing DR performed by an ophthalmologist), the deep learning algorithm achieved an area under the ROC curve of 0.98 (95% CI 0.97-0.98), with a specificity of 94.6% (95% CI 93.8-95.3) and a sensitivity of 93.5% (95% CI 92.2-94.9) at the point of greatest efficiency to detect referable DR.

CONCLUSIONS

A large database showed that this deep learning algorithm was accurate in detecting referable DR. This finding aids to universal healthcare systems like Brazil, optimizing screening processes and can serve as a tool for improving DR screening, making it more agile and expanding care access.

Advances in multimodal imaging for diagnosis of pigmented ocular fundus lesions

Pigmented ocular fundus lesions can range from benign to malignant. While observation is reasonable for asymptomatic benign lesions, early recognition of tumours that are vision or life threatening is critical for long-term prognosis. With recent advances and increased accessibility of multimodal imaging, it is important that providers understand how to best use these tools to detect tumours that require early referral to subspecialty centres. This review aims to provide an overview of pigmented ocular fundus lesions and their defining characteristics using multimodal imaging. We cover the spectrum of pigmented ocular fundus lesions, including freckle and focal aggregates of normal or near-normal uveal melanocytes, retinal pigment epithelium (RPE) hyperplasia, congenital hypertrophy of the RPE, RPE hamartoma associated with familial adenomatous polyposis, congenital simple hamartoma of the RPE, combined hamartoma of the retina and RPE (congenital hypertrophy of the RPE), choroidal nevus, melanocytosis, melanocytoma, melanoma, adenoma, and RPE adenocarcinoma. We describe key diagnostic features using multimodal imaging modalities of ultra-widefield fundus photography, fundus autofluorescence, optical coherence tomography (OCT), enhanced-depth imaging OCT, ultrasonography, fluorescein angiography, indocyanine green angiography, and OCT angiography (OCTA), with particular attention to diagnostic features that could be missed on fundus examination alone. Finally, we review what is on the horizon, including applications of artificial intelligence. Through skilled application of current and emerging imaging technologies, earlier detection of sight- and life-threatening melanocytic ocular fundus tumours can lead to improved patient prognosis.

A systematic review of economic evaluation of artificial intelligence-based screening for eye diseases: From possibility to reality

Artificial Intelligence (AI) has become a focus of research in the rapidly evolving field of ophthalmology. Nevertheless, there is a lack of systematic studies on the health economics of AI in this field. We examine studies from the PubMed, Google Scholar, and Web of Science databases that employed quantitative analysis, retrieved up to July 2023. Most of the studies indicate that AI leads to cost savings and improved efficiency in ophthalmology. On the other hand, some studies suggest that using AI in healthcare may raise costs for patients, especially when taking into account factors such as labor costs, infrastructure, and patient adherence. Future research should cover a wider range of ophthalmic diseases beyond common eye conditions. Moreover, conducting extensive health economic research, designed to collect data relevant to its own context, is imperative.

Diagnostic Accuracy of Artificial Intelligence-Based Automated Diabetic Retinopathy Screening in Real-World Settings: A Systematic Review and Meta-Analysis

PURPOSE

To evaluate the diagnostic accuracy of artificial intelligence (AI)-based automated diabetic retinopathy (DR) screening in real-world settings.

DESIGN

Systematic review and meta-analysis METHODS: We conducted a systematic review of relevant literature from January 2012 to August 2022 using databases including PubMed, Scopus and Web of Science. The quality of studies was evaluated using Quality Assessment for Diagnostic Accuracy Studies 2 (QUADAS-2) checklist. We calculated pooled accuracy, sensitivity, specificity, and diagnostic odds ratio (DOR) as summary measures. The study protocol was registered with the International Prospective Register of Systematic Reviews (PROSPERO - CRD42022367034).

RESULTS

We included 34 studies which utilized AI algorithms for diagnosing DR based on real-world fundus images. Quality assessment of these studies indicated a low risk of bias and low applicability concern. Among gradable images, the overall pooled accuracy, sensitivity, specificity, and DOR were 81%, 94% (95% CI: 92.0-96.0), 89% (95% CI: 85.0-92.0) and 128 (95% CI: 80-204) respectively. Sub-group analysis showed that, when acceptable quality imaging could be obtained, non-mydriatic fundus images had a better DOR of 143 (95% CI: 82-251) and studies using 2 field images had a better DOR of 161 (95% CI 74-347). Our meta-regression analysis revealed a statistically significant association between DOR and variables such as the income status, and the type of fundus camera.

CONCLUSION

Our findings indicate that AI algorithms have acceptable performance in screening for DR using fundus images compared to human graders. Implementing a fundus camera with AI-based software has the potential to assist ophthalmologists in reducing their workload and improving the accuracy of DR diagnosis.

Automated Identification of Different Severity Levels of Diabetic Retinopathy Using a Handheld Fundus Camera and Single-Image Protocol

Purpose

To evaluate the performance of artificial intelligence (AI) systems embedded in a mobile, handheld retinal camera, with a single retinal image protocol, in detecting both diabetic retinopathy (DR) and more-than-mild diabetic retinopathy (mtmDR).

Design

Multicenter cross-sectional diagnostic study, conducted at 3 diabetes care and eye care facilities.

Participants

A total of 327 individuals with diabetes mellitus (type 1 or type 2) underwent a retinal imaging protocol enabling expert reading and automated analysis.

Methods

Participants underwent fundus photographs using a portable retinal camera (Phelcom Eyer). The captured images were automatically analyzed by deep learning algorithms retinal alteration score (RAS) and diabetic retinopathy alteration score (DRAS), consisting of convolutional neural networks trained on EyePACS data sets and fine-tuned using data sets of portable device fundus images. The ground truth was the classification of DR corresponding to adjudicated expert reading, performed by 3 certified ophthalmologists.

Main Outcome Measures

Primary outcome measures included the sensitivity and specificity of the AI system in detecting DR and/or mtmDR using a single-field, macula-centered fundus photograph for each eye, compared with a rigorous clinical reference standard comprising the reading center grading of 2-field imaging protocol using the International Classification of Diabetic Retinopathy severity scale.

Results

Of 327 analyzed patients (mean age, 57.0 ± 16.8 years; mean diabetes duration, 16.3 ± 9.7 years), 307 completed the study protocol. Sensitivity and specificity of the AI system were high in detecting any DR with DRAS (sensitivity, 90.48% [95% confidence interval (CI), 84.99%-94.46%]; specificity, 90.65% [95% CI, 84.54%-94.93%]) and mtmDR with the combination of RAS and DRAS (sensitivity, 90.23% [95% CI, 83.87%-94.69%]; specificity, 85.06% [95% CI, 78.88%-90.00%]). The area under the receiver operating characteristic curve was 0.95 for any DR and 0.89 for mtmDR.

Conclusions

This study showed a high accuracy for the detection of DR in different levels of severity with a single retinal photo per eye in an all-in-one solution, composed of a portable retinal camera powered by AI. Such a strategy holds great potential for increasing coverage rates of screening programs, contributing to prevention of avoidable blindness.

Financial Disclosures

F.K.M. is a medical consultant for Phelcom Technologies. J.A.S. is Chief Executive Officer and proprietary of Phelcom Technologies. D.L. is Chief Technology Officer and proprietary of Phelcom Technologies. P.V.P. is an employee at Phelcom Technologies.

Rate and Predictors of Misclassification of Active Diabetic Macular Edema as Detected by an Automated Retinal Image Analysis System

INTRODUCTION

The aim of this work is to estimate the sensitivity, specificity, and misclassification rate of an automated retinal image analysis system (ARIAS) in diagnosing active diabetic macular edema (DME) and to identify factors associated with true and false positives.

METHODS

We conducted a cross-sectional study of prospectively enrolled patients with diabetes mellitus (DM) referred to a tertiary medical retina center for screening or management of DME. All patients underwent two-field fundus photography (macula- and disc-centered) with a true-color confocal camera; images were processed by EyeArt V.2.1.0 (Woodland Hills, CA, USA). Active DME was defined as the presence of intraretinal or subretinal fluid on spectral-domain optical coherence tomography (SD-OCT). Sensitivity and specificity and their 95% confidence intervals (CIs) were calculated. Variables associated with true (i.e., DME labeled as present by ARIAS + fluid on SD-OCT) and false positives (i.e., DME labeled as present by ARIAS + no fluid on SD-OCT) of active DME were explored.

RESULTS

A total of 298 eyes were included; 92 eyes (31%) had active DME. ARIAS sensitivity and specificity were 82.61% (95% CI 72.37-89.60) and 84.47% (95% CI 78.34-89.10). The misclassification rate was 16%. Factors associated with true positives included younger age (p = 0.01), shorter DM duration (p = 0.006), presence of hard exudates (p = 0.005), and microaneurysms (p = 0.002). Factors associated with false positives included longer DM duration (p = 0.01), worse diabetic retinopathy severity (p = 0.008), history of inactivated DME (p < 0.001), and presence of hard exudates (p < 0.001), microaneurysms (p < 0.001), or epiretinal membrane (p = 0.06).

CONCLUSIONS

The sensitivity of ARIAS was diminished in older patients and those without DME-related fundus lesions, while the specificity was reduced in cases with a history of inactivated DME. ARIAS performed well in screening for naïve DME but is not effective in surveillance inactivated DME.

A pilot cost-analysis study comparing AI-based EyeArt® and ophthalmologist assessment of diabetic retinopathy in minority women in Oslo, Norway

BACKGROUND

Diabetic retinopathy (DR) is the leading cause of adult blindness in the working age population worldwide, which can be prevented by early detection. Regular eye examinations are recommended and crucial for detecting sight-threatening DR. Use of artificial intelligence (AI) to lessen the burden on the healthcare system is needed.

PURPOSE

To perform a pilot cost-analysis study for detecting DR in a cohort of minority women with DM in Oslo, Norway, that have the highest prevalence of diabetes mellitus (DM) in the country, using both manual (ophthalmologist) and autonomous (AI) grading. This is the first study in Norway, as far as we know, that uses AI in DR- grading of retinal images.

METHODS

On Minority Women's Day, November 1, 2017, in Oslo, Norway, 33 patients (66 eyes) over 18 years of age diagnosed with DM (T1D and T2D) were screened. The Eidon - True Color Confocal Scanner (CenterVue, United States) was used for retinal imaging and graded for DR after screening had been completed, by an ophthalmologist and automatically, using EyeArt Automated DR Detection System, version 2.1.0 (EyeArt, EyeNuk, CA, USA). The gradings were based on the International Clinical Diabetic Retinopathy (ICDR) severity scale [1] detecting the presence or absence of referable DR. Cost-minimization analyses were performed for both grading methods.

RESULTS

33 women (64 eyes) were eligible for the analysis. A very good inter-rater agreement was found: 0.98 (P < 0.01), between the human and AI-based EyeArt grading system for detecting DR. The prevalence of DR was 18.6% (95% CI: 11.4-25.8%), and the sensitivity and specificity were 100% (95% CI: 100-100% and 95% CI: 100-100%), respectively. The cost difference for AI screening compared to human screening was $143 lower per patient (cost-saving) in favour of AI.

CONCLUSION

Our results indicate that The EyeArt AI system is both a reliable, cost-saving, and useful tool for DR grading in clinical practice.

Artificial Intelligence (AI) for Early Diagnosis of Retinal Diseases

Artificial intelligence (AI) has emerged as a transformative tool in the field of ophthalmology, revolutionizing disease diagnosis and management. This paper provides a comprehensive overview of AI applications in various retinal diseases, highlighting its potential to enhance screening efficiency, facilitate early diagnosis, and improve patient outcomes. Herein, we elucidate the fundamental concepts of AI, including machine learning (ML) and deep learning (DL), and their application in ophthalmology, underscoring the significance of AI-driven solutions in addressing the complexity and variability of retinal diseases. Furthermore, we delve into the specific applications of AI in retinal diseases such as diabetic retinopathy (DR), age-related macular degeneration (AMD), Macular Neovascularization, retinopathy of prematurity (ROP), retinal vein occlusion (RVO), hypertensive retinopathy (HR), Retinitis Pigmentosa, Stargardt disease, best vitelliform macular dystrophy, and sickle cell retinopathy. We focus on the current landscape of AI technologies, including various AI models, their performance metrics, and clinical implications. Furthermore, we aim to address challenges and pitfalls associated with the integration of AI in clinical practice, including the "black box phenomenon", biases in data representation, and limitations in comprehensive patient assessment. In conclusion, this review emphasizes the collaborative role of AI alongside healthcare professionals, advocating for a synergistic approach to healthcare delivery. It highlights the importance of leveraging AI to augment, rather than replace, human expertise, thereby maximizing its potential to revolutionize healthcare delivery, mitigate healthcare disparities, and improve patient outcomes in the evolving landscape of medicine.

An Economic Analysis for the Use of Artificial Intelligence in Screening for Diabetic Retinopathy in Trinidad and Tobago

This is a systematic review of 25 publications on the topics of the prevalence and cost of diabetic retinopathy (DR) in Trinidad and Tobago, the cost of traditional methods of screening for DR, and the use and cost of artificial intelligence (AI) in screening for DR. Analysis of these publications was used to identify and make estimates for how resources allocated to ophthalmology in public health systems in Trinidad and Tobago can be more efficiently utilized by employing AI in diagnosing treatable DR. DR screening was found to be an effective method of detecting the disease. Screening was found to be a universally cost-effective method of disease prevention and for altering the natural history of the disease in the spectrum of low-middle to high-income economies, such as Rwanda, Thailand, China, South Korea, and Singapore. AI and deep learning systems were found to be clinically superior to, or as effective as, human graders in areas where they were deployed, indicating that the systems are clinically safe. They have been shown to improve access to diabetic retinal screening, improve compliance with screening appointments, and prove to be cost-effective, especially in rural areas. Trinidad and Tobago, which is estimated to be disproportionately more affected by the burden of DR when projected out to the mid-21st century, stands to save as much as US$60 million annually from the implementation of an AI-based system to screen for DR versus conventional manual grading.

Assessing Diabetic Retinopathy Staging With AI: A Comparative Analysis Between Pseudocolor and LED Imaging

Purpose

To compare the diagnostic performance of artificial intelligence (AI)-based diabetic retinopathy (DR) staging system across pseudocolor, simulated white light (SWL), and light-emitting diode (LED) camera imaging modalities.

Methods

A cross-sectional investigation involved patients with diabetes undergoing imaging with an iCare DRSplus confocal LED camera and an Optos confocal, ultra-widefield pseudocolor camera, with and without SWL. Macula-centered and optic nerve-centered 45 × 45-degree photographs were processed using EyeArt v2.1. Human graders established the ground truth (GT) for DR severity on dilated fundus exams. Sensitivity and weighted Cohen's weighted kappa (wκ) were calculated. An ordinal generalized linear mixed model identified factors influencing accurate DR staging.

Results

The study included 362 eyes from 189 patients. The LED camera excelled in identifying sight-threatening DR stages (sensitivity = 0.83, specificity = 0.95 for proliferative DR) and had the highest agreement with the GT (wκ = 0.71). The addition of SWL to pseudocolor imaging resulted in decreased performance (sensitivity = 0.33, specificity = 0.98 for proliferative DR; wκ = 0.55). Peripheral lesions reduced the likelihood of being staged in the same or higher DR category by 80% (P < 0.001).

Conclusions

Pseudocolor and LED cameras, although proficient, demonstrated non-interchangeable performance, with the LED camera exhibiting superior accuracy in identifying advanced DR stages. These findings underscore the importance of implementing AI systems trained for ultra-widefield imaging, considering the impact of peripheral lesions on correct DR staging.

Translational Relevance

This study underscores the need for artificial intelligence-based systems specifically trained for ultra-widefield imaging in diabetic retinopathy assessment.

Feasibility and accuracy of the screening for diabetic retinopathy using a fundus camera and an artificial intelligence pre-evaluation application

AIMS

Periodical screening for diabetic retinopathy (DR) by an ophthalmologist is expensive and demanding. Automated DR image evaluation with Artificial Intelligence tools may represent a clinical and cost-effective alternative for the detection of retinopathy. We aimed to evaluate the accuracy and reliability of a machine learning algorithm.

METHODS

This was an observational diagnostic precision study that compared human grader classification with that of DAIRET®, an algorithm nested in an electronic medical record powered by Retmarker SA. Retinal images were taken from 637 consecutive patients attending a routine annual diabetic visit between June 2021 and February 2023. They were manually graded by an ophthalmologist following the International Clinical Diabetic Retinopathy Severity Scale and the results were compared with those of the AI responses. The main outcome measures were screening performance, such as sensitivity and specificity and diagnostic accuracy by 95% confidence intervals.

RESULTS

The rate of cases classified as ungradable was 1.2%, a figure consistent with the literature. DAIRET® sensitivity in the detection of cases of referable DR (moderate and above, "sight-threatening" forms of retinopathy) was equal to 1 (100%). The specificity, that is the true negative rate of absence of DR, was 80 ± 0.04.

CONCLUSIONS

DAIRET® achieved excellent sensitivity for referable retinopathy compared with that of human graders. This is undoubtedly the key finding of the study and translates into the certainty that no patient in need of the ophthalmologist is misdiagnosed as negative. It also had sufficient specificity to represent a cost-effective alternative to manual grade alone.

Population impact and cost-effectiveness of artificial intelligence-based diabetic retinopathy screening in people living with diabetes in Australia: a cost effectiveness analysis

Background

We aimed to evaluate the cost-effectiveness of an artificial intelligence-(AI) based diabetic retinopathy (DR) screening system in the primary care setting for both non-Indigenous and Indigenous people living with diabetes in Australia.

Methods

We performed a cost-effectiveness analysis between January 01, 2022 and August 01, 2023. A decision-analytic Markov model was constructed to simulate DR progression in a population of 1,197,818 non-Indigenous and 65,160 Indigenous Australians living with diabetes aged ≥20 years over 40 years. From a healthcare provider's perspective, we compared current practice to three primary care AI-based screening scenarios-(A) substitution of current manual grading, (B) scaling up to patient acceptance level, and (C) achieving universal screening. Study results were presented as incremental cost-effectiveness ratio (ICER), benefit-cost ratio (BCR), and net monetary benefits (NMB). A Willingness-to-pay (WTP) threshold of AU$50,000 per quality-adjusted life year (QALY) and a discount rate of 3.5% were adopted in this study.

Findings

With the status quo, the non-Indigenous diabetic population was projected to develop 96,269 blindness cases, resulting in AU$13,039.6 m spending on DR screening and treatment during 2020-2060. In comparison, all three intervention scenarios were effective and cost-saving. In particular, if a universal screening program was to be implemented (Scenario C), it would prevent 38,347 blindness cases, gain 172,090 QALYs and save AU$595.8 m, leading to a BCR of 3.96 and NMB of AU$9,200 m. Similar findings were also reported in the Indigenous population. With the status quo, 3,396 Indigenous individuals would develop blindness, which would cost the health system AU$796.0 m during 2020-2060. All three intervention scenarios were cost-saving for the Indigenous population. Notably, universal AI-based DR screening (Scenario C) would prevent 1,211 blindness cases and gain 9,800 QALYs in the Indigenous population, leading to a saving of AU$19.2 m with a BCR of 1.62 and NMB of AU$509 m.

Interpretation

Our findings suggest that implementing AI-based DR screening in primary care is highly effective and cost-saving in both Indigenous and non-Indigenous populations.

Funding

This project received grant funding from the Australian Government: the National Critical Research Infrastructure Initiative, Medical Research Future Fund (MRFAI00035) and the NHMRC Investigator Grant (APP1175405). The contents of the published material are solely the responsibility of the Administering Institution, a participating institution or individual authors and do not reflect the views of the NHMRC. This work was supported by the Global STEM Professorship Scheme (P0046113), the Fundamental Research Funds of the State Key Laboratory of Ophthalmology, Project of Investigation on Health Status of Employees in Financial Industry in Guangzhou, China (Z012014075). The Centre for Eye Research Australia receives Operational Infrastructure Support from the Victorian State Government. W.H. is supported by the Melbourne Research Scholarship established by the University of Melbourne. The funding source had no role in the design and conduct of the study; collection, management, analysis, and interpretation of the data; preparation, review, or approval of the manuscript; and decision to submit the manuscript for publication.

The Advent of Artificial Intelligence in Diabetic Foot Medicine: A New Horizon, a New Order, or a False Dawn?

In a world where automation is becoming increasingly common, easier collection of mass of data and powerful computer processing has meant a transformation in the field of artificial intelligence (AI). The diabetic foot is a multifactorial problem; its issues render it suitable for analysis, interrogation, and development of AI. The latter has the potential to deliver many solutions to issues of delayed diagnosis, compliance, and defining preventative treatments. We describe the use of AI and the development of artificial neural networks that may supplement the failed networks in the diabetic foot. The potential of this technology, current developing applications, and their limitations for diabetic foot care are suggested.

Two-year recall for people with no diabetic retinopathy: a multi-ethnic population-based retrospective cohort study using real-world data to quantify the effect

BACKGROUND/AIMS

The English Diabetic Eye Screening Programme (DESP) offers people living with diabetes (PLD) annual screening. Less frequent screening has been advocated among PLD without diabetic retinopathy (DR), but evidence for each ethnic group is limited. We examined the potential effect of biennial versus annual screening on the detection of sight-threatening diabetic retinopathy (STDR) and proliferative diabetic retinopathy (PDR) among PLD without DR from a large urban multi-ethnic English DESP.

METHODS

PLD in North-East London DESP (January 2012 to December 2021) with no DR on two prior consecutive screening visits with up to 8 years of follow-up were examined. Annual STDR and PDR incidence rates, overall and by ethnicity, were quantified. Delays in identification of STDR and PDR events had 2-year screening intervals been used were determined.

FINDINGS

Among 82 782 PLD (37% white, 36% South Asian, and 16% black people), there were 1788 incident STDR cases over mean (SD) 4.3 (2.4) years (STDR rate 0.51, 95% CI 0.47 to 0.55 per 100-person-years). STDR incidence rates per 100-person-years by ethnicity were 0.55 (95% CI 0.48 to 0.62) for South Asian, 0.34 (95% CI 0.29 to 0.40) for white, and 0.77 (95% CI 0.65 to 0.90) for black people. Biennial screening would have delayed diagnosis by 1 year for 56.3% (1007/1788) with STDR and 43.6% (45/103) with PDR. Standardised cumulative rates of delayed STDR per 100 000 persons for each ethnic group were 1904 (95% CI 1683 to 2154) for black people, 1276 (95% CI 1153 to 1412) for South Asian people, and 844 (95% CI 745 to 955) for white people.

INTERPRETATION

Biennial screening would have delayed detection of some STDR and PDR by 1 year, especially among those of black ethnic origin, leading to healthcare inequalities.

Artificial Intelligence in Ophthalmic Surgery: Current Applications and Expectations

Artificial Intelligence (AI) has found rapidly growing applications in ophthalmology, achieving robust recognition and classification in most kind of ocular diseases. Ophthalmic surgery is one of the most delicate microsurgery, requiring high fineness and stability of surgeons. The massive demand of the AI assist ophthalmic surgery will constitute an important factor in boosting accelerate precision medicine. In clinical practice, it is instrumental to update and review the considerable evidence of the current AI technologies utilized in the investigation of ophthalmic surgery involved in both the progression and innovation of precision medicine. Bibliographic databases including PubMed and Google Scholar were searched using keywords such as "ophthalmic surgery", "surgical selection", "candidate screening", and "robot-assisted surgery" to find articles about AI technology published from 2018 to 2023. In addition to the Editorials and letters to the editor, all types of approaches are considered. In this paper, we will provide an up-to-date review of artificial intelligence in eye surgery, with a specific focus on its application to candidate screening, surgery selection, postoperative prediction, and real-time intraoperative guidance.

Cost-effectiveness analysis of artificial intelligence-based diabetic retinopathy screening in rural China based on the Markov model

This study assessed the cost-effectiveness of different diabetic retinopathy (DR) screening strategies in rural regions in China by using a Markov model to make health economic evaluations. In this study, we determined the structure of a Markov model according to the research objectives, which required parameters collected through field investigation and literature retrieval. After perfecting the model with parameters and assumptions, we developed a Markov decision analytic model according to the natural history of DR in TreeAge Pro 2011. For this model, we performed Markov cohort and cost-effectiveness analyses to simulate the probabilistic distributions of different developments in DR and the cumulative cost-effectiveness of artificial intelligence (AI)-based screening and ophthalmologist screening for DR in the rural population with diabetes mellitus (DM) in China. Additionally, a model-based health economic evaluation was performed by using quality-adjusted life years (QALYs) and incremental cost-effectiveness ratios. Last, one-way and probabilistic sensitivity analyses were performed to assess the stability of the results. From the perspective of the health system, compared with no screening, AI-based screening cost more (the incremental cost was 37,257.76 RMB (approximately 5,211.31 US dollars)), but the effect was better (the incremental utility was 0.33). Compared with AI-based screening, the cost of ophthalmologist screening was higher (the incremental cost was 14,886.76 RMB (approximately 2,070.19 US dollars)), and the effect was worse (the incremental utility was -0.31). Compared with no screening, the incremental cost-effectiveness ratio (ICER) of AI-based DR screening was 112,146.99 RMB (15,595.47 US dollars)/QALY, which was less than the threshold for the ICER (< 3 times the per capita gross domestic product (GDP), 217,341.00 RMB (30,224.03 US dollars)). Therefore, AI-based screening was cost-effective, which meant that the increased cost for each additional quality-adjusted life year was merited. Compared with no screening and ophthalmologist screening for DR, AI-based screening was the most cost-effective, which not only saved costs but also improved the quality of life of diabetes patients. Popularizing AI-based DR screening strategies in rural areas would be economically effective and feasible and can provide a scientific basis for the further formulation of early screening programs for diabetic retinopathy.

What are the perceptions and concerns of people living with diabetes and National Health Service staff around the potential implementation of AI-assisted screening for diabetic eye disease? Development and validation of a survey for use in a secondary care screening setting

INTRODUCTION

The English National Health Service (NHS) Diabetic Eye Screening Programme (DESP) performs around 2.3 million eye screening appointments annually, generating approximately 13 million retinal images that are graded by humans for the presence or severity of diabetic retinopathy. Previous research has shown that automated retinal image analysis systems, including artificial intelligence (AI), can identify images with no disease from those with diabetic retinopathy as safely and effectively as human graders, and could significantly reduce the workload for human graders. Some algorithms can also determine the level of severity of the retinopathy with similar performance to humans. There is a need to examine perceptions and concerns surrounding AI-assisted eye-screening among people living with diabetes and NHS staff, if AI was to be introduced into the DESP, to identify factors that may influence acceptance of this technology.

METHODS AND ANALYSIS

People living with diabetes and staff from the North East London (NEL) NHS DESP were invited to participate in two respective focus groups to codesign two online surveys exploring their perceptions and concerns around the potential introduction of AI-assisted screening.Focus group participants were representative of the local population in terms of ages and ethnicity. Participants' feedback was taken into consideration to update surveys which were circulated for further feedback. Surveys will be piloted at the NEL DESP and followed by semistructured interviews to assess accessibility, usability and to validate the surveys.Validated surveys will be distributed by other NHS DESP sites, and also via patient groups on social media, relevant charities and the British Association of Retinal Screeners. Post-survey evaluative interviews will be undertaken among those who consent to participate in further research.

ETHICS AND DISSEMINATION

Ethical approval has been obtained by the NHS Research Ethics Committee (IRAS ID: 316631). Survey results will be shared and discussed with focus groups to facilitate preparation of findings for publication and to inform codesign of outreach activities to address concerns and perceptions identified.

A Survey on Artificial Intelligence in Pulmonary Imaging

Over the last decade, deep learning (DL) has contributed a paradigm shift in computer vision and image recognition creating widespread opportunities of using artificial intelligence in research as well as industrial applications. DL has been extensively studied in medical imaging applications, including those related to pulmonary diseases. Chronic obstructive pulmonary disease, asthma, lung cancer, pneumonia, and, more recently, COVID-19 are common lung diseases affecting nearly 7.4% of world population. Pulmonary imaging has been widely investigated toward improving our understanding of disease etiologies and early diagnosis and assessment of disease progression and clinical outcomes. DL has been broadly applied to solve various pulmonary image processing challenges including classification, recognition, registration, and segmentation. This paper presents a survey of pulmonary diseases, roles of imaging in translational and clinical pulmonary research, and applications of different DL architectures and methods in pulmonary imaging with emphasis on DL-based segmentation of major pulmonary anatomies such as lung volumes, lung lobes, pulmonary vessels, and airways as well as thoracic musculoskeletal anatomies related to pulmonary diseases.

Development of LuxIA, a Cloud-Based AI Diabetic Retinopathy Screening Tool Using a Single Color Fundus Image

Purpose

Diabetic retinopathy (DR) is the leading cause of vision impairment in working-age adults. Automated screening can increase DR detection at early stages at relatively low costs. We developed and evaluated a cloud-based screening tool that uses artificial intelligence (AI), the LuxIA algorithm, to detect DR from a single fundus image.

Methods

Color fundus images that were previously graded by expert readers were collected from the Canarian Health Service (Retisalud) and used to train LuxIA, a deep-learning-based algorithm for the detection of more than mild DR. The algorithm was deployed in the Discovery cloud platform to evaluate each test set. Sensitivity, specificity, accuracy, and area under the receiver operating characteristic curve were computed using a bootstrapping method to evaluate the algorithm performance and compared through different publicly available datasets. A usability test was performed to assess the integration into a clinical tool.

Results

Three separate datasets, Messidor-2, APTOS, and a holdout set from Retisalud were evaluated. Mean sensitivity and specificity with 95% confidence intervals (CIs) reached for these three datasets were 0.901 (0.901-0.902) and 0.955 (0.955-0.956), 0.995 (0.995-0.995) and 0.821 (0.821-0.823), and 0.911 (0.907-0.912) and 0.880 (0.879-0.880), respectively. The usability test confirmed the successful integration of LuxIA into Discovery.

Conclusions

Clinical data were used to train the deep-learning-based algorithm LuxIA to an expert-level performance. The whole process (image uploading and analysis) was integrated into the cloud-based platform Discovery, allowing more patients to have access to expert-level screening tools.

Translational Relevance

Using the cloud-based LuxIA tool as part of a screening program may give diabetic patients greater access to specialist-level decisions, without the need for consultation.

Clinician-Driven AI: Code-Free Self-Training on Public Data for Diabetic Retinopathy Referral

Importance

Democratizing artificial intelligence (AI) enables model development by clinicians with a lack of coding expertise, powerful computing resources, and large, well-labeled data sets.

Objective

To determine whether resource-constrained clinicians can use self-training via automated machine learning (ML) and public data sets to design high-performing diabetic retinopathy classification models.

Design, Setting, and Participants

This diagnostic quality improvement study was conducted from January 1, 2021, to December 31, 2021. A self-training method without coding was used on 2 public data sets with retinal images from patients in France (Messidor-2 [n = 1748]) and the UK and US (EyePACS [n = 58 689]) and externally validated on 1 data set with retinal images from patients of a private Egyptian medical retina clinic (Egypt [n = 210]). An AI model was trained to classify referable diabetic retinopathy as an exemplar use case. Messidor-2 images were assigned adjudicated labels available on Kaggle; 4 images were deemed ungradable and excluded, leaving 1744 images. A total of 300 images randomly selected from the EyePACS data set were independently relabeled by 3 blinded retina specialists using the International Classification of Diabetic Retinopathy protocol for diabetic retinopathy grade and diabetic macular edema presence; 19 images were deemed ungradable, leaving 281 images. Data analysis was performed from February 1 to February 28, 2021.

Exposures

Using public data sets, a teacher model was trained with labeled images using supervised learning. Next, the resulting predictions, termed pseudolabels, were used on an unlabeled public data set. Finally, a student model was trained with the existing labeled images and the additional pseudolabeled images.

Main Outcomes and Measures

The analyzed metrics for the models included the area under the receiver operating characteristic curve (AUROC), accuracy, sensitivity, specificity, and F1 score. The Fisher exact test was performed, and 2-tailed P values were calculated for failure case analysis.

Results

For the internal validation data sets, AUROC values for performance ranged from 0.886 to 0.939 for the teacher model and from 0.916 to 0.951 for the student model. For external validation of automated ML model performance, AUROC values and accuracy were 0.964 and 93.3% for the teacher model, 0.950 and 96.7% for the student model, and 0.890 and 94.3% for the manually coded bespoke model, respectively.

Conclusions and Relevance

These findings suggest that self-training using automated ML is an effective method to increase both model performance and generalizability while decreasing the need for costly expert labeling. This approach advances the democratization of AI by enabling clinicians without coding expertise or access to large, well-labeled private data sets to develop their own AI models.

Artificial Intelligence and Diabetic Retinopathy: AI Framework, Prospective Studies, Head-to-head Validation, and Cost-effectiveness

Current guidelines recommend that individuals with diabetes receive yearly eye exams for detection of referable diabetic retinopathy (DR), one of the leading causes of new-onset blindness. For addressing the immense screening burden, artificial intelligence (AI) algorithms have been developed to autonomously screen for DR from fundus photography without human input. Over the last 10 years, many AI algorithms have achieved good sensitivity and specificity (>85%) for detection of referable DR compared with human graders; however, many questions still remain. In this narrative review on AI in DR screening, we discuss key concepts in AI algorithm development as a background for understanding the algorithms. We present the AI algorithms that have been prospectively validated against human graders and demonstrate the variability of reference standards and cohort demographics. We review the limited head-to-head validation studies where investigators attempt to directly compare the available algorithms. Next, we discuss the literature regarding cost-effectiveness, equity and bias, and medicolegal considerations, all of which play a role in the implementation of these AI algorithms in clinical practice. Lastly, we highlight ongoing efforts to bridge gaps in AI model data sets to pursue equitable development and delivery.

Artificial intelligence for telemedicine diabetic retinopathy screening: a review

PURPOSE

This study aims to compare artificial intelligence (AI) systems applied in diabetic retinopathy (DR) teleophthalmology screening, currently deployed systems, fairness initiatives and the challenges for implementation.

METHODS

The review included articles retrieved from PubMed/Medline/EMBASE literature search strategy regarding telemedicine, DR and AI. The screening criteria included human articles in English, Portuguese or Spanish and related to telemedicine and AI for DR screening. The author's affiliations and the study's population income group were classified according to the World Bank Country and Lending Groups.

RESULTS

The literature search yielded a total of 132 articles, and nine were included after full-text assessment. The selected articles were published between 2004 and 2020 and were grouped as telemedicine systems, algorithms, economic analysis and image quality assessment. Four telemedicine systems that perform a quality assessment, image preprocessing and pathological screening were reviewed. A data and post-deployment bias assessment are not performed in any of the algorithms, and none of the studies evaluate the social impact implementations. There is a lack of representativeness in the reviewed articles, with most authors and target populations from high-income countries and no low-income country representation.

CONCLUSIONS

Telemedicine and AI hold great promise for augmenting decision-making in medical care, expanding patient access and enhancing cost-effectiveness. Economic studies and social science analysis are crucial to support the implementation of AI in teleophthalmology screening programs. Promoting fairness and generalizability in automated systems combined with telemedicine screening programs is not straightforward. Improving data representativeness, reducing biases and promoting equity in deployment and post-deployment studies are all critical steps in model development.

Recent evidence of economic evaluation of artificial intelligence in ophthalmology

PURPOSE OF REVIEW

Health economic evaluation (HEE) is essential for assessing value of health interventions, including artificial intelligence. Recent approaches, current challenges, and future directions of HEE of artificial intelligence in ophthalmology are reviewed.

RECENT FINDINGS

Majority of recent HEEs of artificial intelligence in ophthalmology were for diabetic retinopathy screening. Two models, one conducted in the rural USA (5-year period) and another in China (35-year period), found artificial intelligence to be more cost-effective than without screening for diabetic retinopathy. Two additional models, which compared artificial intelligence with human screeners in Brazil and Thailand for the lifetime of patients, found artificial intelligence to be more expensive from a healthcare system perspective. In the Thailand analysis, however, artificial intelligence was less expensive when opportunity loss from blindness was included. An artificial intelligence model for screening retinopathy of prematurity was cost-effective in the USA. A model for screening age-related macular degeneration in Japan and another for primary angle close in China did not find artificial intelligence to be cost-effective, compared with no screening. The costs of artificial intelligence varied widely in these models.

SUMMARY

Like other medical fields, there is limited evidence in assessing the value of artificial intelligence in ophthalmology and more appropriate HEE models are needed.

Is Artificial Intelligence the Cost-Saving Lens to Diabetic Retinopathy Screening in Low- and Middle-Income Countries?

Diabetes is a rapidly growing global health crisis disproportionately affecting low- and middle-income countries (LMICs). The emergence of diabetes as a global pandemic is one of the major challenges to human health, as long-term microvascular complications such as diabetic retinopathy (DR) can lead to irreversible blindness. Leveraging artificial intelligence (AI) technology may improve the diagnostic accuracy, efficiency, and accessibility of DR screenings across LMICs. However, there is a gap between the potential of AI technology and its implementation in clinical practice. The main objective of this systematic review is to summarize the currently available literature on the health economic assessments of AI implementation for DR screening in LMICs. The review was conducted according to the Preferred Reporting Items for Systematic Reviews and Meta-Analysis (PRISMA) guidelines. We conducted an extensive systematic search of PubMed/MEDLINE, Scopus, and the Web of Science on July 15, 2023. Our review included full-text English-language articles from any publication year. The Joanna Briggs Institute's (JBI) critical appraisal checklist for economic evaluations was used to rate the quality and rigor of the selected articles. The initial search generated 1,423 records and was narrowed to five full-text articles through comprehensive inclusion and exclusion criteria. Of the five articles included in our systematic review, two used a cost-effectiveness analysis, two used a cost-utility analysis, and one used both a cost-effectiveness analysis and a cost-utility analysis. Across the five articles, LMICs such as China, Thailand, and Brazil were represented in the economic evaluations and models. Overall, three out of the five articles concluded that AI-based DR screening was more cost-effective in comparison to standard-of-care screening methods. Our systematic review highlights the need for more primary health economic analyses that carefully evaluate the economic implications of adopting AI technology for DR screening in LMICs. We hope this systematic review will offer valuable guidance to healthcare providers, scientists, and legislators to support appropriate decision-making regarding the implementation of AI algorithms for DR screening in healthcare workflows.

Cross-camera Performance of Deep Learning Algorithms to Diagnose Common Ophthalmic Diseases: A Comparative Study Highlighting Feasibility to Portable Fundus Camera Use

PURPOSE

To compare the inter-camera performance and consistency of various deep learning (DL) diagnostic algorithms applied to fundus images taken from desktop Topcon and portable Optain cameras.

METHODS

Participants over 18 years of age were enrolled between November 2021 and April 2022. Pair-wise fundus photographs from each patient were collected in a single visit; once by Topcon (used as the reference camera) and once by a portable Optain camera (the new target camera). These were analyzed by three previously validated DL models for the detection of diabetic retinopathy (DR), age-related macular degeneration (AMD), and glaucomatous optic neuropathy (GON). Ophthalmologists manually analyzed all fundus photos for the presence of DR and these were referred to as the ground truth. Sensitivity, specificity, the area under the curve (AUC) and agreement between cameras (estimated by Cohen's weighted kappa, K) were the primary outcomes of this study.

RESULTS

A total of 504 patients were recruited. After excluding 12 photographs with matching errors and 59 photographs with low quality, 906 pairs of Topcon-Optain fundus photos were available for algorithm assessment. Topcon and Optain cameras had excellent consistency (Κ=0.80) when applied to the referable DR algorithm, while AMD had moderate consistency (Κ=0.41) and GON had poor consistency (Κ=0.32). For the DR model, Topcon and Optain achieved a sensitivity of 97.70% and 97.67% and a specificity of 97.92% and 97.93%, respectively. There was no significant difference between the two camera models (McNemar's test: x2=0.08, p = .78).

CONCLUSION

Topcon and Optain cameras had excellent consistency for detecting referable DR, albeit performances for detecting AMD and GON models were unsatisfactory. This study highlights the methods of using pair-wise images to evaluate DL models between reference and new fundus cameras.

Automatic Detection and Classification of Diabetic Retinopathy Using the Improved Pooling Function in the Convolution Neural Network

Diabetic retinopathy (DR) is an eye disease associated with diabetes that can lead to blindness. Early diagnosis is critical to ensure that patients with diabetes are not affected by blindness. Deep learning plays an important role in diagnosing diabetes, reducing the human effort to diagnose and classify diabetic and non-diabetic patients. The main objective of this study was to provide an improved convolution neural network (CNN) model for automatic DR diagnosis from fundus images. The pooling function increases the receptive field of convolution kernels over layers. It reduces computational complexity and memory requirements because it reduces the resolution of feature maps while preserving the essential characteristics required for subsequent layer processing. In this study, an improved pooling function combined with an activation function in the ResNet-50 model was applied to the retina images in autonomous lesion detection with reduced loss and processing time. The improved ResNet-50 model was trained and tested over the two datasets (i.e., APTOS and Kaggle). The proposed model achieved an accuracy of 98.32% for APTOS and 98.71% for Kaggle datasets. It is proven that the proposed model has produced greater accuracy when compared to their state-of-the-art work in diagnosing DR with retinal fundus images.

Ethnicity is not biology: retinal pigment score to evaluate biological variability from ophthalmic imaging using machine learning

Background

Few metrics exist to describe phenotypic diversity within ophthalmic imaging datasets, with researchers often using ethnicity as an inappropriate marker for biological variability.

Methods

We derived a continuous, measured metric, the retinal pigment score (RPS), that quantifies the degree of pigmentation from a colour fundus photograph of the eye. RPS was validated using two large epidemiological studies with demographic and genetic data (UK Biobank and EPIC-Norfolk Study).

Findings

A genome-wide association study (GWAS) of RPS from UK Biobank identified 20 loci with known associations with skin, iris and hair pigmentation, of which 8 were replicated in the EPIC-Norfolk cohort. There was a strong association between RPS and ethnicity, however, there was substantial overlap between each ethnicity and the respective distributions of RPS scores.

Interpretation

RPS serves to decouple traditional demographic variables, such as ethnicity, from clinical imaging characteristics. RPS may serve as a useful metric to quantify the diversity of the training, validation, and testing datasets used in the development of AI algorithms to ensure adequate inclusion and explainability of the model performance, critical in evaluating all currently deployed AI models. The code to derive RPS is publicly available at: https://github.com/uw-biomedical-ml/retinal-pigmentation-score.

Funding

The authors did not receive support from any organisation for the submitted work.

Variability in Grading Diabetic Retinopathy Using Retinal Photography and Its Comparison with an Automated Deep Learning Diabetic Retinopathy Screening Software

BACKGROUND

Diabetic retinopathy (DR) screening using colour retinal photographs is cost-effective and time-efficient. In real-world clinical settings, DR severity is frequently graded by individuals of different expertise levels. We aim to determine the agreement in DR severity grading between human graders of varying expertise and an automated deep learning DR screening software (ADLS).

METHODS

Using the International Clinical DR Disease Severity Scale, two hundred macula-centred fundus photographs were graded by retinal specialists, ophthalmology residents, family medicine physicians, medical students, and the ADLS. Based on referral urgency, referral grading was divided into no referral, non-urgent referral, and urgent referral to an ophthalmologist. Inter-observer and intra-group variations were analysed using Gwet's agreement coefficient, and the performance of ADLS was evaluated using sensitivity and specificity.

RESULTS

The agreement coefficient for inter-observer and intra-group variability ranged from fair to very good, and moderate to good, respectively. The ADLS showed a high area under curve of 0.879, 0.714, and 0.836 for non-referable DR, non-urgent referable DR, and urgent referable DR, respectively, with varying sensitivity and specificity values.

CONCLUSION

Inter-observer and intra-group agreements among human graders vary widely, but ADLS is a reliable and reasonably sensitive tool for mass screening to detect referable DR and urgent referable DR.

Cost-Utility Analysis of Deep Learning and Trained Human Graders for Diabetic Retinopathy Screening in a Nationwide Program

INTRODUCTION

Deep learning (DL) for screening diabetic retinopathy (DR) has the potential to address limited healthcare resources by enabling expanded access to healthcare. However, there is still limited health economic evaluation, particularly in low- and middle-income countries, on this subject to aid decision-making for DL adoption.

METHODS

In the context of a middle-income country (MIC), using Thailand as a model, we constructed a decision tree-Markov hybrid model to estimate lifetime costs and outcomes of Thailand's national DR screening program via DL and trained human graders (HG). We calculated the incremental cost-effectiveness ratio (ICER) between the two strategies. Sensitivity analyses were performed to probe the influence of modeling parameters.

RESULTS

From a societal perspective, screening with DL was associated with a reduction in costs of ~ US$ 2.70, similar quality-adjusted life-years (QALY) of + 0.0043, and an incremental net monetary benefit of ~ US$ 24.10 in the base case. In sensitivity analysis, DL remained cost-effective even with a price increase from US$ 1.00 to US$ 4.00 per patient at a Thai willingness-to-pay threshold of ~ US$ 4.997 per QALY gained. When further incorporating recent findings suggesting improved compliance to treatment referral with DL, our analysis models effectiveness benefits of ~ US$ 20 to US$ 50 depending on compliance.

CONCLUSION

DR screening using DL in an MIC using Thailand as a model may result in societal cost-savings and similar health outcomes compared with HG. This study may provide an economic rationale to expand DL-based DR screening in MICs as an alternative solution for limited availability of skilled human resources for primary screening, particularly in MICs with similar prevalence of diabetes and low compliance to referrals for treatment.

Artificial Intelligence for Diabetic Retinopathy Screening Using Color Retinal Photographs: From Development to Deployment

Diabetic retinopathy (DR), a leading cause of preventable blindness, is expected to remain a growing health burden worldwide. Screening to detect early sight-threatening lesions of DR can reduce the burden of vision loss; nevertheless, the process requires intensive manual labor and extensive resources to accommodate the increasing number of patients with diabetes. Artificial intelligence (AI) has been shown to be an effective tool which can potentially lower the burden of screening DR and vision loss. In this article, we review the use of AI for DR screening on color retinal photographs in different phases of application, ranging from development to deployment. Early studies of machine learning (ML)-based algorithms using feature extraction to detect DR achieved a high sensitivity but relatively lower specificity. Robust sensitivity and specificity were achieved with the application of deep learning (DL), although ML is still used in some tasks. Public datasets were utilized in retrospective validations of the developmental phases in most algorithms, which require a large number of photographs. Large prospective clinical validation studies led to the approval of DL for autonomous screening of DR although the semi-autonomous approach may be preferable in some real-world settings. There have been few reports on real-world implementations of DL for DR screening. It is possible that AI may improve some real-world indicators for eye care in DR, such as increased screening uptake and referral adherence, but this has not been proven. The challenges in deployment may include workflow issues, such as mydriasis to lower ungradable cases; technical issues, such as integration into electronic health record systems and integration into existing camera systems; ethical issues, such as data privacy and security; acceptance of personnel and patients; and health-economic issues, such as the need to conduct health economic evaluations of using AI in the context of the country. The deployment of AI for DR screening should follow the governance model for AI in healthcare which outlines four main components: fairness, transparency, trustworthiness, and accountability.

Economic evaluation of combined population-based screening for multiple blindness-causing eye diseases in China: a cost-effectiveness analysis

BACKGROUND

More than 90% of vision impairment is avoidable. However, in China, a routine screening programme is currently unavailable in primary health care. With the dearth of economic evidence on screening programmes for multiple blindness-causing eye diseases, delivery options, and screening frequencies, we aimed to evaluate the costs and benefits of a population-based screening programme for multiple eye diseases in China.

METHODS

We developed a decision-analytic Markov model for a cohort of individuals aged 50 years and older with a total of 30 1-year cycles. We calculated the cost-effectiveness and cost-utility of screening programmes for multiple major blindness-causing eye diseases in China, including age-related macular degeneration, glaucoma, diabetic retinopathy, cataracts, and pathological myopia, from a societal perspective (including direct and indirect costs). We analysed rural and urban settings separately by different screening delivery options (non-telemedicine [ie, face-to-face] screening, artificial intelligence [AI] telemedicine screening, and non-AI telemedicine screening) and frequencies. We calculated incremental cost-utility ratios (ICURs) using quality-adjusted life-years and incremental cost-effectiveness ratios (ICERs) in terms of the cost per blindness year avoided. One-way deterministic and simulated probabilistic sensitivity analyses were used to assess the robustness of the main outcomes.

FINDINGS

Compared with no screening, non-telemedicine combined screening of multiple eye diseases satisfied the criterion for a highly cost-effective health intervention, with an ICUR of US$2494 (95% CI 1130 to 2716) and an ICER of $12 487 (8773 to 18 791) in rural settings. In urban areas, the ICUR was $624 (395 to 907), and the ICER was $7251 (4238 to 13 501). Non-AI telemedicine screening could result in fewer costs and greater gains in health benefits (ICUR $2326 [1064 to 2538] and ICER $11 766 [8200 to 18 000] in rural settings; ICUR $581 [368 to 864] and ICER $6920 [3926 to 13 231] in urban settings). AI telemedicine screening dominated no screening in rural settings, and in urban settings the ICUR was $244 (-315 to 1073) and the ICER was $2567 (-4111 to 15 389). Sensitivity analyses showed all results to be robust. By further comparison, annual AI telemedicine screening was the most cost-effective strategy in both rural and urban areas.

INTERPRETATION

Combined screening of multiple eye diseases is cost-effective in both rural and urban China. AI coupled with teleophthalmology presents an opportunity to promote equity in eye health.

FUNDING

National Natural Science Foundation of China.

Artificial Intelligence in Community-Based Diabetic Retinopathy Telemedicine Screening in Urban China: Cost-effectiveness and Cost-Utility Analyses With Real-world Data

BACKGROUND

Community-based telemedicine screening for diabetic retinopathy (DR) has been highly recommended worldwide. However, evidence from low- and middle-income countries (LMICs) on the choice between artificial intelligence (AI)-based and manual grading-based telemedicine screening is inadequate for policy making.

OBJECTIVE

The aim of this study was to test whether the AI model is more worthwhile than manual grading in community-based telemedicine screening for DR in the context of labor costs in urban China.

METHODS

We conducted cost-effectiveness and cost-utility analyses by using decision-analytic Markov models with 30 one-year cycles from a societal perspective to compare the cost, effectiveness, and utility of 2 scenarios in telemedicine screening for DR: manual grading and an AI model. Sensitivity analyses were performed. Real-world data were obtained mainly from the Shanghai Digital Eye Disease Screening Program. The main outcomes were the incremental cost-effectiveness ratio (ICER) and the incremental cost-utility ratio (ICUR). The ICUR thresholds were set as 1 and 3 times the local gross domestic product per capita.

RESULTS

The total expected costs for a 65-year-old resident were US $3182.50 and US $3265.40, while the total expected years without blindness were 9.80 years and 9.83 years, and the utilities were 6.748 quality-adjusted life years (QALYs) and 6.753 QALYs in the AI model and manual grading, respectively. The ICER for the AI-assisted model was US $2553.39 per year without blindness, and the ICUR was US $15,216.96 per QALY, which indicated that AI-assisted model was not cost-effective. The sensitivity analysis suggested that if there is an increase in compliance with referrals after the adoption of AI by 7.5%, an increase in on-site screening costs in manual grading by 50%, or a decrease in on-site screening costs in the AI model by 50%, then the AI model could be the dominant strategy.

CONCLUSIONS

Our study may provide a reference for policy making in planning community-based telemedicine screening for DR in LMICs. Our findings indicate that unless the referral compliance of patients with suspected DR increases, the adoption of the AI model may not improve the value of telemedicine screening compared to that of manual grading in LMICs. The main reason is that in the context of the low labor costs in LMICs, the direct health care costs saved by replacing manual grading with AI are less, and the screening effectiveness (QALYs and years without blindness) decreases. Our study suggests that the magnitude of the value generated by this technology replacement depends primarily on 2 aspects. The first is the extent of direct health care costs reduced by AI, and the second is the change in health care service utilization caused by AI. Therefore, our research can also provide analytical ideas for other health care sectors in their decision to use AI.

Development and Application of a Standardized Testset for an Artificial Intelligence Medical Device Intended for the Computer-Aided Diagnosis of Diabetic Retinopathy

Objective

To explore a centralized approach to build test sets and assess the performance of an artificial intelligence medical device (AIMD) which is intended for computer-aided diagnosis of diabetic retinopathy (DR).

Method

A framework was proposed to conduct data collection, data curation, and annotation. Deidentified colour fundus photographs were collected from 11 partner hospitals with raw labels. Photographs with sensitive information or authenticity issues were excluded during vetting. A team of annotators was recruited through qualification examinations and trained. The annotation process included three steps: initial annotation, review, and arbitration. The annotated data then composed a standardized test set, which was further imported to algorithms under test (AUT) from different developers. The algorithm outputs were compared with the final annotation results (reference standard).

Result

The test set consists of 6327 digital colour fundus photographs. The final labels include 5 stages of DR and non-DR, as well as other ocular diseases and photographs with unacceptable quality. The Fleiss Kappa was 0.75 among the annotators. The Cohen's kappa between raw labels and final labels is 0.5. Using this test set, five AUTs were tested and compared quantitatively. The metrics include accuracy, sensitivity, and specificity. The AUTs showed inhomogeneous capabilities to classify different types of fundus photographs.

Conclusions

This article demonstrated a workflow to build standardized test sets and conduct algorithm testing of the AIMD for computer-aided diagnosis of diabetic retinopathy. It may provide a reference to develop technical standards that promote product verification and quality control, improving the comparability of products.

Research progress and application of artificial intelligence in thyroid associated ophthalmopathy

Thyroid-associated ophthalmopathy (TAO) is a complicated orbitopathy related to dysthyroid, which severely destroys the facial appearance and life quality without medical interference. The diagnosis and management of thyroid-associated ophthalmopathy are extremely intricate, as the number of professional ophthalmologists is limited and inadequate compared with the number of patients. Nowadays, medical applications based on artificial intelligence (AI) algorithms have been developed, which have proved effective in screening many chronic eye diseases. The advanced characteristics of automated artificial intelligence devices, such as rapidity, portability, and multi-platform compatibility, have led to significant progress in the early diagnosis and elaborate evaluation of these diseases in clinic. This study aimed to provide an overview of recent artificial intelligence applications in clinical diagnosis, activity and severity grading, and prediction of therapeutic outcomes in thyroid-associated ophthalmopathy. It also discussed the current challenges and future prospects of the development of artificial intelligence applications in treating thyroid-associated ophthalmopathy.

Development of a Computer System for Automatically Generating a Laser Photocoagulation Plan to Improve the Retinal Coagulation Quality in the Treatment of Diabetic Retinopathy

In this article, the development of a computer system for high-tech medical uses in ophthalmology is proposed. An overview of the main methods and algorithms that formed the basis of the coagulation plan planning system is presented. The system provides the formation of a more effective plan for laser coagulation in comparison with the use of existing coagulation techniques. An analysis of monopulse- and pattern-based laser coagulation techniques in the treatment of diabetic retinopathy has shown that modern treatment methods do not provide the required efficacy of medical laser coagulation procedures, as the laser energy is nonuniformly distributed across the pigment epithelium and may exert an excessive effect on parts of the retina and anatomical elements. The analysis has shown that the efficacy of retinal laser coagulation for the treatment of diabetic retinopathy is determined by the relative position of coagulates and parameters of laser exposure. In the course of the development of the computer system proposed herein, main stages of processing diagnostic data were identified. They are as follows: the allocation of the laser exposure zone, the evaluation of laser pulse parameters that would be safe for the fundus, mapping a coagulation plan in the laser exposure zone, followed by the analysis of the generated plan for predicting the therapeutic effect. In the course of the study, it was found that the developed algorithms for placing coagulates in the area of laser exposure provide a more uniform distribution of laser energy across the pigment epithelium when compared to monopulse- and pattern-based laser coagulation techniques.

AI-Aided Disease Prediction in Visualized Medicine

Artificial intelligence (AI) is playing a vitally important role in promoting the revolution of future technology. Healthcare is one of the promising applications in AI, which covers medical imaging, diagnosis, robotics, disease prediction, pharmacy, health management, and hospital management. Numbers of achievements that made in these fields overturn every aspect in traditional healthcare system. Therefore, to understand the state-of-art AI in healthcare, as well as the chances and obstacles in its development, the applications of AI in disease detection and outlook and the future trends of AI-aided disease prediction were discussed in this chapter.

Deep Learning for the Detection and Classification of Diabetic Retinopathy with an Improved Activation Function

Diabetic retinopathy (DR) is an eye disease triggered due to diabetes, which may lead to blindness. To prevent diabetic patients from becoming blind, early diagnosis and accurate detection of DR are vital. Deep learning models, such as convolutional neural networks (CNNs), are largely used in DR detection through the classification of blood vessel pixels from the remaining pixels. In this paper, an improved activation function was proposed for diagnosing DR from fundus images that automatically reduces loss and processing time. The DIARETDB0, DRIVE, CHASE, and Kaggle datasets were used to train and test the enhanced activation function in the different CNN models. The ResNet-152 model has the highest accuracy of 99.41% with the Kaggle dataset. This enhanced activation function is suitable for DR diagnosis from retinal fundus images.

Digital innovations for retinal care in diabetic retinopathy

AIM

The purpose of this review is to examine the applications of novel digital technology domains for the screening and management of patients with diabetic retinopathy (DR).

METHODS

A PubMed engine search was performed, using the terms "Telemedicine", "Digital health", "Telehealth", "Telescreening", "Artificial intelligence", "Deep learning", "Smartphone", "Triage", "Screening", "Home-based", "Monitoring", "Ophthalmology", "Diabetes", "Diabetic Retinopathy", "Retinal imaging". Full-text English language studies from January 1, 2010, to February 1, 2022, and reference lists were considered for the conceptual framework of this review.

RESULTS

Diabetes mellitus and its eye complications, including DR, are particularly well suited to digital technologies, providing an ideal model for telehealth initiatives and real-world applications. The current development in the adoption of telemedicine, artificial intelligence and remote monitoring as an alternative to or in addition to traditional forms of care will be discussed.

CONCLUSIONS

Advances in digital health have created an ecosystem ripe for telemedicine in the field of DR to thrive. Stakeholders and policymakers should adopt a participatory approach to ensure sustained implementation of these technologies after the COVID-19 pandemic. This article belongs to the Topical Collection "Diabetic Eye Disease", managed by Giuseppe Querques.

Automatic Diagnosis of Diabetic Retinopathy Stage Focusing Exclusively on Retinal Hemorrhage

Background and Objectives: The present study evaluated the detection of diabetic retinopathy (DR) using an automated fundus camera focusing exclusively on retinal hemorrhage (RH) using a deep convolutional neural network, which is a machine-learning technology. Materials and Methods: This investigation was conducted via a prospective and observational study. The study included 89 fundus ophthalmoscopy images. Seventy images passed an image quality review and were graded as showing no apparent DR (n = 51), mild nonproliferative DR (NPDR; n = 16), moderate NPDR (n = 1), severe NPDR (n = 1), and proliferative DR (n = 1) by three retinal experts according to the International Clinical Diabetic Retinopathy Severity scale. The RH numbers and areas were automatically detected and the results of two tests-the detection of mild-or-worse NPDR and the detection of moderate-or-worse NPDR-were examined. Results: The detection of mild-or-worse DR showed a sensitivity of 0.812 (95% confidence interval: 0.680-0.945), specificity of 0.888, and area under the curve (AUC) of 0.884, whereas the detection of moderate-or-worse DR showed a sensitivity of 1.0, specificity of 1.0, and AUC of 1.0. Conclusions: Automated diagnosis using artificial intelligence focusing exclusively on RH could be used to diagnose DR requiring ophthalmologist intervention.

Use of artificial intelligence in ophthalmology: a narrative review

BACKGROUND

Artificial intelligence (AI) deals with development of algorithms that seek to perceive one's environment and perform actions that maximize one's chance of successfully reaching one's predetermined goals.

OBJECTIVE

To provide an overview of the basic principles of AI and its main studies in the fields of glaucoma, retinopathy of prematurity, age-related macular degeneration and diabetic retinopathy. From this perspective, the limitations and potential challenges that have accompanied the implementation and development of this new technology within ophthalmology are presented.

DESIGN AND SETTING

Narrative review developed by a research group at the Universidade Federal de São Paulo (UNIFESP), São Paulo (SP), Brazil.

METHODS

We searched the literature on the main applications of AI within ophthalmology, using the keywords "artificial intelligence", "diabetic retinopathy", "macular degeneration age-related", "glaucoma" and "retinopathy of prematurity," covering the period from January 1, 2007, to May 3, 2021. We used the MEDLINE database (via PubMed) and the LILACS database (via Virtual Health Library) to identify relevant articles.

RESULTS

We retrieved 457 references, of which 47 were considered eligible for intensive review and critical analysis.

CONCLUSION

Use of technology, as embodied in AI algorithms, is a way of providing an increasingly accurate service and enhancing scientific research. This forms a source of complement and innovation in relation to the daily skills of ophthalmologists. Thus, AI adds technology to human expertise.

Health Economic Implications of Artificial Intelligence Implementation for Ophthalmology in Australia: A Systematic Review

PURPOSE

The health care industry is an inherently resource-intense sector. Emerging technologies such as artificial intelligence (AI) are at the forefront of advancements in health care. The health economic implications of this technology have not been clearly established and represent a substantial barrier to adoption both in Australia and globally. This review aims to determine the health economic impact of implementing AI to ophthalmology in Australia.

METHODS

A systematic search of the databases PubMed/MEDLINE, EMBASE, and CENTRAL was conducted to March 2022, before data collection and risk of bias analysis in accordance with preferred reporting items for systematic ceviews and meta-analyses 2020 guidelines (PROSPERO number CRD42022325511). Included were full-text primary research articles analyzing a population of patients who have or are being evaluated for an ophthalmological diagnosis, using a health economic assessment system to assess the cost-effectiveness of AI.

RESULTS

Seven articles were identified for inclusion. Economic viability was defined as direct cost to the patient that is equal to or less than costs incurred with human clinician assessment. Despite the lack of Australia-specific data, foreign analyses overwhelmingly showed that AI is just as economically viable, if not more so, than traditional human screening programs while maintaining comparable clinical effectiveness. This evidence was largely in the setting of diabetic retinopathy screening.

CONCLUSIONS

Primary Australian research is needed to accurately analyze the health economic implications of implementing AI on a large scale. Further research is also required to analyze the economic feasibility of adoption of AI technology in other areas of ophthalmology, such as glaucoma and cataract screening.

Evaluation of a New Neural Network Classifier for Diabetic Retinopathy

BACKGROUND

Medical image segmentation is a well-studied subject within the field of image processing. The goal of this research is to create an AI retinal screening grading system that is both accurate and fast. We introduce a new segmentation network which achieves state-of-the-art results on semantic segmentation of color fundus photographs. By applying the net-work to identify anatomical markers of diabetic retinopathy (DR) and diabetic macular edema (DME), we collect sufficient information to classify patients by grades R0 and R1 or above, M0 and M1.

METHODS

The AI grading system was trained on screening data to evaluate the presence of DR and DME. The core algorithm of the system is a deep learning network that segments relevant anatomical features in a retinal image. Patients were graded according to the standard NHS Diabetic Eye Screening Program feature-based grading protocol.

RESULTS

The algorithm performance was evaluated with a series of 6,981 patient retinal images from routine diabetic eye screenings. It correctly predicted 98.9% of retinopathy events and 95.5% of maculopathy events. Non-disease events prediction rate was 68.6% for retinopathy and 81.2% for maculopathy.

CONCLUSION

This novel deep learning model was trained and tested on patient data from annual diabetic retinopathy screenings can classify with high accuracy the DR and DME status of a person with diabetes. The system can be easily reconfigured according to any grading protocol, without running a long AI training procedure. The incorporation of the AI grading system can increase the graders' productivity and improve the final outcome accuracy of the screening process.

Trustworthy AI: Closing the gap between development and integration of AI systems in ophthalmic practice

An increasing number of artificial intelligence (AI) systems are being proposed in ophthalmology, motivated by the variety and amount of clinical and imaging data, as well as their potential benefits at the different stages of patient care. Despite achieving close or even superior performance to that of experts, there is a critical gap between development and integration of AI systems in ophthalmic practice. This work focuses on the importance of trustworthy AI to close that gap. We identify the main aspects or challenges that need to be considered along the AI design pipeline so as to generate systems that meet the requirements to be deemed trustworthy, including those concerning accuracy, resiliency, reliability, safety, and accountability. We elaborate on mechanisms and considerations to address those aspects or challenges, and define the roles and responsibilities of the different stakeholders involved in AI for ophthalmic care, i.e., AI developers, reading centers, healthcare providers, healthcare institutions, ophthalmological societies and working groups or committees, patients, regulatory bodies, and payers. Generating trustworthy AI is not a responsibility of a sole stakeholder. There is an impending necessity for a collaborative approach where the different stakeholders are represented along the AI design pipeline, from the definition of the intended use to post-market surveillance after regulatory approval. This work contributes to establish such multi-stakeholder interaction and the main action points to be taken so that the potential benefits of AI reach real-world ophthalmic settings.

Deep learning: applications in retinal and optic nerve diseases

Deep learning (DL) represents a paradigm-shifting, burgeoning field of research with emerging clinical applications in optometry. Unlike traditional programming, which relies on human-set specific rules, DL works by exposing the algorithm to a large amount of annotated data and allowing the software to develop its own set of rules (i.e. learn) by adjusting the parameters inside the model (network) during a training process in order to complete the task on its own. One major limitation of traditional programming is that, with complex tasks, it may require an extensive set of rules to accurately complete the assignment. Additionally, traditional programming can be susceptible to human bias from programmer experience. With the dramatic increase in the amount and the complexity of clinical data, DL has been utilised to automate data analysis and thus to assist clinicians in patient management. This review will present the latest advances in DL, for managing posterior eye diseases as well as DL-based solutions for patients with vision loss.