Real-time diabetic retinopathy screening by deep learning in a multisite national screening programme: a prospective interventional cohort study

The AI revolution in glaucoma: Bridging challenges with opportunities

Recent advancements in artificial intelligence (AI) herald transformative potentials for reshaping glaucoma clinical management, improving screening efficacy, sharpening diagnosis precision, and refining the detection of disease progression. However, incorporating AI into healthcare usages faces significant hurdles in terms of developing algorithms and putting them into practice. When creating algorithms, issues arise due to the intensive effort required to label data, inconsistent diagnostic standards, and a lack of thorough testing, which often limits the algorithms' widespread applicability. Additionally, the "black box" nature of AI algorithms may cause doctors to be wary or skeptical. When it comes to using these tools, challenges include dealing with lower-quality images in real situations and the systems' limited ability to work well with diverse ethnic groups and different diagnostic equipment. Looking ahead, new developments aim to protect data privacy through federated learning paradigms, improving algorithm generalizability by diversifying input data modalities, and augmenting datasets with synthetic imagery. The integration of smartphones appears promising for using AI algorithms in both clinical and non-clinical settings. Furthermore, bringing in large language models (LLMs) to act as interactive tool in medicine may signify a significant change in how healthcare will be delivered in the future. By navigating through these challenges and leveraging on these as opportunities, the field of glaucoma AI will not only have improved algorithmic accuracy and optimized data integration but also a paradigmatic shift towards enhanced clinical acceptance and a transformative improvement in glaucoma care.

Integrated image-based deep learning and language models for primary diabetes care

Primary diabetes care and diabetic retinopathy (DR) screening persist as major public health challenges due to a shortage of trained primary care physicians (PCPs), particularly in low-resource settings. Here, to bridge the gaps, we developed an integrated image-language system (DeepDR-LLM), combining a large language model (LLM module) and image-based deep learning (DeepDR-Transformer), to provide individualized diabetes management recommendations to PCPs. In a retrospective evaluation, the LLM module demonstrated comparable performance to PCPs and endocrinology residents when tested in English and outperformed PCPs and had comparable performance to endocrinology residents in Chinese. For identifying referable DR, the average PCP's accuracy was 81.0% unassisted and 92.3% assisted by DeepDR-Transformer. Furthermore, we performed a single-center real-world prospective study, deploying DeepDR-LLM. We compared diabetes management adherence of patients under the unassisted PCP arm (n = 397) with those under the PCP+DeepDR-LLM arm (n = 372). Patients with newly diagnosed diabetes in the PCP+DeepDR-LLM arm showed better self-management behaviors throughout follow-up (P < 0.05). For patients with referral DR, those in the PCP+DeepDR-LLM arm were more likely to adhere to DR referrals (P < 0.01). Additionally, DeepDR-LLM deployment improved the quality and empathy level of management recommendations. Given its multifaceted performance, DeepDR-LLM holds promise as a digital solution for enhancing primary diabetes care and DR screening.

Unmasking biases and navigating pitfalls in the ophthalmic artificial intelligence lifecycle: A narrative review

Over the past 2 decades, exponential growth in data availability, computational power, and newly available modeling techniques has led to an expansion in interest, investment, and research in Artificial Intelligence (AI) applications. Ophthalmology is one of many fields that seek to benefit from AI given the advent of telemedicine screening programs and the use of ancillary imaging. However, before AI can be widely deployed, further work must be done to avoid the pitfalls within the AI lifecycle. This review article breaks down the AI lifecycle into seven steps-data collection; defining the model task; data preprocessing and labeling; model development; model evaluation and validation; deployment; and finally, post-deployment evaluation, monitoring, and system recalibration-and delves into the risks for harm at each step and strategies for mitigating them.

Effects of artificial intelligence implementation on efficiency in medical imaging-a systematic literature review and meta-analysis

In healthcare, integration of artificial intelligence (AI) holds strong promise for facilitating clinicians' work, especially in clinical imaging. We aimed to assess the impact of AI implementation for medical imaging on efficiency in real-world clinical workflows and conducted a systematic review searching six medical databases. Two reviewers double-screened all records. Eligible records were evaluated for methodological quality. The outcomes of interest were workflow adaptation due to AI implementation, changes in time for tasks, and clinician workload. After screening 13,756 records, we identified 48 original studies to be incuded in the review. Thirty-three studies measured time for tasks, with 67% reporting reductions. Yet, three separate meta-analyses of 12 studies did not show significant effects after AI implementation. We identified five different workflows adapting to AI use. Most commonly, AI served as a secondary reader for detection tasks. Alternatively, AI was used as the primary reader for identifying positive cases, resulting in reorganizing worklists or issuing alerts. Only three studies scrutinized workload calculations based on the time saved through AI use. This systematic review and meta-analysis represents an assessment of the efficiency improvements offered by AI applications in real-world clinical imaging, predominantly revealing enhancements across the studies. However, considerable heterogeneity in available studies renders robust inferences regarding overall effectiveness in imaging tasks. Further work is needed on standardized reporting, evaluation of system integration, and real-world data collection to better understand the technological advances of AI in real-world healthcare workflows. Systematic review registration: Prospero ID CRD42022303439, International Registered Report Identifier (IRRID): RR2-10.2196/40485.

Digital ray: enhancing cataractous fundus images using style transfer generative adversarial networks to improve retinopathy detection

BACKGROUND/AIMS

The aim of this study was to develop and evaluate digital ray, based on preoperative and postoperative image pairs using style transfer generative adversarial networks (GANs), to enhance cataractous fundus images for improved retinopathy detection.

METHODS

For eligible cataract patients, preoperative and postoperative colour fundus photographs (CFP) and ultra-wide field (UWF) images were captured. Then, both the original CycleGAN and a modified CycleGAN (C2ycleGAN) framework were adopted for image generation and quantitatively compared using Frechet Inception Distance (FID) and Kernel Inception Distance (KID). Additionally, CFP and UWF images from another cataract cohort were used to test model performances. Different panels of ophthalmologists evaluated the quality, authenticity and diagnostic efficacy of the generated images.

RESULTS

A total of 959 CFP and 1009 UWF image pairs were included in model development. FID and KID indicated that images generated by C2ycleGAN presented significantly improved quality. Based on ophthalmologists' average ratings, the percentages of inadequate-quality images decreased from 32% to 18.8% for CFP, and from 18.7% to 14.7% for UWF. Only 24.8% and 13.8% of generated CFP and UWF images could be recognised as synthetic. The accuracy of retinopathy detection significantly increased from 78% to 91% for CFP and from 91% to 93% for UWF. For retinopathy subtype diagnosis, the accuracies also increased from 87%-94% to 91%-100% for CFP and from 87%-95% to 93%-97% for UWF.

CONCLUSION

Digital ray could generate realistic postoperative CFP and UWF images with enhanced quality and accuracy for overall detection and subtype diagnosis of retinopathies, especially for CFP.\ TRIAL REGISTRATION NUMBER: This study was registered with ClinicalTrials.gov (NCT05491798).

Implementation of Artificial Intelligence-Based Diabetic Retinopathy Screening in a Tertiary Care Hospital in Quebec: Prospective Validation Study

BACKGROUND

Diabetic retinopathy (DR) affects about 25% of people with diabetes in Canada. Early detection of DR is essential for preventing vision loss.

OBJECTIVE

We evaluated the real-world performance of an artificial intelligence (AI) system that analyzes fundus images for DR screening in a Quebec tertiary care center.

METHODS

We prospectively recruited adult patients with diabetes at the Centre hospitalier de l'Université de Montréal (CHUM) in Montreal, Quebec, Canada. Patients underwent dual-pathway screening: first by the Computer Assisted Retinal Analysis (CARA) AI system (index test), then by standard ophthalmological examination (reference standard). We measured the AI system's sensitivity and specificity for detecting referable disease at the patient level, along with its performance for detecting any retinopathy and diabetic macular edema (DME) at the eye level, and potential cost savings.

RESULTS

This study included 115 patients. CARA demonstrated a sensitivity of 87.5% (95% CI 71.9-95.0) and specificity of 66.2% (95% CI 54.3-76.3) for detecting referable disease at the patient level. For any retinopathy detection at the eye level, CARA showed 88.2% sensitivity (95% CI 76.6-94.5) and 71.4% specificity (95% CI 63.7-78.1). For DME detection, CARA had 100% sensitivity (95% CI 64.6-100) and 81.9% specificity (95% CI 75.6-86.8). Potential yearly savings from implementing CARA at the CHUM were estimated at CAD $245,635 (US $177,643.23, as of July 26, 2024) considering 5000 patients with diabetes.

CONCLUSIONS

Our study indicates that integrating a semiautomated AI system for DR screening demonstrates high sensitivity for detecting referable disease in a real-world setting. This system has the potential to improve screening efficiency and reduce costs at the CHUM, but more work is needed to validate it.

Polyphenol-Rich Extract of Apocynum venetum L. Leaves Protects Human Retinal Pigment Epithelial Cells against High Glucose-Induced Damage through Polyol Pathway and Autophagy

Diabetic retinopathy (DR) is a specific microvascular problem of diabetes, which is mainly caused by hyperglycemia and may lead to rapid vision loss. Dietary polyphenols have been reported to decrease the risk of DR. Apocynum venetum L. leaves are rich in polyphenolic compounds and are popular worldwide for their health benefits as a national tea drink. Building on previous findings of antioxidant activity and aldose reductase inhibition of A. venetum, this study investigated the chemical composition of polyphenol-rich extract of A. venetum leaves (AVL) and its protective mechanism on ARPE-19 cells in hyperglycemia. Ninety-three compounds were identified from AVL by LC-MS/MS, including sixty-eight flavonoids, twenty-one organic acids, and four coumarins. AVL regulated the polyol pathway by decreasing the expression of aldose reductase and the content of sorbitol, enhancing the Na+K+-ATPase activity, and weakening intracellular oxidative stress effectively; it also could regulate the expression of autophagy-related proteins via the AMPK/mTOR/ULK1 signaling pathway to maintain intracellular homeostasis. AVL could restore the polyol pathway, inhibit oxidative stress, and maintain intracellular autophagy to protect cellular morphology and improve DR. The study reveals the phytochemical composition and protective mechanisms of AVL against DR, which could be developed as a functional food and/or candidate pharmaceutical, aiming for retina protection in diabetic retinopathy.

Ocular biomarkers: useful incidental findings by deep learning algorithms in fundus photographs

BACKGROUND/OBJECTIVES

Artificial intelligence can assist with ocular image analysis for screening and diagnosis, but it is not yet capable of autonomous full-spectrum screening. Hypothetically, false-positive results may have unrealized screening potential arising from signals persisting despite training and/or ambiguous signals such as from biomarker overlap or high comorbidity. The study aimed to explore the potential to detect clinically useful incidental ocular biomarkers by screening fundus photographs of hypertensive adults using diabetic deep learning algorithms.

SUBJECTS/METHODS

Patients referred for treatment-resistant hypertension were imaged at a hospital unit in Perth, Australia, between 2016 and 2022. The same 45° colour fundus photograph selected for each of the 433 participants imaged was processed by three deep learning algorithms. Two expert retinal specialists graded all false-positive results for diabetic retinopathy in non-diabetic participants.

RESULTS

Of the 29 non-diabetic participants misclassified as positive for diabetic retinopathy, 28 (97%) had clinically useful retinal biomarkers. The models designed to screen for fewer diseases captured more incidental disease. All three algorithms showed a positive correlation between severity of hypertensive retinopathy and misclassified diabetic retinopathy.

CONCLUSIONS

The results suggest that diabetic deep learning models may be responsive to hypertensive and other clinically useful retinal biomarkers within an at-risk, hypertensive cohort. Observing that models trained for fewer diseases captured more incidental pathology increases confidence in signalling hypotheses aligned with using self-supervised learning to develop autonomous comprehensive screening. Meanwhile, non-referable and false-positive outputs of other deep learning screening models could be explored for immediate clinical use in other populations.

Advancing healthcare with artificial intelligence: diagnostic accuracy of machine learning algorithm in diagnosis of diabetic retinopathy in the Brazilian population

BACKGROUND

In healthcare systems in general, access to diabetic retinopathy (DR) screening is limited. Artificial intelligence has the potential to increase care delivery. Therefore, we trained and evaluated the diagnostic accuracy of a machine learning algorithm for automated detection of DR.

METHODS

We included color fundus photographs from individuals from 4 databases (primary and specialized care settings), excluding uninterpretable images. The datasets consist of images from Brazilian patients, which differs from previous work. This modification allows for a more tailored application of the model to Brazilian patients, ensuring that the nuances and characteristics of this specific population are adequately captured. The sample was fractionated in training (70%) and testing (30%) samples. A convolutional neural network was trained for image classification. The reference test was the combined decision from three ophthalmologists. The sensitivity, specificity, and area under the ROC curve of the algorithm for detecting referable DR (moderate non-proliferative DR; severe non-proliferative DR; proliferative DR and/or clinically significant macular edema) were estimated.

RESULTS

A total of 15,816 images (4590 patients) were included. The overall prevalence of any degree of DR was 26.5%. Compared with human evaluators (manual method of diagnosing DR performed by an ophthalmologist), the deep learning algorithm achieved an area under the ROC curve of 0.98 (95% CI 0.97-0.98), with a specificity of 94.6% (95% CI 93.8-95.3) and a sensitivity of 93.5% (95% CI 92.2-94.9) at the point of greatest efficiency to detect referable DR.

CONCLUSIONS

A large database showed that this deep learning algorithm was accurate in detecting referable DR. This finding aids to universal healthcare systems like Brazil, optimizing screening processes and can serve as a tool for improving DR screening, making it more agile and expanding care access.

Visual interpretability of image-based classification models by generative latent space disentanglement applied to in vitro fertilization

The success of deep learning in identifying complex patterns exceeding human intuition comes at the cost of interpretability. Non-linear entanglement of image features makes deep learning a "black box" lacking human meaningful explanations for the models' decision. We present DISCOVER, a generative model designed to discover the underlying visual properties driving image-based classification models. DISCOVER learns disentangled latent representations, where each latent feature encodes a unique classification-driving visual property. This design enables "human-in-the-loop" interpretation by generating disentangled exaggerated counterfactual explanations. We apply DISCOVER to interpret classification of in vitro fertilization embryo morphology quality. We quantitatively and systematically confirm the interpretation of known embryo properties, discover properties without previous explicit measurements, and quantitatively determine and empirically verify the classification decision of specific embryo instances. We show that DISCOVER provides human-interpretable understanding of "black box" classification models, proposes hypotheses to decipher underlying biomedical mechanisms, and provides transparency for the classification of individual predictions.

Impact of Gold-Standard Label Errors on Evaluating Performance of Deep Learning Models in Diabetic Retinopathy Screening: Nationwide Real-World Validation Study

BACKGROUND

For medical artificial intelligence (AI) training and validation, human expert labels are considered the gold standard that represents the correct answers or desired outputs for a given data set. These labels serve as a reference or benchmark against which the model's predictions are compared.

OBJECTIVE

This study aimed to assess the accuracy of a custom deep learning (DL) algorithm on classifying diabetic retinopathy (DR) and further demonstrate how label errors may contribute to this assessment in a nationwide DR-screening program.

METHODS

Fundus photographs from the Lifeline Express, a nationwide DR-screening program, were analyzed to identify the presence of referable DR using both (1) manual grading by National Health Service England-certificated graders and (2) a DL-based DR-screening algorithm with validated good lab performance. To assess the accuracy of labels, a random sample of images with disagreement between the DL algorithm and the labels was adjudicated by ophthalmologists who were masked to the previous grading results. The error rates of labels in this sample were then used to correct the number of negative and positive cases in the entire data set, serving as postcorrection labels. The DL algorithm's performance was evaluated against both pre- and postcorrection labels.

RESULTS

The analysis included 736,083 images from 237,824 participants. The DL algorithm exhibited a gap between the real-world performance and the lab-reported performance in this nationwide data set, with a sensitivity increase of 12.5% (from 79.6% to 92.5%, P<.001) and a specificity increase of 6.9% (from 91.6% to 98.5%, P<.001). In the random sample, 63.6% (560/880) of negative images and 5.2% (140/2710) of positive images were misclassified in the precorrection human labels. High myopia was the primary reason for misclassifying non-DR images as referable DR images, while laser spots were predominantly responsible for misclassified referable cases. The estimated label error rate for the entire data set was 1.2%. The label correction was estimated to bring about a 12.5% enhancement in the estimated sensitivity of the DL algorithm (P<.001).

CONCLUSIONS

Label errors based on human image grading, although in a small percentage, can significantly affect the performance evaluation of DL algorithms in real-world DR screening.

Retina Fundus Photograph-Based Artificial Intelligence Algorithms in Medicine: A Systematic Review

We conducted a systematic review of research in artificial intelligence (AI) for retinal fundus photographic images. We highlighted the use of various AI algorithms, including deep learning (DL) models, for application in ophthalmic and non-ophthalmic (i.e., systemic) disorders. We found that the use of AI algorithms for the interpretation of retinal images, compared to clinical data and physician experts, represents an innovative solution with demonstrated superior accuracy in identifying many ophthalmic (e.g., diabetic retinopathy (DR), age-related macular degeneration (AMD), optic nerve disorders), and non-ophthalmic disorders (e.g., dementia, cardiovascular disease). There has been a significant amount of clinical and imaging data for this research, leading to the potential incorporation of AI and DL for automated analysis. AI has the potential to transform healthcare by improving accuracy, speed, and workflow, lowering cost, increasing access, reducing mistakes, and transforming healthcare worker education and training.

Automated early detection of acute retinal necrosis from ultra-widefield color fundus photography using deep learning

BACKGROUND

Acute retinal necrosis (ARN) is a relatively rare but highly damaging and potentially sight-threatening type of uveitis caused by infection with the human herpesvirus. Without timely diagnosis and appropriate treatment, ARN can lead to severe vision loss. We aimed to develop a deep learning framework to distinguish ARN from other types of intermediate, posterior, and panuveitis using ultra-widefield color fundus photography (UWFCFP).

METHODS

We conducted a two-center retrospective discovery and validation study to develop and validate a deep learning model called DeepDrARN for automatic uveitis detection and differentiation of ARN from other uveitis types using 11,508 UWFCFPs from 1,112 participants. Model performance was evaluated with the area under the receiver operating characteristic curve (AUROC), the area under the precision and recall curves (AUPR), sensitivity and specificity, and compared with seven ophthalmologists.

RESULTS

DeepDrARN for uveitis screening achieved an AUROC of 0.996 (95% CI: 0.994-0.999) in the internal validation cohort and demonstrated good generalizability with an AUROC of 0.973 (95% CI: 0.956-0.990) in the external validation cohort. DeepDrARN also demonstrated excellent predictive ability in distinguishing ARN from other types of uveitis with AUROCs of 0.960 (95% CI: 0.943-0.977) and 0.971 (95% CI: 0.956-0.986) in the internal and external validation cohorts. DeepDrARN was also tested in the differentiation of ARN, non-ARN uveitis (NAU) and normal subjects, with sensitivities of 88.9% and 78.7% and specificities of 93.8% and 89.1% in the internal and external validation cohorts, respectively. The performance of DeepDrARN is comparable to that of ophthalmologists and even exceeds the average accuracy of seven ophthalmologists, showing an improvement of 6.57% in uveitis screening and 11.14% in ARN identification.

CONCLUSIONS

Our study demonstrates the feasibility of deep learning algorithms in enabling early detection, reducing treatment delays, and improving outcomes for ARN patients.

Multimodal Retinal Imaging Classification for Parkinson's Disease Using a Convolutional Neural Network

Purpose

Changes in retinal structure and microvasculature are connected to parallel changes in the brain. Two recent studies described machine learning algorithms trained on retinal images and quantitative data that identified Alzheimer's dementia and mild cognitive impairment with high accuracy. Prior studies also demonstrated retinal differences in individuals with PD. Herein, we developed a convolutional neural network (CNN) to classify multimodal retinal imaging from either a Parkinson's disease (PD) or control group.

Methods

We trained a CNN to receive retinal image inputs of optical coherence tomography (OCT) ganglion cell-inner plexiform layer (GC-IPL) thickness color maps, OCT angiography 6 × 6-mm en face macular images of the superficial capillary plexus, and ultra-widefield (UWF) fundus color and autofluorescence photographs to classify the retinal imaging as PD or control. The model consists of a shared pretrained VGG19 feature extractor and image-specific feature transformations which converge to a single output. Model results were assessed using receiver operating characteristic (ROC) curves and bootstrapped 95% confidence intervals for area under the ROC curve (AUC) values.

Results

In total, 371 eyes of 249 control subjects and 75 eyes of 52 PD subjects were used for training, validation, and testing. Our best CNN variant achieved an AUC of 0.918. UWF color photographs were the most effective imaging input, and GC-IPL thickness maps were the least contributory.

Conclusions

Using retinal images, our pilot CNN was able to identify individuals with PD and serves as a proof of concept to spur the collection of larger imaging datasets needed for clinical-grade algorithms.

Translational Relevance

Developing machine learning models for automated detection of Parkinson's disease from retinal imaging could lead to earlier and more widespread diagnoses.

Resilience to diabetic retinopathy

Chronic elevation of blood glucose at first causes relatively minor changes to the neural and vascular components of the retina. As the duration of hyperglycemia persists, the nature and extent of damage increases and becomes readily detectable. While this second, overt manifestation of diabetic retinopathy (DR) has been studied extensively, what prevents maximal damage from the very start of hyperglycemia remains largely unexplored. Recent studies indicate that diabetes (DM) engages mitochondria-based defense during the retinopathy-resistant phase, and thereby enables the retina to remain healthy in the face of hyperglycemia. Such resilience is transient, and its deterioration results in progressive accumulation of retinal damage. The concepts that co-emerge with these discoveries set the stage for novel intellectual and therapeutic opportunities within the DR field. Identification of biomarkers and mediators of protection from DM-mediated damage will enable development of resilience-based therapies that will indefinitely delay the onset of DR.

Automated Identification of Different Severity Levels of Diabetic Retinopathy Using a Handheld Fundus Camera and Single-Image Protocol

Purpose

To evaluate the performance of artificial intelligence (AI) systems embedded in a mobile, handheld retinal camera, with a single retinal image protocol, in detecting both diabetic retinopathy (DR) and more-than-mild diabetic retinopathy (mtmDR).

Design

Multicenter cross-sectional diagnostic study, conducted at 3 diabetes care and eye care facilities.

Participants

A total of 327 individuals with diabetes mellitus (type 1 or type 2) underwent a retinal imaging protocol enabling expert reading and automated analysis.

Methods

Participants underwent fundus photographs using a portable retinal camera (Phelcom Eyer). The captured images were automatically analyzed by deep learning algorithms retinal alteration score (RAS) and diabetic retinopathy alteration score (DRAS), consisting of convolutional neural networks trained on EyePACS data sets and fine-tuned using data sets of portable device fundus images. The ground truth was the classification of DR corresponding to adjudicated expert reading, performed by 3 certified ophthalmologists.

Main Outcome Measures

Primary outcome measures included the sensitivity and specificity of the AI system in detecting DR and/or mtmDR using a single-field, macula-centered fundus photograph for each eye, compared with a rigorous clinical reference standard comprising the reading center grading of 2-field imaging protocol using the International Classification of Diabetic Retinopathy severity scale.

Results

Of 327 analyzed patients (mean age, 57.0 ± 16.8 years; mean diabetes duration, 16.3 ± 9.7 years), 307 completed the study protocol. Sensitivity and specificity of the AI system were high in detecting any DR with DRAS (sensitivity, 90.48% [95% confidence interval (CI), 84.99%-94.46%]; specificity, 90.65% [95% CI, 84.54%-94.93%]) and mtmDR with the combination of RAS and DRAS (sensitivity, 90.23% [95% CI, 83.87%-94.69%]; specificity, 85.06% [95% CI, 78.88%-90.00%]). The area under the receiver operating characteristic curve was 0.95 for any DR and 0.89 for mtmDR.

Conclusions

This study showed a high accuracy for the detection of DR in different levels of severity with a single retinal photo per eye in an all-in-one solution, composed of a portable retinal camera powered by AI. Such a strategy holds great potential for increasing coverage rates of screening programs, contributing to prevention of avoidable blindness.

Financial Disclosures

F.K.M. is a medical consultant for Phelcom Technologies. J.A.S. is Chief Executive Officer and proprietary of Phelcom Technologies. D.L. is Chief Technology Officer and proprietary of Phelcom Technologies. P.V.P. is an employee at Phelcom Technologies.

Diabetes management in the era of artificial intelligence

Artificial intelligence is growing quickly, and its application in the global diabetes pandemic has the potential to completely change the way this chronic illness is identified and treated. Machine learning methods have been used to construct algorithms supporting predictive models for the risk of getting diabetes or its complications. Social media and Internet forums also increase patient participation in diabetes care. Diabetes resource usage optimisation has benefited from technological improvements. As a lifestyle therapy intervention, digital therapies have made a name for themselves in the treatment of diabetes. Artificial intelligence will cause a paradigm shift in diabetes care, moving away from current methods and toward the creation of focused, data-driven precision treatment.

Deep learning detection of diabetic retinopathy in Scotland's diabetic eye screening programme

BACKGROUND/AIMS

Support vector machine-based automated grading (known as iGradingM) has been shown to be safe, cost-effective and robust in the diabetic retinopathy (DR) screening (DES) programme in Scotland. It triages screening episodes as gradable with no DR versus manual grading required. The study aim was to develop a deep learning-based autograder using images and gradings from DES and to compare its performance with that of iGradingM.

METHODS

Retinal images, quality assurance (QA) data and routine DR grades were obtained from national datasets in 179 944 patients for years 2006-2016. QA grades were available for 744 images. We developed a deep learning-based algorithm to detect whether either eye contained ungradable images or any DR. The sensitivity and specificity were evaluated against consensus QA grades and routine grades.

RESULTS

Images used in QA which were ungradable or with DR were detected by deep learning with better specificity compared with manual graders (p<0.001) and with iGradingM (p<0.001) at the same sensitivities. Any DR according to the DES final grade was detected with 89.19% (270 392/303 154) sensitivity and 77.41% (500 945/647 158) specificity. Observable disease and referable disease were detected with sensitivities of 96.58% (16 613/17 201) and 98.48% (22 600/22 948), respectively. Overall, 43.84% of screening episodes would require manual grading.

CONCLUSION

A deep learning-based system for DR grading was evaluated in QA data and images from 11 years in 50% of people attending a national DR screening programme. The system could reduce the manual grading workload at the same sensitivity compared with the current automated grading system.

Novel artificial intelligence algorithms for diabetic retinopathy and diabetic macular edema

BACKGROUND

Diabetic retinopathy (DR) and diabetic macular edema (DME) are major causes of visual impairment that challenge global vision health. New strategies are needed to tackle these growing global health problems, and the integration of artificial intelligence (AI) into ophthalmology has the potential to revolutionize DR and DME management to meet these challenges.

MAIN TEXT

This review discusses the latest AI-driven methodologies in the context of DR and DME in terms of disease identification, patient-specific disease profiling, and short-term and long-term management. This includes current screening and diagnostic systems and their real-world implementation, lesion detection and analysis, disease progression prediction, and treatment response models. It also highlights the technical advancements that have been made in these areas. Despite these advancements, there are obstacles to the widespread adoption of these technologies in clinical settings, including regulatory and privacy concerns, the need for extensive validation, and integration with existing healthcare systems. We also explore the disparity between the potential of AI models and their actual effectiveness in real-world applications.

CONCLUSION

AI has the potential to revolutionize the management of DR and DME, offering more efficient and precise tools for healthcare professionals. However, overcoming challenges in deployment, regulatory compliance, and patient privacy is essential for these technologies to realize their full potential. Future research should aim to bridge the gap between technological innovation and clinical application, ensuring AI tools integrate seamlessly into healthcare workflows to enhance patient outcomes.

Image quality comparison of AirDoc portable retina camera versus eyer in a diabetic retinopathy screening program

BACKGROUND

Diabetic retinopathy (DR) stands as the foremost cause of preventable blindness in adults. Despite efforts to expand DR screening coverage in the Brazilian public healthcare system, challenges persist due to various factors including social, medical, and financial constraints. Our objective was to evaluate the quality of images obtained with the AirDoc, a novel device, compared to Eyer portable camera which has already been clinically validated.

METHODS

Images were captured by two portable retinal devices: AirDoc and Eyer. The included patients had their fundus images obtained in a screening program conducted in Blumenau, Santa Catarina. Two retina specialists independently assessed image's quality. A comparison was performed between both devices regarding image quality and the presence of artifacts.

RESULTS

The analysis included 129 patients (mean age of 61 years), with 29 (43.28%) male and an average disease duration of 11.1 ± 8 years. In Ardoc, 21 (16.28%) images were classified as poor quality, with 88 (68%) presenting artifacts; in Eyer, 4 (3.1%) images were classified as poor quality, with 94 (72.87%) presenting artifacts.

CONCLUSIONS

Although both Eyer and AirDoc devices show potential as screening tools, the AirDoc images displayed higher rates of ungradable and low-quality images, that may directly affect the DR and DME grading. We must acknowledge the limitations of our study, including the relatively small sample size. Therefore, the interpretations of our analyses should be approached with caution, and further investigations with larger patient cohorts are warranted to validate our findings.

Real-world artificial intelligence-based interpretation of fundus imaging as part of an eyewear prescription renewal protocol

OBJECTIVE

A real-world evaluation of the diagnostic accuracy of the Opthai® software for artificial intelligence-based detection of fundus image abnormalities in the context of the French eyewear prescription renewal protocol (RNO).

METHODS

A single-center, retrospective review of the sensitivity and specificity of the software in detecting fundus abnormalities among consecutive patients seen in our ophthalmology center in the context of the RNO protocol from July 28 through October 22, 2021. We compared abnormalities detected by the software operated by ophthalmic technicians (index test) to diagnoses confirmed by the ophthalmologist following additional examinations and/or consultation (reference test).

RESULTS

The study included 2056 eyes/fundus images of 1028 patients aged 6-50years. The software detected fundus abnormalities in 149 (7.2%) eyes or 107 (10.4%) patients. After examining the same fundus images, the ophthalmologist detected abnormalities in 35 (1.7%) eyes or 20 (1.9%) patients. The ophthalmologist did not detect abnormalities in fundus images deemed normal by the software. The most frequent diagnoses made by the ophthalmologist were glaucoma suspect (0.5% of eyes), peripapillary atrophy (0.44% of eyes), and drusen (0.39% of eyes). The software showed an overall sensitivity of 100% (95% CI 0.879-1.00) and an overall specificity of 94.4% (95% CI 0.933-0.953). The majority of false-positive software detections (5.6%) were glaucoma suspect, with the differential diagnosis of large physiological optic cups. Immediate OCT imaging by the technician allowed diagnosis by the ophthalmologist without separate consultation for 43/53 (81%) patients.

CONCLUSION

Ophthalmic technicians can use this software for highly-sensitive screening for fundus abnormalities that require evaluation by an ophthalmologist.

Deep Learning Approaches for Detecting of Nascent Geographic Atrophy in Age-Related Macular Degeneration

Purpose

Nascent geographic atrophy (nGA) refers to specific features seen on OCT B-scans, which are strongly associated with the future development of geographic atrophy (GA). This study sought to develop a deep learning model to screen OCT B-scans for nGA that warrant further manual review (an artificial intelligence [AI]-assisted approach), and to determine the extent of reduction in OCT B-scan load requiring manual review while maintaining near-perfect nGA detection performance.

Design

Development and evaluation of a deep learning model.

Participants

One thousand eight hundred and eighty four OCT volume scans (49 B-scans per volume) without neovascular age-related macular degeneration from 280 eyes of 140 participants with bilateral large drusen at baseline, seen at 6-monthly intervals up to a 36-month period (from which 40 eyes developed nGA).

Methods

OCT volume and B-scans were labeled for the presence of nGA. Their presence at the volume scan level provided the ground truth for training a deep learning model to identify OCT B-scans that potentially showed nGA requiring manual review. Using a threshold that provided a sensitivity of 0.99, the B-scans identified were assigned the ground truth label with the AI-assisted approach. The performance of this approach for detecting nGA across all visits, or at the visit of nGA onset, was evaluated using fivefold cross-validation.

Main Outcome Measures

Sensitivity for detecting nGA, and proportion of OCT B-scans requiring manual review.

Results

The AI-assisted approach (utilizing outputs from the deep learning model to guide manual review) had a sensitivity of 0.97 (95% confidence interval [CI] = 0.93-1.00) and 0.95 (95% CI = 0.87-1.00) for detecting nGA across all visits and at the visit of nGA onset, respectively, when requiring manual review of only 2.7% and 1.9% of selected OCT B-scans, respectively.

Conclusions

A deep learning model could be used to enable near-perfect detection of nGA onset while reducing the number of OCT B-scans requiring manual review by over 50-fold. This AI-assisted approach shows promise for substantially reducing the current burden of manual review of OCT B-scans to detect this crucial feature that portends future development of GA.

Financial Disclosures

Proprietary or commercial disclosure may be found in the Footnotes and Disclosures at the end of this article.

Development of deep-learning models for real-time anaerobic threshold and peak VO2 prediction during cardiopulmonary exercise testing

AIMS

Exercise intolerance is a clinical feature of patients with heart failure (HF). Cardiopulmonary exercise testing (CPET) is the first-line examination for assessing exercise capacity in patients with HF. However, the need for extensive experience in assessing anaerobic threshold (AT) and the potential risk associated with the excessive exercise load when measuring peak oxygen uptake (peak VO2) limit the utility of CPET. This study aimed to use deep-learning approaches to identify AT in real time during testing (defined as real-time AT) and to predict peak VO2 at real-time AT.

METHODS AND RESULTS

This study included the time-series data of CPET recorded at the Department of Cardiovascular Medicine, Kyushu University Hospital. Two deep neural network models were developed to: (i) estimate the AT probability using breath-by-breath data and (ii) predict peak VO2 using the data at the real-time AT. The eligible CPET contained 1472 records of 1053 participants aged 18-90 years and 20% were used for model evaluation. The developed model identified real-time AT with 0.82 for correlation coefficient (Corr) and 1.20 mL/kg/min for mean absolute error (MAE), and the corresponding AT time with 0.86 for Corr and 0.66 min for MAE. The peak VO2 prediction model achieved 0.87 for Corr and 2.25 mL/kg/min for MAE.

CONCLUSION

Deep-learning models for real-time CPET analysis can accurately identify AT and predict peak VO2. The developed models can be a competent assistant system to assess a patient's condition in real time, expanding CPET utility.

CRA-Net: Transformer guided category-relation attention network for diabetic retinopathy grading

Automated grading of diabetic retinopathy (DR) is an important means for assisting clinical diagnosis and preventing further retinal damage. However, imbalances and similarities between categories in the DR dataset make it highly challenging to accurately grade the severity of the condition. Furthermore, DR images encompass various lesions, and the pathological relationship information among these lesions can be easily overlooked. For instance, under different severity levels, the varying contributions of different lesions to accurate model grading differ significantly. To address the aforementioned issues, we design a transformer guided category-relation attention network (CRA-Net). Specifically, we propose a novel category attention block that enhances feature information within the class from the perspective of DR image categories, thereby alleviating class imbalance problems. Additionally, we design a lesion relation attention block that captures relationships between lesions by incorporating attention mechanisms in two primary aspects: capsule attention models the relative importance of different lesions, allowing the model to focus on more "informative" ones. Spatial attention captures the global position relationship between lesion features under transformer guidance, facilitating more accurate localization of lesions. Experimental and ablation studies on two datasets DDR and APTOS 2019 demonstrate the effectiveness of CRA-Net and obtain competitive performance.

Application of artificial intelligence system for screening multiple fundus diseases in Chinese primary healthcare settings: a real-world, multicentre and cross-sectional study of 4795 cases

BACKGROUND/AIMS

This study evaluates the performance of the Airdoc retinal artificial intelligence system (ARAS) for detecting multiple fundus diseases in real-world scenarios in primary healthcare settings and investigates the fundus disease spectrum based on ARAS.

METHODS

This real-world, multicentre, cross-sectional study was conducted in Shanghai and Xinjiang, China. Six primary healthcare settings were included in this study. Colour fundus photographs were taken and graded by ARAS and retinal specialists. The performance of ARAS is described by its accuracy, sensitivity, specificity and positive and negative predictive values. The spectrum of fundus diseases in primary healthcare settings has also been investigated.

RESULTS

A total of 4795 participants were included. The median age was 57.0 (IQR 39.0-66.0) years, and 3175 (66.2%) participants were female. The accuracy, specificity and negative predictive value of ARAS for detecting normal fundus and 14 retinal abnormalities were high, whereas the sensitivity and positive predictive value varied in detecting different abnormalities. The proportion of retinal drusen, pathological myopia and glaucomatous optic neuropathy was significantly higher in Shanghai than in Xinjiang. Moreover, the percentages of referable diabetic retinopathy, retinal vein occlusion and macular oedema in middle-aged and elderly people in Xinjiang were significantly higher than in Shanghai.

CONCLUSION

This study demonstrated the dependability of ARAS for detecting multiple retinal diseases in primary healthcare settings. Implementing the AI-assisted fundus disease screening system in primary healthcare settings might be beneficial in reducing regional disparities in medical resources. However, the ARAS algorithm must be improved to achieve better performance.

TRIAL REGISTRATION NUMBER

NCT04592068.

Economic evaluation for medical artificial intelligence: accuracy vs. cost-effectiveness in a diabetic retinopathy screening case

Artificial intelligence (AI) models have shown great accuracy in health screening. However, for real-world implementation, high accuracy may not guarantee cost-effectiveness. Improving AI's sensitivity finds more high-risk patients but may raise medical costs while increasing specificity reduces unnecessary referrals but may weaken detection capability. To evaluate the trade-off between AI model performance and the long-running cost-effectiveness, we conducted a cost-effectiveness analysis in a nationwide diabetic retinopathy (DR) screening program in China, comprising 251,535 participants with diabetes over 30 years. We tested a validated AI model in 1100 different diagnostic performances (presented as sensitivity/specificity pairs) and modeled annual screening scenarios. The status quo was defined as the scenario with the most accurate AI performance. The incremental cost-effectiveness ratio (ICER) was calculated for other scenarios against the status quo as cost-effectiveness metrics. Compared to the status quo (sensitivity/specificity: 93.3%/87.7%), six scenarios were cost-saving and seven were cost-effective. To achieve cost-saving or cost-effective, the AI model should reach a minimum sensitivity of 88.2% and specificity of 80.4%. The most cost-effective AI model exhibited higher sensitivity (96.3%) and lower specificity (80.4%) than the status quo. In settings with higher DR prevalence and willingness-to-pay levels, the AI needed higher sensitivity for optimal cost-effectiveness. Urban regions and younger patient groups also required higher sensitivity in AI-based screening. In real-world DR screening, the most accurate AI model may not be the most cost-effective. Cost-effectiveness should be independently evaluated, which is most likely to be affected by the AI's sensitivity.

Present and future screening programs for diabetic retinopathy: a narrative review

Diabetes is a prevalent global concern, with an estimated 12% of the global adult population affected by 2045. Diabetic retinopathy (DR), a sight-threatening complication, has spurred diverse screening approaches worldwide due to advances in DR knowledge, rapid technological developments in retinal imaging and variations in healthcare resources.Many high income countries have fully implemented or are on the verge of completing a national Diabetic Eye Screening Programme (DESP). Although there have been some improvements in DR screening in Africa, Asia, and American countries further progress is needed. In low-income countries, only one out of 29, partially implemented a DESP, while 21 out of 50 lower-middle-income countries have started the DR policy cycle. Among upper-middle-income countries, a third of 59 nations have advanced in DR agenda-setting, with five having a comprehensive national DESP and 11 in the early stages of implementation.Many nations use 2-4 fields fundus images, proven effective with 80-98% sensitivity and 86-100% specificity compared to the traditional seven-field evaluation for DR. A cell phone based screening with a hand held retinal camera presents a potential low-cost alternative as imaging device. While this method in low-resource settings may not entirely match the sensitivity and specificity of seven-field stereoscopic photography, positive outcomes are observed.Individualized DR screening intervals are the standard in many high-resource nations. In countries that lacks a national DESP and resources, screening are more sporadic, i.e. screening intervals are not evidence-based and often less frequently, which can lead to late recognition of treatment required DR.The rising global prevalence of DR poses an economic challenge to nationwide screening programs AI-algorithms have showed high sensitivity and specificity for detection of DR and could provide a promising solution for the future screening burden.In summary, this narrative review enlightens on the epidemiology of DR and the necessity for effective DR screening programs. Worldwide evolution in existing approaches for DR screening has showed promising results but has also revealed limitations. Technological advancements, such as handheld imaging devices, tele ophthalmology and artificial intelligence enhance cost-effectiveness, but also the accessibility of DR screening in countries with low resources or where distance to or a shortage of ophthalmologists exists.

Validation of a deep learning system for the detection of diabetic retinopathy in Indigenous Australians

BACKGROUND/AIMS

Deep learning systems (DLSs) for diabetic retinopathy (DR) detection show promising results but can underperform in racial and ethnic minority groups, therefore external validation within these populations is critical for health equity. This study evaluates the performance of a DLS for DR detection among Indigenous Australians, an understudied ethnic group who suffer disproportionately from DR-related blindness.

METHODS

We performed a retrospective external validation study comparing the performance of a DLS against a retinal specialist for the detection of more-than-mild DR (mtmDR), vision-threatening DR (vtDR) and all-cause referable DR. The validation set consisted of 1682 consecutive, single-field, macula-centred retinal photographs from 864 patients with diabetes (mean age 54.9 years, 52.4% women) at an Indigenous primary care service in Perth, Australia. Three-person adjudication by a panel of specialists served as the reference standard.

RESULTS

For mtmDR detection, sensitivity of the DLS was superior to the retina specialist (98.0% (95% CI, 96.5 to 99.4) vs 87.1% (95% CI, 83.6 to 90.6), McNemar's test p<0.001) with a small reduction in specificity (95.1% (95% CI, 93.6 to 96.4) vs 97.0% (95% CI, 95.9 to 98.0), p=0.006). For vtDR, the DLS's sensitivity was again superior to the human grader (96.2% (95% CI, 93.4 to 98.6) vs 84.4% (95% CI, 79.7 to 89.2), p<0.001) with a slight drop in specificity (95.8% (95% CI, 94.6 to 96.9) vs 97.8% (95% CI, 96.9 to 98.6), p=0.002). For all-cause referable DR, there was a substantial increase in sensitivity (93.7% (95% CI, 91.8 to 95.5) vs 74.4% (95% CI, 71.1 to 77.5), p<0.001) and a smaller reduction in specificity (91.7% (95% CI, 90.0 to 93.3) vs 96.3% (95% CI, 95.2 to 97.4), p<0.001).

CONCLUSION

The DLS showed improved sensitivity and similar specificity compared with a retina specialist for DR detection. This demonstrates its potential to support DR screening among Indigenous Australians, an underserved population with a high burden of diabetic eye disease.

Research progress on ocular complications caused by type 2 diabetes mellitus and the function of tears and blepharons

The purpose of this study was to explore the related research progress of ocular complications (OCs) caused by type 2 diabetes mellitus (T2DM), tear and tarsal function, and the application of deep learning (DL) in the diagnosis of diabetes and OCs caused by it, to provide reference for the prevention and control of OCs in T2DM patients. This study reviewed the pathogenesis and treatment of diabetes retinopathy, keratopathy, dry eye disease, glaucoma, and cataract, analyzed the relationship between OCs and tear function and tarsal function, and discussed the application value of DL in the diagnosis of diabetes and OCs. Diabetes retinopathy is related to hyperglycemia, angiogenic factors, oxidative stress, hypertension, hyperlipidemia, and other factors. The increase in water content in the corneal stroma leads to corneal relaxation, loss of transparency, and elasticity, and can lead to the occurrence of corneal lesions. Dry eye syndrome is related to abnormal stability of the tear film and imbalance in neural and immune regulation. Elevated intraocular pressure, inflammatory reactions, atrophy of the optic nerve head, and damage to optic nerve fibers are the causes of glaucoma. Cataract is a common eye disease in the elderly, which is a visual disorder caused by lens opacity. Oxidative stress is an important factor in the occurrence of cataracts. In clinical practice, blood sugar control, laser therapy, and drug therapy are used to control the above eye complications. The function of tear and tarsal plate will be affected by eye diseases. Retinopathy and dry eye disease caused by diabetes will cause dysfunction of tear and tarsal plate, which will affect the eye function of patients. Furthermore, DL can automatically diagnose and classify eye diseases, automatically analyze fundus images, and accurately diagnose diabetes retinopathy, macular degeneration, and other diseases by analyzing and processing eye images and data. The treatment of T2DM is difficult and prone to OCs, which seriously threatens the normal life of patients. The occurrence of OCs is closely related to abnormal tear and tarsal function. Based on DL, clinical diagnosis and treatment of diabetes and its OCs can be carried out, which has positive application value.

[Prevalence of diabetic retinopathy assessed using two-field mydriatic fundus photography]

Early detection of diabetic retinopathy (DR) is an urgent ophthalmological problem in Russia and globally.

PURPOSE

This study assesses the prevalence of asymptomatic retinopathy and attempts to identify risk groups for its development in patients with type 1 and 2 diabetes mellitus (T1DM and T2DM).

MATERIAL AND METHODS

The study involved clinics from 5 cities in the Russian Federation and it included 367 patients with DM, 34.88% men and 65.12% women, aged 50.88±20.55 years. 34.88% of patients suffered from T1DM, 65.12% suffered from T2DM, the average duration of the disease was 9.02±7.22 years. 58.31% of patients had a history of arterial hypertension, 13.08% had a history of smoking. The primary endpoint was the frequency of detection of diabetic changes in the eye fundus of patients with T1DM and T2DM in general; the secondary endpoint - same but separately, and for T2DM patients depending on the duration of the disease. The exploratory endpoint was the assessment of the influence of various factors on the development of DR. The patients underwent visometry (modified ETDRS table), biomicroscopy, mydriatic fundus photography according to the «2 fields» protocol.

RESULTS

The average detection rate of DR was 12.26%, primarily observed in patients with T2DM (13.81%), women (9.26%), in both eyes (8.17%). Among patients with DR, 26 (19.55%) had glycated hemoglobin (HbA1c) level exceeding 7.5% (p=0.002), indicating a direct relationship between this indicator and the incidence of DR. Logistic regression analysis showed that the duration of diabetes of more than 10 years has a statistically significant effect on the development of DR. In the modified model for odds estimation, the likelihood of developing DR is increased by the duration of DM for more than 10 years; increased blood pressure; HbA1c level >7.5%.

CONCLUSION

The obtained results, some of which will be presented in subsequent publications, highlight the effectiveness of using two-field mydriatic fundus photography as a screening for DR.

Population impact and cost-effectiveness of artificial intelligence-based diabetic retinopathy screening in people living with diabetes in Australia: a cost effectiveness analysis

Background

We aimed to evaluate the cost-effectiveness of an artificial intelligence-(AI) based diabetic retinopathy (DR) screening system in the primary care setting for both non-Indigenous and Indigenous people living with diabetes in Australia.

Methods

We performed a cost-effectiveness analysis between January 01, 2022 and August 01, 2023. A decision-analytic Markov model was constructed to simulate DR progression in a population of 1,197,818 non-Indigenous and 65,160 Indigenous Australians living with diabetes aged ≥20 years over 40 years. From a healthcare provider's perspective, we compared current practice to three primary care AI-based screening scenarios-(A) substitution of current manual grading, (B) scaling up to patient acceptance level, and (C) achieving universal screening. Study results were presented as incremental cost-effectiveness ratio (ICER), benefit-cost ratio (BCR), and net monetary benefits (NMB). A Willingness-to-pay (WTP) threshold of AU$50,000 per quality-adjusted life year (QALY) and a discount rate of 3.5% were adopted in this study.

Findings

With the status quo, the non-Indigenous diabetic population was projected to develop 96,269 blindness cases, resulting in AU$13,039.6 m spending on DR screening and treatment during 2020-2060. In comparison, all three intervention scenarios were effective and cost-saving. In particular, if a universal screening program was to be implemented (Scenario C), it would prevent 38,347 blindness cases, gain 172,090 QALYs and save AU$595.8 m, leading to a BCR of 3.96 and NMB of AU$9,200 m. Similar findings were also reported in the Indigenous population. With the status quo, 3,396 Indigenous individuals would develop blindness, which would cost the health system AU$796.0 m during 2020-2060. All three intervention scenarios were cost-saving for the Indigenous population. Notably, universal AI-based DR screening (Scenario C) would prevent 1,211 blindness cases and gain 9,800 QALYs in the Indigenous population, leading to a saving of AU$19.2 m with a BCR of 1.62 and NMB of AU$509 m.

Interpretation

Our findings suggest that implementing AI-based DR screening in primary care is highly effective and cost-saving in both Indigenous and non-Indigenous populations.

Funding

This project received grant funding from the Australian Government: the National Critical Research Infrastructure Initiative, Medical Research Future Fund (MRFAI00035) and the NHMRC Investigator Grant (APP1175405). The contents of the published material are solely the responsibility of the Administering Institution, a participating institution or individual authors and do not reflect the views of the NHMRC. This work was supported by the Global STEM Professorship Scheme (P0046113), the Fundamental Research Funds of the State Key Laboratory of Ophthalmology, Project of Investigation on Health Status of Employees in Financial Industry in Guangzhou, China (Z012014075). The Centre for Eye Research Australia receives Operational Infrastructure Support from the Victorian State Government. W.H. is supported by the Melbourne Research Scholarship established by the University of Melbourne. The funding source had no role in the design and conduct of the study; collection, management, analysis, and interpretation of the data; preparation, review, or approval of the manuscript; and decision to submit the manuscript for publication.

Real-world evaluation of smartphone-based artificial intelligence to screen for diabetic retinopathy in Dominica: a clinical validation study

OBJECTIVE

Several artificial intelligence (AI) systems for diabetic retinopathy screening have been validated but there is limited evidence on their performance in real-world settings. This study aimed to assess the performance of an AI software deployed within the diabetic retinopathy screening programme in Dominica.

METHODS AND ANALYSIS

We conducted a prospective, cross-sectional clinical validation study. Patients with diabetes aged 18 years and above attending the diabetic retinopathy screening in primary care facilities in Dominica from 5 June to 3 July 2021 were enrolled.Grading was done at the point of care by the field grader, followed by counselling and referral to the eye clinic. Images were then graded by an AI system. Sensitivity, specificity with 95% CIs and area under the curve (AUC) were calculated for comparing the AI to field grader as gold standard.

RESULTS

A total of 587 participants were screened. The AI had a sensitivity and specificity for detecting referable diabetic retinopathy of 77.5% and 91.5% compared with the grader, for all participants, including ungradable images. The AUC was 0.8455. Excluding 52 participants deemed ungradable by the grader, the AI had a sensitivity and specificity of 81.4% and 91.5%, with an AUC of 0.9648.

CONCLUSION

This study provides evidence that AI has the potential to be deployed to assist a diabetic screening programme in a middle-income real-world setting and perform with reasonable accuracy compared with a specialist grader.

Protocol to analyze fundus images for multidimensional quality grading and real-time guidance using deep learning techniques

Data quality issues have been acknowledged as one of the greatest obstacles in medical artificial intelligence research. Here, we present DeepFundus, which employs deep learning techniques to perform multidimensional classification of fundus image quality and provide real-time guidance for on-site image acquisition. We describe steps for data preparation, model training, model inference, model evaluation, and the visualization of results using heatmaps. This protocol can be implemented in Python using either the suggested dataset or a customized dataset. For complete details on the use and execution of this protocol, please refer to Liu et al.1.

Teleophthalmology and retina: a review of current tools, pathways and services

Telemedicine, the use of telecommunication and information technology to deliver healthcare remotely, has evolved beyond recognition since its inception in the 1970s. Advances in telecommunication infrastructure, the advent of the Internet, exponential growth in computing power and associated computer-aided diagnosis, and medical imaging developments have created an environment where telemedicine is more accessible and capable than ever before, particularly in the field of ophthalmology. Ever-increasing global demand for ophthalmic services due to population growth and ageing together with insufficient supply of ophthalmologists requires new models of healthcare provision integrating telemedicine to meet present day challenges, with the recent COVID-19 pandemic providing the catalyst for the widespread adoption and acceptance of teleophthalmology. In this review we discuss the history, present and future application of telemedicine within the field of ophthalmology, and specifically retinal disease. We consider the strengths and limitations of teleophthalmology, its role in screening, community and hospital management of retinal disease, patient and clinician attitudes, and barriers to its adoption.

Translation of Color Fundus Photography into Fluorescein Angiography Using Deep Learning for Enhanced Diabetic Retinopathy Screening

Purpose

To develop and validate a deep learning model that can transform color fundus (CF) photography into corresponding venous and late-phase fundus fluorescein angiography (FFA) images.

Design

Cross-sectional study.

Participants

We included 51 370 CF-venous FFA pairs and 14 644 CF-late FFA pairs from 4438 patients for model development. External testing involved 50 eyes with CF-FFA pairs and 2 public datasets for diabetic retinopathy (DR) classification, with 86 952 CF from EyePACs, and 1744 CF from MESSIDOR2.

Methods

We trained a deep-learning model to transform CF into corresponding venous and late-phase FFA images. The translated FFA images' quality was evaluated quantitatively on the internal test set and subjectively on 100 eyes with CF-FFA paired images (50 from external), based on the realisticity of the global image, anatomical landmarks (macula, optic disc, and vessels), and lesions. Moreover, we validated the clinical utility of the translated FFA for classifying 5-class DR and diabetic macular edema (DME) in the EyePACs and MESSIDOR2 datasets.

Main Outcome Measures

Image generation was quantitatively assessed by structural similarity measures (SSIM), and subjectively by 2 clinical experts on a 5-point scale (1 refers real FFA); intragrader agreement was assessed by kappa. The DR classification accuracy was assessed by area under the receiver operating characteristic curve.

Results

The SSIM of the translated FFA images were > 0.6, and the subjective quality scores ranged from 1.37 to 2.60. Both experts reported similar quality scores with substantial agreement (all kappas > 0.8). Adding the generated FFA on top of CF improved DR classification in the EyePACs and MESSIDOR2 datasets, with the area under the receiver operating characteristic curve increased from 0.912 to 0.939 on the EyePACs dataset and from 0.952 to 0.972 on the MESSIDOR2 dataset. The DME area under the receiver operating characteristic curve also increased from 0.927 to 0.974 in the MESSIDOR2 dataset.

Conclusions

Our CF-to-FFA framework produced realistic FFA images. Moreover, adding the translated FFA images on top of CF improved the accuracy of DR screening. These results suggest that CF-to-FFA translation could be used as a surrogate method when FFA examination is not feasible and as a simple add-on to improve DR screening.

Financial Disclosures

Proprietary or commercial disclosure may be found in the Footnotes and Disclosures at the end of this article.

Risk Stratification for Diabetic Retinopathy Screening Order Using Deep Learning: A Multicenter Prospective Study

Purpose

Real-world evaluation of a deep learning model that prioritizes patients based on risk of progression to moderate or worse (MOD+) diabetic retinopathy (DR).

Methods

This nonrandomized, single-arm, prospective, interventional study included patients attending DR screening at four centers across Thailand from September 2019 to January 2020, with mild or no DR. Fundus photographs were input into the model, and patients were scheduled for their subsequent screening from September 2020 to January 2021 in order of predicted risk. Evaluation focused on model sensitivity, defined as correctly ranking patients that developed MOD+ within the first 50% of subsequent screens.

Results

We analyzed 1,757 patients, of which 52 (3.0%) developed MOD+. Using the model-proposed order, the model's sensitivity was 90.4%. Both the model-proposed order and mild/no DR plus HbA1c had significantly higher sensitivity than the random order (P < 0.001). Excluding one major (rural) site that had practical implementation challenges, the remaining sites included 567 patients and 15 (2.6%) developed MOD+. Here, the model-proposed order achieved 86.7% versus 73.3% for the ranking that used DR grade and hemoglobin A1c.

Conclusions

The model can help prioritize follow-up visits for the largest subgroups of DR patients (those with no or mild DR). Further research is needed to evaluate the impact on clinical management and outcomes.

Translational Relevance

Deep learning demonstrated potential for risk stratification in DR screening. However, real-world practicalities must be resolved to fully realize the benefit.

Past, present, and future of global research on artificial intelligence applications in dermatology: A bibliometric analysis

In recent decades, artificial intelligence (AI) has played an increasingly important role in medicine, including dermatology. Worldwide, numerous studies have reported on AI applications in dermatology, rapidly increasing interest in this field. However, no bibliometric studies have been conducted to evaluate the past, present, or future of this topic. This study aimed to illustrate past and present research and outline future directions for global research on AI applications in dermatology using bibliometric analysis. We conducted an online search of the Web of Science Core Collection database to identify scientific papers on AI applications in dermatology. The bibliometric metadata of each selected paper were extracted, analyzed, and visualized using VOS viewer and Cite Space. A total of 406 papers, comprising 8 randomized controlled trials and 20 prospective studies, were deemed eligible for inclusion. The United States had the highest number of papers (n = 166). The University of California System (n = 24) and Allan C. Halpern (n = 11) were the institution and author with the highest number of papers, respectively. Based on keyword co-occurrence analysis, the studies were categorized into 9 distinct clusters, with clusters 2, 3, and 7 containing keywords with the latest average publication year. Wound progression prediction using machine learning, the integration of AI into teledermatology, and applications of the algorithms in skin diseases, are the current research priorities and will remain future research aims in this field.

Deep Learning Performance of Ultra-Widefield Fundus Imaging for Screening Retinal Lesions in Rural Locales

Importance

Retinal diseases are the leading cause of irreversible blindness worldwide, and timely detection contributes to prevention of permanent vision loss, especially for patients in rural areas with limited medical resources. Deep learning systems (DLSs) based on fundus images with a 45° field of view have been extensively applied in population screening, while the feasibility of using ultra-widefield (UWF) fundus image-based DLSs to detect retinal lesions in patients in rural areas warrants exploration.

Objective

To explore the performance of a DLS for multiple retinal lesion screening using UWF fundus images from patients in rural areas.

Design, Setting, and Participants

In this diagnostic study, a previously developed DLS based on UWF fundus images was used to screen for 5 retinal lesions (retinal exudates or drusen, glaucomatous optic neuropathy, retinal hemorrhage, lattice degeneration or retinal breaks, and retinal detachment) in 24 villages of Yangxi County, China, between November 17, 2020, and March 30, 2021.

Interventions

The captured images were analyzed by the DLS and ophthalmologists.

Main Outcomes and Measures

The performance of the DLS in rural screening was compared with that of the internal validation in the previous model development stage. The image quality, lesion proportion, and complexity of lesion composition were compared between the model development stage and the rural screening stage.

Results

A total of 6222 eyes in 3149 participants (1685 women [53.5%]; mean [SD] age, 70.9 [9.1] years) were screened. The DLS achieved a mean (SD) area under the receiver operating characteristic curve (AUC) of 0.918 (0.021) (95% CI, 0.892-0.944) for detecting 5 retinal lesions in the entire data set when applied for patients in rural areas, which was lower than that reported at the model development stage (AUC, 0.998 [0.002] [95% CI, 0.995-1.000]; P < .001). Compared with the fundus images in the model development stage, the fundus images in this rural screening study had an increased frequency of poor quality (13.8% [860 of 6222] vs 0%), increased variation in lesion proportions (0.1% [6 of 6222]-36.5% [2271 of 6222] vs 14.0% [2793 of 19 891]-21.3% [3433 of 16 138]), and an increased complexity of lesion composition.

Conclusions and Relevance

This diagnostic study suggests that the DLS exhibited excellent performance using UWF fundus images as a screening tool for 5 retinal lesions in patients in a rural setting. However, poor image quality, diverse lesion proportions, and a complex set of lesions may have reduced the performance of the DLS; these factors in targeted screening scenarios should be taken into consideration in the model development stage to ensure good performance.

Deep-Ocular: Improved Transfer Learning Architecture Using Self-Attention and Dense Layers for Recognition of Ocular Diseases

It is difficult for clinicians or less-experienced ophthalmologists to detect early eye-related diseases. By hand, eye disease diagnosis is labor-intensive, prone to mistakes, and challenging because of the variety of ocular diseases such as glaucoma (GA), diabetic retinopathy (DR), cataract (CT), and normal eye-related diseases (NL). An automated ocular disease detection system with computer-aided diagnosis (CAD) tools is required to recognize eye-related diseases. Nowadays, deep learning (DL) algorithms enhance the classification results of retinograph images. To address these issues, we developed an intelligent detection system based on retinal fundus images. To create this system, we used ODIR and RFMiD datasets, which included various retinographics of distinct classes of the fundus, using cutting-edge image classification algorithms like ensemble-based transfer learning. In this paper, we suggest a three-step hybrid ensemble model that combines a classifier, a feature extractor, and a feature selector. The original image features are first extracted using a pre-trained AlexNet model with an enhanced structure. The improved AlexNet (iAlexNet) architecture with attention and dense layers offers enhanced feature extraction, task adaptability, interpretability, and potential accuracy benefits compared to other transfer learning architectures, making it particularly suited for tasks like retinograph classification. The extracted features are then selected using the ReliefF method, and then the most crucial elements are chosen to minimize the feature dimension. Finally, an XgBoost classifier offers classification outcomes based on the desired features. These classifications represent different ocular illnesses. We utilized data augmentation techniques to control class imbalance issues. The deep-ocular model, based mainly on the AlexNet-ReliefF-XgBoost model, achieves an accuracy of 95.13%. The results indicate the proposed ensemble model can assist dermatologists in making early decisions for the diagnosing and screening of eye-related diseases.

Artificial Intelligence and Diabetic Retinopathy: AI Framework, Prospective Studies, Head-to-head Validation, and Cost-effectiveness

Current guidelines recommend that individuals with diabetes receive yearly eye exams for detection of referable diabetic retinopathy (DR), one of the leading causes of new-onset blindness. For addressing the immense screening burden, artificial intelligence (AI) algorithms have been developed to autonomously screen for DR from fundus photography without human input. Over the last 10 years, many AI algorithms have achieved good sensitivity and specificity (>85%) for detection of referable DR compared with human graders; however, many questions still remain. In this narrative review on AI in DR screening, we discuss key concepts in AI algorithm development as a background for understanding the algorithms. We present the AI algorithms that have been prospectively validated against human graders and demonstrate the variability of reference standards and cohort demographics. We review the limited head-to-head validation studies where investigators attempt to directly compare the available algorithms. Next, we discuss the literature regarding cost-effectiveness, equity and bias, and medicolegal considerations, all of which play a role in the implementation of these AI algorithms in clinical practice. Lastly, we highlight ongoing efforts to bridge gaps in AI model data sets to pursue equitable development and delivery.

Machine learning in precision diabetes care and cardiovascular risk prediction

Artificial intelligence and machine learning are driving a paradigm shift in medicine, promising data-driven, personalized solutions for managing diabetes and the excess cardiovascular risk it poses. In this comprehensive review of machine learning applications in the care of patients with diabetes at increased cardiovascular risk, we offer a broad overview of various data-driven methods and how they may be leveraged in developing predictive models for personalized care. We review existing as well as expected artificial intelligence solutions in the context of diagnosis, prognostication, phenotyping, and treatment of diabetes and its cardiovascular complications. In addition to discussing the key properties of such models that enable their successful application in complex risk prediction, we define challenges that arise from their misuse and the role of methodological standards in overcoming these limitations. We also identify key issues in equity and bias mitigation in healthcare and discuss how the current regulatory framework should ensure the efficacy and safety of medical artificial intelligence products in transforming cardiovascular care and outcomes in diabetes.

Artificial intelligence for telemedicine diabetic retinopathy screening: a review

PURPOSE

This study aims to compare artificial intelligence (AI) systems applied in diabetic retinopathy (DR) teleophthalmology screening, currently deployed systems, fairness initiatives and the challenges for implementation.

METHODS

The review included articles retrieved from PubMed/Medline/EMBASE literature search strategy regarding telemedicine, DR and AI. The screening criteria included human articles in English, Portuguese or Spanish and related to telemedicine and AI for DR screening. The author's affiliations and the study's population income group were classified according to the World Bank Country and Lending Groups.

RESULTS

The literature search yielded a total of 132 articles, and nine were included after full-text assessment. The selected articles were published between 2004 and 2020 and were grouped as telemedicine systems, algorithms, economic analysis and image quality assessment. Four telemedicine systems that perform a quality assessment, image preprocessing and pathological screening were reviewed. A data and post-deployment bias assessment are not performed in any of the algorithms, and none of the studies evaluate the social impact implementations. There is a lack of representativeness in the reviewed articles, with most authors and target populations from high-income countries and no low-income country representation.

CONCLUSIONS

Telemedicine and AI hold great promise for augmenting decision-making in medical care, expanding patient access and enhancing cost-effectiveness. Economic studies and social science analysis are crucial to support the implementation of AI in teleophthalmology screening programs. Promoting fairness and generalizability in automated systems combined with telemedicine screening programs is not straightforward. Improving data representativeness, reducing biases and promoting equity in deployment and post-deployment studies are all critical steps in model development.

Artificial intelligence and digital health in global eye health: opportunities and challenges

Global eye health is defined as the degree to which vision, ocular health, and function are maximised worldwide, thereby optimising overall wellbeing and quality of life. Improving eye health is a global priority as a key to unlocking human potential by reducing the morbidity burden of disease, increasing productivity, and supporting access to education. Although extraordinary progress fuelled by global eye health initiatives has been made over the last decade, there remain substantial challenges impeding further progress. The accelerated development of digital health and artificial intelligence (AI) applications provides an opportunity to transform eye health, from facilitating and increasing access to eye care to supporting clinical decision making with an objective, data-driven approach. Here, we explore the opportunities and challenges presented by digital health and AI in global eye health and describe how these technologies could be leveraged to improve global eye health. AI, telehealth, and emerging technologies have great potential, but require specific work to overcome barriers to implementation. We suggest that a global digital eye health task force could facilitate coordination of funding, infrastructural development, and democratisation of AI and digital health to drive progress forwards in this domain.

Toward safer ophthalmic artificial intelligence via distributed validation on real-world data

PURPOSE OF REVIEW

The current article provides an overview of the present approaches to algorithm validation, which are variable and largely self-determined, as well as solutions to address inadequacies.

RECENT FINDINGS

In the last decade alone, numerous machine learning applications have been proposed for ophthalmic diagnosis or disease monitoring. Remarkably, of these, less than 15 have received regulatory approval for implementation into clinical practice. Although there exists a vast pool of structured and relatively clean datasets from which to develop and test algorithms in the computational 'laboratory', real-world validation remains key to allow for safe, equitable, and clinically reliable implementation. Bottlenecks in the validation process stem from a striking paucity of regulatory guidance surrounding safety and performance thresholds, lack of oversight on critical postdeployment monitoring and context-specific recalibration, and inherent complexities of heterogeneous disease states and clinical environments. Implementation of secure, third-party, unbiased, pre and postdeployment validation offers the potential to address existing shortfalls in the validation process.

SUMMARY

Given the criticality of validation to the algorithm pipeline, there is an urgent need for developers, machine learning researchers, and end-user clinicians to devise a consensus approach, allowing for the rapid introduction of safe, equitable, and clinically valid machine learning implementations.

Deep learning assisted diagnosis system: improving the diagnostic accuracy of distal radius fractures

Objectives

To explore an intelligent detection technology based on deep learning algorithms to assist the clinical diagnosis of distal radius fractures (DRFs), and further compare it with human performance to verify the feasibility of this method.

Methods

A total of 3,240 patients (fracture: n = 1,620, normal: n = 1,620) were included in this study, with a total of 3,276 wrist joint anteroposterior (AP) X-ray films (1,639 fractured, 1,637 normal) and 3,260 wrist joint lateral X-ray films (1,623 fractured, 1,637 normal). We divided the patients into training set, validation set and test set in a ratio of 7:1.5:1.5. The deep learning models were developed using the data from the training and validation sets, and then their effectiveness were evaluated using the data from the test set. Evaluate the diagnostic performance of deep learning models using receiver operating characteristic (ROC) curves and area under the curve (AUC), accuracy, sensitivity, and specificity, and compare them with medical professionals.

Results

The deep learning ensemble model had excellent accuracy (97.03%), sensitivity (95.70%), and specificity (98.37%) in detecting DRFs. Among them, the accuracy of the AP view was 97.75%, the sensitivity 97.13%, and the specificity 98.37%; the accuracy of the lateral view was 96.32%, the sensitivity 94.26%, and the specificity 98.37%. When the wrist joint is counted, the accuracy was 97.55%, the sensitivity 98.36%, and the specificity 96.73%. In terms of these variables, the performance of the ensemble model is superior to that of both the orthopedic attending physician group and the radiology attending physician group.

Conclusion

This deep learning ensemble model has excellent performance in detecting DRFs on plain X-ray films. Using this artificial intelligence model as a second expert to assist clinical diagnosis is expected to improve the accuracy of diagnosing DRFs and enhance clinical work efficiency.

Artificial intelligence for diabetic retinopathy in low-income and middle-income countries: a scoping review

Diabetic retinopathy (DR) is a leading cause of blindness globally. There is growing evidence to support the use of artificial intelligence (AI) in diabetic eye care, particularly for screening populations at risk of sight loss from DR in low-income and middle-income countries (LMICs) where resources are most stretched. However, implementation into clinical practice remains limited. We conducted a scoping review to identify what AI tools have been used for DR in LMICs and to report their performance and relevant characteristics. 81 articles were included. The reported sensitivities and specificities were generally high providing evidence to support use in clinical practice. However, the majority of studies focused on sensitivity and specificity only and there was limited information on cost, regulatory approvals and whether the use of AI improved health outcomes. Further research that goes beyond reporting sensitivities and specificities is needed prior to wider implementation.

Deep learning for detecting visually impaired cataracts using fundus images

Purpose: To develop a visual function-based deep learning system (DLS) using fundus images to screen for visually impaired cataracts. Materials and methods: A total of 8,395 fundus images (5,245 subjects) with corresponding visual function parameters collected from three clinical centers were used to develop and evaluate a DLS for classifying non-cataracts, mild cataracts, and visually impaired cataracts. Three deep learning algorithms (DenseNet121, Inception V3, and ResNet50) were leveraged to train models to obtain the best one for the system. The performance of the system was evaluated using the area under the receiver operating characteristic curve (AUC), sensitivity, and specificity. Results: The AUC of the best algorithm (DenseNet121) on the internal test dataset and the two external test datasets were 0.998 (95% CI, 0.996-0.999) to 0.999 (95% CI, 0.998-1.000),0.938 (95% CI, 0.924-0.951) to 0.966 (95% CI, 0.946-0.983) and 0.937 (95% CI, 0.918-0.953) to 0.977 (95% CI, 0.962-0.989), respectively. In the comparison between the system and cataract specialists, better performance was observed in the system for detecting visually impaired cataracts (p < 0.05). Conclusion: Our study shows the potential of a function-focused screening tool to identify visually impaired cataracts from fundus images, enabling timely patient referral to tertiary eye hospitals.

Deep learning system assisted detection and localization of lumbar spondylolisthesis

Objective: Explore a new deep learning (DL) object detection algorithm for clinical auxiliary diagnosis of lumbar spondylolisthesis and compare it with doctors' evaluation to verify the effectiveness and feasibility of the DL algorithm in the diagnosis of lumbar spondylolisthesis. Methods: Lumbar lateral radiographs of 1,596 patients with lumbar spondylolisthesis from three medical institutions were collected, and senior orthopedic surgeons and radiologists jointly diagnosed and marked them to establish a database. These radiographs were randomly divided into a training set (n = 1,117), a validation set (n = 240), and a test set (n = 239) in a ratio of 0.7 : 0.15: 0.15. We trained two DL models for automatic detection of spondylolisthesis and evaluated their diagnostic performance by PR curves, areas under the curve, precision, recall, F1-score. Then we chose the model with better performance and compared its results with professionals' evaluation. Results: A total of 1,780 annotations were marked for training (1,242), validation (263), and test (275). The Faster Region-based Convolutional Neural Network (R-CNN) showed better precision (0.935), recall (0.935), and F1-score (0.935) in the detection of spondylolisthesis, which outperformed the doctor group with precision (0.927), recall (0.892), f1-score (0.910). In addition, with the assistance of the DL model, the precision of the doctor group increased by 4.8%, the recall by 8.2%, the F1-score by 6.4%, and the average diagnosis time per plain X-ray was shortened by 7.139 s. Conclusion: The DL detection algorithm is an effective method for clinical diagnosis of lumbar spondylolisthesis. It can be used as an assistant expert to improve the accuracy of lumbar spondylolisthesis diagnosis and reduce the clinical workloads.

Single retinal image for diabetic retinopathy screening: performance of a handheld device with embedded artificial intelligence

BACKGROUND

Diabetic retinopathy (DR) is a leading cause of blindness. Our objective was to evaluate the performance of an artificial intelligence (AI) system integrated into a handheld smartphone-based retinal camera for DR screening using a single retinal image per eye.

METHODS

Images were obtained from individuals with diabetes during a mass screening program for DR in Blumenau, Southern Brazil, conducted by trained operators. Automatic analysis was conducted using an AI system (EyerMaps™, Phelcom Technologies LLC, Boston, USA) with one macula-centered, 45-degree field of view retinal image per eye. The results were compared to the assessment by a retinal specialist, considered as the ground truth, using two images per eye. Patients with ungradable images were excluded from the analysis.

RESULTS

A total of 686 individuals (average age 59.2 ± 13.3 years, 56.7% women, diabetes duration 12.1 ± 9.4 years) were included in the analysis. The rates of insulin use, daily glycemic monitoring, and systemic hypertension treatment were 68.4%, 70.2%, and 70.2%, respectively. Although 97.3% of patients were aware of the risk of blindness associated with diabetes, more than half of them underwent their first retinal examination during the event. The majority (82.5%) relied exclusively on the public health system. Approximately 43.4% of individuals were either illiterate or had not completed elementary school. DR classification based on the ground truth was as follows: absent or nonproliferative mild DR 86.9%, more than mild (mtm) DR 13.1%. The AI system achieved sensitivity, specificity, positive predictive value, and negative predictive value percentages (95% CI) for mtmDR as follows: 93.6% (87.8-97.2), 71.7% (67.8-75.4), 42.7% (39.3-46.2), and 98.0% (96.2-98.9), respectively. The area under the ROC curve was 86.4%.

CONCLUSION

The portable retinal camera combined with AI demonstrated high sensitivity for DR screening using only one image per eye, offering a simpler protocol compared to the traditional approach of two images per eye. Simplifying the DR screening process could enhance adherence rates and overall program coverage.

Application of a Comprehensive Evaluation Framework to COVID-19 Studies: Systematic Review of Translational Aspects of Artificial Intelligence in Health Care

Background

Despite immense progress in artificial intelligence (AI) models, there has been limited deployment in health care environments. The gap between potential and actual AI applications is likely due to the lack of translatability between controlled research environments (where these models are developed) and clinical environments for which the AI tools are ultimately intended.

Objective

We previously developed the Translational Evaluation of Healthcare AI (TEHAI) framework to assess the translational value of AI models and to support successful transition to health care environments. In this study, we applied the TEHAI framework to the COVID-19 literature in order to assess how well translational topics are covered.

Methods

A systematic literature search for COVID-19 AI studies published between December 2019 and December 2020 resulted in 3830 records. A subset of 102 (2.7%) papers that passed the inclusion criteria was sampled for full review. The papers were assessed for translational value and descriptive data collected by 9 reviewers (each study was assessed by 2 reviewers). Evaluation scores and extracted data were compared by a third reviewer for resolution of discrepancies. The review process was conducted on the Covidence software platform.

Results

We observed a significant trend for studies to attain high scores for technical capability but low scores for the areas essential for clinical translatability. Specific questions regarding external model validation, safety, nonmaleficence, and service adoption received failed scores in most studies.

Conclusions

Using TEHAI, we identified notable gaps in how well translational topics of AI models are covered in the COVID-19 clinical sphere. These gaps in areas crucial for clinical translatability could, and should, be considered already at the model development stage to increase translatability into real COVID-19 health care environments.