Predicting conversion to wet age-related macular degeneration using deep learning

Deep learning for multi-type infectious keratitis diagnosis: A nationwide, cross-sectional, multicenter study

The main cause of corneal blindness worldwide is keratitis, especially the infectious form caused by bacteria, fungi, viruses, and Acanthamoeba. The key to effective management of infectious keratitis hinges on prompt and precise diagnosis. Nevertheless, the current gold standard, such as cultures of corneal scrapings, remains time-consuming and frequently yields false-negative results. Here, using 23,055 slit-lamp images collected from 12 clinical centers nationwide, this study constructed a clinically feasible deep learning system, DeepIK, that could emulate the diagnostic process of a human expert to identify and differentiate bacterial, fungal, viral, amebic, and noninfectious keratitis. DeepIK exhibited remarkable performance in internal, external, and prospective datasets (all areas under the receiver operating characteristic curves > 0.96) and outperformed three other state-of-the-art algorithms (DenseNet121, InceptionResNetV2, and Swin-Transformer). Our study indicates that DeepIK possesses the capability to assist ophthalmologists in accurately and swiftly identifying various infectious keratitis types from slit-lamp images, thereby facilitating timely and targeted treatment.

Home-monitoring for neovascular age-related macular degeneration in older adults within the UK: the MONARCH diagnostic accuracy study

Background

Most neovascular age-related macular degeneration treatments involve long-term follow-up of disease activity. Home monitoring would reduce the burden on patients and those they depend on for transport, and release clinic appointments for other patients. The study aimed to evaluate three home-monitoring tests for patients to use to detect active neovascular age-related macular degeneration compared with diagnosing active neovascular age-related macular degeneration by hospital follow-up.

Objectives

There were five objectives: Estimate the accuracy of three home-monitoring tests to detect active neovascular age-related macular degeneration. Determine the acceptability of home monitoring to patients and carers and adherence to home monitoring. Explore whether inequalities exist in recruitment, participants' ability to self-test and their adherence to weekly testing during follow-up. Provide pilot data about the accuracy of home monitoring to detect conversion to neovascular age-related macular degeneration in fellow eyes of patients with unilateral neovascular age-related macular degeneration. Describe challenges experienced when implementing home-monitoring tests.

Design

Diagnostic test accuracy cohort study, stratified by time since starting treatment.

Setting

Six United Kingdom Hospital Eye Service macular clinics (Belfast, Liverpool, Moorfields, James Paget, Southampton, Gloucester).

Participants

Patients with at least one study eye being monitored by hospital follow-up.

Reference standard

Detection of active neovascular age-related macular degeneration by an ophthalmologist at hospital follow-up.

Index tests

KeepSight Journal: paper-based near-vision tests presented as word puzzles. MyVisionTrack®: electronic test, viewed on a tablet device. MultiBit: electronic test, viewed on a tablet device. Participants provided test scores weekly. Raw scores between hospital follow-ups were summarised as averages.

Results

Two hundred and ninety-seven patients (mean age 74.9 years) took part. At least one hospital follow-up was available for 317 study eyes, including 9 second eyes that became eligible during follow-up, in 261 participants (1549 complete visits). Median testing frequency was three times/month. Estimated areas under receiver operating curves were < 0.6 for all index tests, and only KeepSight Journal summary score was significantly associated with the lesion activity (odds ratio = 3.48, 95% confidence interval 1.09 to 11.13, p = 0.036). Older age and worse deprivation for home address were associated with lower participation (χ2 = 50.5 and 24.3, respectively, p < 0.001) but not ability or adherence to self-testing. Areas under receiver operating curves appeared higher for conversion of fellow eyes to neovascular age-related macular degeneration (0.85 for KeepSight Journal) but were estimated with less precision. Almost half of participants called a study helpline, most often due to inability to test electronically.

Limitations

Pre-specified sample size not met; participants' difficulties using the devices; electronic tests not always available.

Conclusions

No index test provided adequate test accuracy to identify lesion diagnosed as active in follow-up clinics. If used to detect conversion, patients would still need to be monitored at hospital. Associations of older age and worse deprivation with study participation highlight the potential for inequities with such interventions. Provision of reliable electronic testing was challenging.

Future work

Future studies evaluating similar technologies should consider: Independent monitoring with clear stopping rules based on test performance. Deployment of apps on patients' own devices since providing devices did not reduce inequalities in participation and complicated home testing. Alternative methods to summarise multiple scores over the period preceding a follow-up.

Trial registration

This trial is registered as ISRCTN79058224.

Funding

This award was funded by the National Institute of Health and Care Research (NIHR) Health Technology Assessment programme (NIHR award ref: 15/97/02) and is published in full in Health Technology Assessment; Vol. 28, No. 32. See the NIHR Funding and Awards website for further award information.

Advancements in the treatment of age-related macular degeneration: a comprehensive review

Age-related macular degeneration (AMD) stands as a leading cause of irreversible blindness, particularly affecting central vision and impeding daily tasks. This paper provides a thorough exploration of AMD, distinguishing between its two main subtypes-Wet and Dry AMD-while shedding light on the prevalence and risk factors, including age, genetics, and smoking. The focus shifts to the current and future treatment landscape, examining both Dry and Wet AMD. Regarding Dry AMD, interventions such as antioxidant supplementation and ongoing clinical trials offer hope. Notable among these is Pegcetacoplan which is the only Food and Drug Administration (FDA)-approved medication, displaying promising results in reducing geographic atrophy lesions. For Wet AMD, anti-Vascular Endothelial Growth Factor therapies like Ranibizumab (Lucentis®) have been instrumental, and newer drugs like Faricimab and OPT-302 show comparable efficacy with extended dosing intervals. Additionally, gene therapies such as RGX-314 present a potential paradigm shift, reducing or eliminating the need for frequent injections. Biosimilars offer cost-effective alternatives. The paper also delves into the integration of technology and artificial intelligence in AMD management, highlighting the role of smartphone apps for patient monitoring and artificial intelligence algorithms for diagnosis and surveillance. Furthermore, patient perspectives on artificial intelligence demonstrate a positive correlation between understanding and trust. The narrative concludes with a glimpse into ground-breaking technologies, including retinal implants and bionic chips, offering hope for vision restoration. Overall, this paper underscores the multifaceted approach in addressing AMD, combining traditional and innovative strategies, paving the way for a more promising future in AMD treatment.

Deep Learning in Neovascular Age-Related Macular Degeneration

Background and objectives: Age-related macular degeneration (AMD) is a complex and multifactorial condition that can lead to permanent vision loss once it progresses to the neovascular exudative stage. This review aims to summarize the use of deep learning in neovascular AMD. Materials and Methods: Pubmed search. Results: Deep learning has demonstrated effectiveness in analyzing structural OCT images in patients with neovascular AMD. This review outlines the role of deep learning in identifying and measuring biomarkers linked to an elevated risk of transitioning to the neovascular form of AMD. Additionally, deep learning techniques can quantify critical OCT features associated with neovascular AMD, which have prognostic implications for these patients. Incorporating deep learning into the assessment of neovascular AMD eyes holds promise for enhancing clinical management strategies for affected individuals. Conclusion: Several studies have demonstrated effectiveness of deep learning in assessing neovascular AMD patients and this has a promising role in the assessment of these patients.

Injectable hydrogels based on biopolymers for the treatment of ocular diseases

Injectable hydrogels based on biopolymers, fabricated utilizing diverse chemical and physical methodologies, exhibit exceptional physical, chemical, and biological properties. They have multifaceted applications encompassing wound healing, tissue regeneration, and across diverse scientific realms. This review critically evaluates their largely uncharted potential in ophthalmology, elucidating their diverse applications across an array of ocular diseases. These conditions include glaucoma, cataracts, corneal disorders (spanning from age-related degeneration to trauma, infections, and underlying chronic illnesses), retina-associated ailments (such as diabetic retinopathy, retinitis pigmentosa, and age-related macular degeneration (AMD)), eyelid abnormalities, and uveal melanoma (UM). This study provides a thorough analysis of applications of injectable hydrogels based on biopolymers across these ocular disorders. Injectable hydrogels based on biopolymers can be customized to have specific physical, chemical, and biological properties that make them suitable as drug delivery vehicles, tissue scaffolds, and sealants in the eye. For example, they can be engineered to have optimum viscosity to be injected intravitreally and sustain drug release to treat retinal diseases. Their porous structure and biocompatibility promote cellular infiltration to regenerate diseased corneal tissue. By accentuating their indispensable role in ocular disease treatment, this review strives to present innovative and targeted approaches in this domain, thereby advancing ocular therapeutics.

Clinical performance of predicting late age-related macular degeneration development using multimodal imaging

BACKGROUND

To examine whether the clinical performance of predicting late age-related macular degeneration (AMD) development is improved through using multimodal imaging (MMI) compared to using colour fundus photography (CFP) alone, and how this compares with a basic prediction model using well-established AMD risk factors.

METHODS

Individuals with AMD in this study underwent MMI, including optical coherence tomography (OCT), fundus autofluorescence, near-infrared reflectance and CFP at baseline, and then at 6-monthly intervals for 3-years to determine MMI-defined late AMD development. Four retinal specialists independently assessed the likelihood that each eye at baseline would progress to MMI-defined late AMD over 3-years with CFP, and then with MMI. Predictive performance with CFP and MMI were compared to each other, and to a basic prediction model using age, presence of pigmentary abnormalities, and OCT-based drusen volume.

RESULTS

The predictive performance of the clinicians using CFP [AUC = 0.75; 95% confidence interval (CI) = 0.68-0.82] improved when using MMI (AUC = 0.79; 95% CI = 0.72-0.85; p = 0.034). However, a basic prediction model outperformed clinicians using either CFP or MMI (AUC = 0.85; 95% CI = 0.78-91; p ≤ 0.002).

CONCLUSIONS

Clinical performance for predicting late AMD development was improved by using MMI compared to CFP. However, a basic prediction model using well-established AMD risk factors outperformed retinal specialists, suggesting that such a model could further improve personalised counselling and monitoring of individuals with the early stages of AMD in clinical practice.

Human versus Artificial Intelligence: Validation of a Deep Learning Model for Retinal Layer and Fluid Segmentation in Optical Coherence Tomography Images from Patients with Age-Related Macular Degeneration

Artificial intelligence (AI) models have received considerable attention in recent years for their ability to identify optical coherence tomography (OCT) biomarkers with clinical diagnostic potential and predict disease progression. This study aims to externally validate a deep learning (DL) algorithm by comparing its segmentation of retinal layers and fluid with a gold-standard method for manually adjusting the automatic segmentation of the Heidelberg Spectralis HRA + OCT software Version 6.16.8.0. A total of sixty OCT images of healthy subjects and patients with intermediate and exudative age-related macular degeneration (AMD) were included. A quantitative analysis of the retinal thickness and fluid area was performed, and the discrepancy between these methods was investigated. The results showed a moderate-to-strong correlation between the metrics extracted by both software types, in all the groups, and an overall near-perfect area overlap was observed, except for in the inner segment ellipsoid (ISE) layer. The DL system detected a significant difference in the outer retinal thickness across disease stages and accurately identified fluid in exudative cases. In more diseased eyes, there was significantly more disagreement between these methods. This DL system appears to be a reliable method for accessing important OCT biomarkers in AMD. However, further accuracy testing should be conducted to confirm its validity in real-world settings to ultimately aid ophthalmologists in OCT imaging management and guide timely treatment approaches.

AI-based support for optical coherence tomography in age-related macular degeneration

Artificial intelligence (AI) has emerged as a transformative technology across various fields, and its applications in the medical domain, particularly in ophthalmology, has gained significant attention. The vast amount of high-resolution image data, such as optical coherence tomography (OCT) images, has been a driving force behind AI growth in this field. Age-related macular degeneration (AMD) is one of the leading causes for blindness in the world, affecting approximately 196 million people worldwide in 2020. Multimodal imaging has been for a long time the gold standard for diagnosing patients with AMD, however, currently treatment and follow-up in routine disease management are mainly driven by OCT imaging. AI-based algorithms have by their precision, reproducibility and speed, the potential to reliably quantify biomarkers, predict disease progression and assist treatment decisions in clinical routine as well as academic studies. This review paper aims to provide a summary of the current state of AI in AMD, focusing on its applications, challenges, and prospects.

Behind the mask: a critical perspective on the ethical, moral, and legal implications of AI in ophthalmology

PURPOSE

This narrative review aims to provide an overview of the dangers, controversial aspects, and implications of artificial intelligence (AI) use in ophthalmology and other medical-related fields.

METHODS

We conducted a decade-long comprehensive search (January 2013-May 2023) of both academic and grey literature, focusing on the application of AI in ophthalmology and healthcare. This search included key web-based academic databases, non-traditional sources, and targeted searches of specific organizations and institutions. We reviewed and selected documents for relevance to AI, healthcare, ethics, and guidelines, aiming for a critical analysis of ethical, moral, and legal implications of AI in healthcare.

RESULTS

Six main issues were identified, analyzed, and discussed. These include bias and clinical safety, cybersecurity, health data and AI algorithm ownership, the "black-box" problem, medical liability, and the risk of widening inequality in healthcare.

CONCLUSION

Solutions to address these issues include collecting high-quality data of the target population, incorporating stronger security measures, using explainable AI algorithms and ensemble methods, and making AI-based solutions accessible to everyone. With careful oversight and regulation, AI-based systems can be used to supplement physician decision-making and improve patient care and outcomes.

DBPF-net: dual-branch structural feature extraction reinforcement network for ocular surface disease image classification

Pterygium and subconjunctival hemorrhage are two common types of ocular surface diseases that can cause distress and anxiety in patients. In this study, 2855 ocular surface images were collected in four categories: normal ocular surface, subconjunctival hemorrhage, pterygium to be observed, and pterygium requiring surgery. We propose a diagnostic classification model for ocular surface diseases, dual-branch network reinforced by PFM block (DBPF-Net), which adopts the conformer model with two-branch architectural properties as the backbone of a four-way classification model for ocular surface diseases. In addition, we propose a block composed of a patch merging layer and a FReLU layer (PFM block) for extracting spatial structure features to further strengthen the feature extraction capability of the model. In practice, only the ocular surface images need to be input into the model to discriminate automatically between the disease categories. We also trained the VGG16, ResNet50, EfficientNetB7, and Conformer models, and evaluated and analyzed the results of all models on the test set. The main evaluation indicators were sensitivity, specificity, F1-score, area under the receiver operating characteristics curve (AUC), kappa coefficient, and accuracy. The accuracy and kappa coefficient of the proposed diagnostic model in several experiments were averaged at 0.9789 and 0.9681, respectively. The sensitivity, specificity, F1-score, and AUC were, respectively, 0.9723, 0.9836, 0.9688, and 0.9869 for diagnosing pterygium to be observed, and, respectively, 0.9210, 0.9905, 0.9292, and 0.9776 for diagnosing pterygium requiring surgery. The proposed method has high clinical reference value for recognizing these four types of ocular surface images.

Retinal Disease Diagnosis Using Deep Learning on Ultra-Wide-Field Fundus Images

Ultra-wide-field fundus imaging (UFI) provides comprehensive visualization of crucial eye components, including the optic disk, fovea, and macula. This in-depth view facilitates doctors in accurately diagnosing diseases and recommending suitable treatments. This study investigated the application of various deep learning models for detecting eye diseases using UFI. We developed an automated system that processes and enhances a dataset of 4697 images. Our approach involves brightness and contrast enhancement, followed by applying feature extraction, data augmentation and image classification, integrated with convolutional neural networks. These networks utilize layer-wise feature extraction and transfer learning from pre-trained models to accurately represent and analyze medical images. Among the five evaluated models, including ResNet152, Vision Transformer, InceptionResNetV2, RegNet and ConVNext, ResNet152 is the most effective, achieving a testing area under the curve (AUC) score of 96.47% (with a 95% confidence interval (CI) of 0.931-0.974). Additionally, the paper presents visualizations of the model's predictions, including confidence scores and heatmaps that highlight the model's focal points-particularly where lesions due to damage are evident. By streamlining the diagnosis process and providing intricate prediction details without human intervention, our system serves as a pivotal tool for ophthalmologists. This research underscores the compatibility and potential of utilizing ultra-wide-field images in conjunction with deep learning.

Research on an artificial intelligence-based myopic maculopathy grading method using EfficientNet

PURPOSE

We aimed to develop an artificial intelligence-based myopic maculopathy grading method using EfficientNet to overcome the delayed grading and diagnosis of different myopic maculopathy degrees.

METHODS

The cooperative hospital provided 4642 healthy and myopic maculopathy color fundus photographs, comprising the four degrees of myopic maculopathy and healthy fundi. The myopic maculopathy grading models were trained using EfficientNet-B0 to EfficientNet-B7 models. The diagnostic results were compared with those of the VGG16 and ResNet50 classification models. The leading evaluation indicators were sensitivity, specificity, F1 score, area under the receiver operating characteristic (ROC) curve area under curve (AUC), 95% confidence interval, kappa value, and accuracy. The ROC curves of the ten grading models were also compared.

RESULTS

We used 1199 color fundus photographs to evaluate the myopic maculopathy grading models. The size of the EfficientNet-B0 myopic maculopathy grading model was 15.6 MB, and it had the highest kappa value (88.32%) and accuracy (83.58%). The model's sensitivities to diagnose tessellated fundus (TF), diffuse chorioretinal atrophy (DCA), patchy chorioretinal atrophy (PCA), and macular atrophy (MA) were 96.86%, 75.98%, 64.67%, and 88.75%, respectively. The specificity was above 93%, and the AUCs were 0.992, 0.960, 0.964, and 0.989, respectively.

CONCLUSION

The EfficientNet models were used to design grading diagnostic models for myopic maculopathy. Based on the collected fundus images, the models could diagnose a healthy fundus and four types of myopic maculopathy. The models might help ophthalmologists to make preliminary diagnoses of different degrees of myopic maculopathy.

AI-based diagnosis of nuclear cataract from slit-lamp videos

In ophthalmology, the availability of many fundus photographs and optical coherence tomography images has spurred consideration of using artificial intelligence (AI) for diagnosing retinal and optic nerve disorders. However, AI application for diagnosing anterior segment eye conditions remains unfeasible due to limited standardized images and analysis models. We addressed this limitation by augmenting the quantity of standardized optical images using a video-recordable slit-lamp device. We then investigated whether our proposed machine learning (ML) AI algorithm could accurately diagnose cataracts from videos recorded with this device. We collected 206,574 cataract frames from 1812 cataract eye videos. Ophthalmologists graded the nuclear cataracts (NUCs) using the cataract grading scale of the World Health Organization. These gradings were used to train and validate an ML algorithm. A validation dataset was used to compare the NUC diagnosis and grading of AI and ophthalmologists. The results of individual cataract gradings were: NUC 0: area under the curve (AUC) = 0.967; NUC 1: AUC = 0.928; NUC 2: AUC = 0.923; and NUC 3: AUC = 0.949. Our ML-based cataract diagnostic model achieved performance comparable to a conventional device, presenting a promising and accurate auto diagnostic AI tool.

Recent Advances in Imaging Macular Atrophy for Late-Stage Age-Related Macular Degeneration

Age-related macular degeneration (AMD) is a leading cause of blindness worldwide. In late-stage AMD, geographic atrophy (GA) of dry AMD or choroidal neovascularization (CNV) of neovascular AMD eventually results in macular atrophy (MA), leading to significant visual loss. Despite the development of innovative therapies, there are currently no established effective treatments for MA. As a result, early detection of MA is critical in identifying later central macular involvement throughout time. Accurate and early diagnosis is achieved through a combination of clinical examination and imaging techniques. Our review of the literature depicts advances in retinal imaging to identify biomarkers of progression and risk factors for late AMD. Imaging methods like fundus photography; dye-based angiography; fundus autofluorescence (FAF); near-infrared reflectance (NIR); optical coherence tomography (OCT); and optical coherence tomography angiography (OCTA) can be used to detect and monitor the progression of retinal atrophy. These evolving diverse imaging modalities optimize detection of pathologic anatomy and measurement of visual function; they may also contribute to the understanding of underlying mechanistic pathways, particularly the underlying MA changes in late AMD.

[Guidelines for clinical trial protocols for interventions involving artificial intelligence: the SPIRIT-AI extensionDiretrizes para protocolos de ensaios clínicos com intervenções que utilizam inteligência artificial: a extensão SPIRIT-AI]

The SPIRIT 2013 statement aims to improve the completeness of clinical trial protocol reporting by providing evidence-based recommendations for the minimum set of items to be addressed. This guidance has been instrumental in promoting transparent evaluation of new interventions. More recently, there has been a growing recognition that interventions involving artificial intelligence (AI) need to undergo rigorous, prospective evaluation to demonstrate their impact on health outcomes. The SPIRIT-AI (Standard Protocol Items: Recommendations for Interventional Trials-Artificial Intelligence) extension is a new reporting guideline for clinical trial protocols evaluating interventions with an AI component. It was developed in parallel with its companion statement for trial reports: CONSORT-AI (Consolidated Standards of Reporting Trials-Artificial Intelligence). Both guidelines were developed through a staged consensus process involving literature review and expert consultation to generate 26 candidate items, which were consulted upon by an international multi-stakeholder group in a two-stage Delphi survey (103 stakeholders), agreed upon in a consensus meeting (31 stakeholders) and refined through a checklist pilot (34 participants). The SPIRIT-AI extension includes 15 new items that were considered sufficiently important for clinical trial protocols of AI interventions. These new items should be routinely reported in addition to the core SPIRIT 2013 items. SPIRIT-AI recommends that investigators provide clear descriptions of the AI intervention, including instructions and skills required for use, the setting in which the AI intervention will be integrated, considerations for the handling of input and output data, the human-AI interaction and analysis of error cases. SPIRIT-AI will help promote transparency and completeness for clinical trial protocols for AI interventions. Its use will assist editors and peer reviewers, as well as the general readership, to understand, interpret and critically appraise the design and risk of bias for a planned clinical trial.

Generative Artificial Intelligence Through ChatGPT and Other Large Language Models in Ophthalmology: Clinical Applications and Challenges

The rapid progress of large language models (LLMs) driving generative artificial intelligence applications heralds the potential of opportunities in health care. We conducted a review up to April 2023 on Google Scholar, Embase, MEDLINE, and Scopus using the following terms: "large language models," "generative artificial intelligence," "ophthalmology," "ChatGPT," and "eye," based on relevance to this review. From a clinical viewpoint specific to ophthalmologists, we explore from the different stakeholders' perspectives-including patients, physicians, and policymakers-the potential LLM applications in education, research, and clinical domains specific to ophthalmology. We also highlight the foreseeable challenges of LLM implementation into clinical practice, including the concerns of accuracy, interpretability, perpetuating bias, and data security. As LLMs continue to mature, it is essential for stakeholders to jointly establish standards for best practices to safeguard patient safety.

Financial Disclosures

Proprietary or commercial disclosure may be found in the Footnotes and Disclosures at the end of this article.

A clinically applicable AI system for diagnosis of congenital heart diseases based on computed tomography images

Congenital heart disease (CHD) is the most common type of birth defect. Without timely detection and treatment, approximately one-third of children with CHD would die in the infant period. However, due to the complicated heart structures, early diagnosis of CHD and its types is quite challenging, even for experienced radiologists. Here, we present an artificial intelligence (AI) system that achieves a comparable performance of human experts in the critical task of classifying 17 categories of CHD types. We collected the first-large CT dataset from three different CT machines, including more than 3750 CHD patients over 14 years. Experimental results demonstrate that it can achieve diagnosis accuracy (86.03%) comparable with junior cardiovascular radiologists (86.27%) in a World Health Organization appointed research and cooperation center in China on most types of CHD, and obtains a higher sensitivity (82.91%) than junior cardiovascular radiologists (76.18%). The accuracy of the combination of our AI system (97.20%) and senior radiologists achieves comparable results to that of junior radiologists and senior radiologists (97.16%) which is the current clinical routine. Our AI system can further provide 3D visualization of hearts to senior radiologists for interpretation and flexible review, surgeons for precise intuition of heart structures, and clinicians for more precise outcome prediction. We demonstrate the potential of our model to be integrated into current clinic practice to improve the diagnosis of CHD globally, especially in regions where experienced radiologists can be scarce.

AI-Human Hybrid Workflow Enhances Teleophthalmology for the Detection of Diabetic Retinopathy

Objective

Detection of diabetic retinopathy (DR) outside of specialized eye care settings is an important means of access to vision-preserving health maintenance. Remote interpretation of fundus photographs acquired in a primary care or other nonophthalmic setting in a store-and-forward manner is a predominant paradigm of teleophthalmology screening programs. Artificial intelligence (AI)-based image interpretation offers an alternative means of DR detection. IDx-DR (Digital Diagnostics Inc) is a Food and Drug Administration-authorized autonomous testing device for DR. We evaluated the diagnostic performance of IDx-DR compared with human-based teleophthalmology over 2 and a half years. Additionally, we evaluated an AI-human hybrid workflow that combines AI-system evaluation with human expert-based assessment for referable cases.

Design

Prospective cohort study and retrospective analysis.

Participants

Diabetic patients ≥ 18 years old without a prior DR diagnosis or DR examination in the past year presenting for routine DR screening in a primary care clinic.

Methods

Macula-centered and optic nerve-centered fundus photographs were evaluated by an AI algorithm followed by consensus-based overreading by retina specialists at the Stanford Ophthalmic Reading Center. Detection of more-than-mild diabetic retinopathy (MTMDR) was compared with in-person examination by a retina specialist.

Main Outcome Measures

Sensitivity, specificity, accuracy, positive predictive value, and gradability achieved by the AI algorithm and retina specialists.

Results

The AI algorithm had higher sensitivity (95.5% sensitivity; 95% confidence interval [CI], 86.7%-100%) but lower specificity (60.3% specificity; 95% CI, 47.7%-72.9%) for detection of MTMDR compared with remote image interpretation by retina specialists (69.5% sensitivity; 95% CI, 50.7%-88.3%; 96.9% specificity; 95% CI, 93.5%-100%). Gradability of encounters was also lower for the AI algorithm (62.5%) compared with retina specialists (93.1%). A 2-step AI-human hybrid workflow in which the AI algorithm initially rendered an assessment followed by overread by a retina specialist of MTMDR-positive encounters resulted in a sensitivity of 95.5% (95% CI, 86.7%-100%) and a specificity of 98.2% (95% CI, 94.6%-100%). Similarly, a 2-step overread by retina specialists of AI-ungradable encounters improved gradability from 63.5% to 95.6% of encounters.

Conclusions

Implementation of an AI-human hybrid teleophthalmology workflow may both decrease reliance on human specialist effort and improve diagnostic accuracy.

Financial Disclosures

Proprietary or commercial disclosure may be found after the references.

Predicting OCT biological marker localization from weak annotations

Recent developments in deep learning have shown success in accurately predicting the location of biological markers in Optical Coherence Tomography (OCT) volumes of patients with Age-Related Macular Degeneration (AMD) and Diabetic Retinopathy (DR). We propose a method that automatically locates biological markers to the Early Treatment Diabetic Retinopathy Study (ETDRS) rings, only requiring B-scan-level presence annotations. We trained a neural network using 22,723 OCT B-Scans of 460 eyes (433 patients) with AMD and DR, annotated with slice-level labels for Intraretinal Fluid (IRF) and Subretinal Fluid (SRF). The neural network outputs were mapped into the corresponding ETDRS rings. We incorporated the class annotations and domain knowledge into a loss function to constrain the output with biologically plausible solutions. The method was tested on a set of OCT volumes with 322 eyes (189 patients) with Diabetic Macular Edema, with slice-level SRF and IRF presence annotations for the ETDRS rings. Our method accurately predicted the presence of IRF and SRF in each ETDRS ring, outperforming previous baselines even in the most challenging scenarios. Our model was also successfully applied to en-face marker segmentation and showed consistency within C-scans, despite not incorporating volume information in the training process. We achieved a correlation coefficient of 0.946 for the prediction of the IRF area.

Deep learning in optical coherence tomography: Where are the gaps?

Optical coherence tomography (OCT) is a non-invasive optical imaging modality, which provides rapid, high-resolution and cross-sectional morphology of macular area and optic nerve head for diagnosis and managing of different eye diseases. However, interpreting OCT images requires experts in both OCT images and eye diseases since many factors such as artefacts and concomitant diseases can affect the accuracy of quantitative measurements made by post-processing algorithms. Currently, there is a growing interest in applying deep learning (DL) methods to analyse OCT images automatically. This review summarises the trends in DL-based OCT image analysis in ophthalmology, discusses the current gaps, and provides potential research directions. DL in OCT analysis shows promising performance in several tasks: (1) layers and features segmentation and quantification; (2) disease classification; (3) disease progression and prognosis; and (4) referral triage level prediction. Different studies and trends in the development of DL-based OCT image analysis are described and the following challenges are identified and described: (1) public OCT data are scarce and scattered; (2) models show performance discrepancies in real-world settings; (3) models lack of transparency; (4) there is a lack of societal acceptance and regulatory standards; and (5) OCT is still not widely available in underprivileged areas. More work is needed to tackle the challenges and gaps, before DL is further applied in OCT image analysis for clinical use.

Artificial intelligence in retinal disease: clinical application, challenges, and future directions

Retinal diseases are a leading cause of blindness in developed countries, accounting for the largest share of visually impaired children, working-age adults (inherited retinal disease), and elderly individuals (age-related macular degeneration). These conditions need specialised clinicians to interpret multimodal retinal imaging, with diagnosis and intervention potentially delayed. With an increasing and ageing population, this is becoming a global health priority. One solution is the development of artificial intelligence (AI) software to facilitate rapid data processing. Herein, we review research offering decision support for the diagnosis, classification, monitoring, and treatment of retinal disease using AI. We have prioritised diabetic retinopathy, age-related macular degeneration, inherited retinal disease, and retinopathy of prematurity. There is cautious optimism that these algorithms will be integrated into routine clinical practice to facilitate access to vision-saving treatments, improve efficiency of healthcare systems, and assist clinicians in processing the ever-increasing volume of multimodal data, thereby also liberating time for doctor-patient interaction and co-development of personalised management plans.

Federated learning for diagnosis of age-related macular degeneration

This paper presents a federated learning (FL) approach to train deep learning models for classifying age-related macular degeneration (AMD) using optical coherence tomography image data. We employ the use of residual network and vision transformer encoders for the normal vs. AMD binary classification, integrating four unique domain adaptation techniques to address domain shift issues caused by heterogeneous data distribution in different institutions. Experimental results indicate that FL strategies can achieve competitive performance similar to centralized models even though each local model has access to a portion of the training data. Notably, the Adaptive Personalization FL strategy stood out in our FL evaluations, consistently delivering high performance across all tests due to its additional local model. Furthermore, the study provides valuable insights into the efficacy of simpler architectures in image classification tasks, particularly in scenarios where data privacy and decentralization are critical using both encoders. It suggests future exploration into deeper models and other FL strategies for a more nuanced understanding of these models' performance. Data and code are available at https://github.com/QIAIUNCC/FL_UNCC_QIAI.

Review of smartphone funduscopy for diabetic retinopathy screening

I detail advances in funduscopy diagnostic systems integrating smartphones. Smartphone funduscopy devices are comprised of lens devices connecting with smartphones and software applications to be used for mobile retinal image capturing and diagnosis of diabetic retinopathy. This is particularly beneficial to automate and mobilize retinopathy screening techniques and methods in remote and rural areas as those diabetes patients are often not meeting the required regular screening for diabetic retinopathy. Smartphone retinal image grading systems enable retinopathy to be screened remotely as teleophthalmology or as a stand-alone point-of-care-testing system. Smartphone funduscopy aims to avoid the need for patients to be seen by expert ophthalmologists, which can reduce patient travel, time taken for images to be processed, appointment backlog, health service overhead costs, and the workload burden for expert ophthalmologists.

Radiomics-Based Prediction of Anti-VEGF Treatment Response in Neovascular Age-Related Macular Degeneration With Pigment Epithelial Detachment

Purpose

Machine learning models based on radiomic feature extraction from clinical imaging data provide effective and interpretable means for clinical decision making. This pilot study evaluated whether radiomics features in baseline optical coherence tomography (OCT) images of eyes with pigment epithelial detachment (PED) associated with neovascular age-related macular degeneration (nAMD) can predict treatment response to as-needed anti-vascular endothelial growth factor (VEGF) therapy.

Methods

Thirty-nine eyes of patients with PED undergoing anti-VEGF therapy were included. All eyes underwent a loading dose followed by as-needed therapy. OCT images at baseline, month 3, and month 6 were analyzed. Images were manually separated into non-responding, recurring, and responding eyes based on the presence or absence of subretinal fluid at month 6. PED radiomics features were then extracted from each image and images were classified as responding or recurring using a machine learning classifier applied to the radiomics features.

Results

Linear discriminant analysis classification of baseline features as responsive versus recurring resulted in classification performance of 64.0% (95% confidence interval [CI] = 0.63-0.65), area under the curve (AUC = 0.78, 95% CI = 0.72-0.82), sensitivity 0.79 (95% CI = 0.63-0.87), and specificity 0.58 (95% CI = 0.50-0.67). Further analysis of features in recurring eyes identified a significant shift toward non-responding mean feature values over 6 months.

Conclusions

Our results demonstrate the use of radiomics features as predictors for treatment response to as-needed anti-VEGF therapy. Our study demonstrates the potential for radiomics feature in clinical decision support for personalizing anti-VEGF therapy.

Translational Relevance

The ability to use PED texture features to predict treatment response facilitates personalized clinical decision making.

Deep Learning Applications to Classification and Detection of Age-Related Macular Degeneration on Optical Coherence Tomography Imaging: A Review

Age-related macular degeneration (AMD) is one of the leading causes of blindness in the elderly, more commonly in developed countries. Optical coherence tomography (OCT) is a non-invasive imaging device widely used for the diagnosis and management of AMD. Deep learning (DL) uses multilayered artificial neural networks (NN) for feature extraction, and is the cutting-edge technique for medical image analysis for diagnostic and prognostication purposes. Application of DL models to OCT image analysis has garnered significant interest in recent years. In this review, we aimed to summarize studies focusing on DL models used in classification and detection of AMD. Additionally, we provide a brief introduction to other DL applications in AMD, such as segmentation, prediction/prognostication, and models trained on multimodal imaging.

Segmentation of paracentral acute middle maculopathy lesions in spectral-domain optical coherence tomography images through weakly supervised deep convolutional networks

BACKGROUND AND OBJECTIVES

Spectral-domain optical coherence tomography (SD-OCT) is a valuable tool for non-invasive imaging of the retina, allowing the discovery and visualization of localized lesions, the presence of which is associated with eye diseases. The present study introduces X-Net, a weakly supervised deep-learning framework for automated segmentation of paracentral acute middle maculopathy (PAMM) lesions in retinal SD-OCT images. Despite recent advances in the development of automatic methods for clinical analysis of OCT scans, there remains a scarcity of studies focusing on the automated detection of small retinal focal lesions. Additionally, most existing solutions depend on supervised learning, which can be time-consuming and require extensive image labeling, whereas X-Net offers a solution to these challenges. As far as we can determine, no prior study has addressed the segmentation of PAMM lesions in SD-OCT images.

METHODS

This study leverages 133 SD-OCT retinal images, each containing instances of paracentral acute middle maculopathy lesions. A team of eye experts annotated the PAMM lesions in these images using bounding boxes. Then, labeled data were used to train a U-Net that performs pre-segmentation, producing region labels of pixel-level accuracy. To attain a highly-accurate final segmentation, we introduced X-Net, a novel neural network made up of a master and a slave U-Net. During training, it takes the expert annotated, and pixel-level pre-segment annotated images and employs sophisticated strategies to ensure the highest segmentation accuracy.

RESULTS

The proposed method was rigorously evaluated on clinical retinal images excluded from training and achieved an accuracy of 99% with a high level of similarity between the automatic segmentation and expert annotation, as demonstrated by a mean Intersection-over-Union of 0.8. Alternative methods were tested on the same data. Single-stage neural networks proved insufficient for achieving satisfactory results, confirming that more advanced solutions, such as the proposed method, are necessary. We also found that X-Net using Attention U-net for both the pre-segmentation and X-Net arms for the final segmentation shows comparable performance to the proposed method, suggesting that the proposed approach remains a viable solution even when implemented with variants of the classic U-Net.

CONCLUSIONS

The proposed method exhibits reasonably high performance, validated through quantitative and qualitative evaluations. Medical eye specialists have also verified its validity and accuracy. Thus, it could be a viable tool in the clinical assessment of the retina. Additionally, the demonstrated approach for annotating the training set has proven to be effective in reducing the expert workload.

AI based image analysis of red blood cells in oscillating microchannels

The flow dynamics of red blood cells in vivo in blood capillaries and in vitro in microfluidic channels is complex. Cells can obtain different shapes such as discoid, parachute, slipper-like shapes and various intermediate states depending on flow conditions and their viscoelastic properties. We use artificial intelligence based analysis of red blood cells (RBCs) in an oscillating microchannel to distinguish healthy red blood cells from red blood cells treated with formaldehyde to chemically modify their viscoelastic behavior. We used TensorFlow to train and validate a deep learning model and achieved a testing accuracy of over 97%. This method is a first step to a non-invasive, label-free characterization of diseased red blood cells and will be useful for diagnostic purposes in haematology labs. This method provides quantitative data on the number of affected cells based on single cell classification.

Recent evidence of economic evaluation of artificial intelligence in ophthalmology

PURPOSE OF REVIEW

Health economic evaluation (HEE) is essential for assessing value of health interventions, including artificial intelligence. Recent approaches, current challenges, and future directions of HEE of artificial intelligence in ophthalmology are reviewed.

RECENT FINDINGS

Majority of recent HEEs of artificial intelligence in ophthalmology were for diabetic retinopathy screening. Two models, one conducted in the rural USA (5-year period) and another in China (35-year period), found artificial intelligence to be more cost-effective than without screening for diabetic retinopathy. Two additional models, which compared artificial intelligence with human screeners in Brazil and Thailand for the lifetime of patients, found artificial intelligence to be more expensive from a healthcare system perspective. In the Thailand analysis, however, artificial intelligence was less expensive when opportunity loss from blindness was included. An artificial intelligence model for screening retinopathy of prematurity was cost-effective in the USA. A model for screening age-related macular degeneration in Japan and another for primary angle close in China did not find artificial intelligence to be cost-effective, compared with no screening. The costs of artificial intelligence varied widely in these models.

SUMMARY

Like other medical fields, there is limited evidence in assessing the value of artificial intelligence in ophthalmology and more appropriate HEE models are needed.

Toward safer ophthalmic artificial intelligence via distributed validation on real-world data

PURPOSE OF REVIEW

The current article provides an overview of the present approaches to algorithm validation, which are variable and largely self-determined, as well as solutions to address inadequacies.

RECENT FINDINGS

In the last decade alone, numerous machine learning applications have been proposed for ophthalmic diagnosis or disease monitoring. Remarkably, of these, less than 15 have received regulatory approval for implementation into clinical practice. Although there exists a vast pool of structured and relatively clean datasets from which to develop and test algorithms in the computational 'laboratory', real-world validation remains key to allow for safe, equitable, and clinically reliable implementation. Bottlenecks in the validation process stem from a striking paucity of regulatory guidance surrounding safety and performance thresholds, lack of oversight on critical postdeployment monitoring and context-specific recalibration, and inherent complexities of heterogeneous disease states and clinical environments. Implementation of secure, third-party, unbiased, pre and postdeployment validation offers the potential to address existing shortfalls in the validation process.

SUMMARY

Given the criticality of validation to the algorithm pipeline, there is an urgent need for developers, machine learning researchers, and end-user clinicians to devise a consensus approach, allowing for the rapid introduction of safe, equitable, and clinically valid machine learning implementations.

Large language models encode clinical knowledge

Large language models (LLMs) have demonstrated impressive capabilities, but the bar for clinical applications is high. Attempts to assess the clinical knowledge of models typically rely on automated evaluations based on limited benchmarks. Here, to address these limitations, we present MultiMedQA, a benchmark combining six existing medical question answering datasets spanning professional medicine, research and consumer queries and a new dataset of medical questions searched online, HealthSearchQA. We propose a human evaluation framework for model answers along multiple axes including factuality, comprehension, reasoning, possible harm and bias. In addition, we evaluate Pathways Language Model1 (PaLM, a 540-billion parameter LLM) and its instruction-tuned variant, Flan-PaLM2 on MultiMedQA. Using a combination of prompting strategies, Flan-PaLM achieves state-of-the-art accuracy on every MultiMedQA multiple-choice dataset (MedQA3, MedMCQA4, PubMedQA5 and Measuring Massive Multitask Language Understanding (MMLU) clinical topics6), including 67.6% accuracy on MedQA (US Medical Licensing Exam-style questions), surpassing the prior state of the art by more than 17%. However, human evaluation reveals key gaps. To resolve this, we introduce instruction prompt tuning, a parameter-efficient approach for aligning LLMs to new domains using a few exemplars. The resulting model, Med-PaLM, performs encouragingly, but remains inferior to clinicians. We show that comprehension, knowledge recall and reasoning improve with model scale and instruction prompt tuning, suggesting the potential utility of LLMs in medicine. Our human evaluations reveal limitations of today's models, reinforcing the importance of both evaluation frameworks and method development in creating safe, helpful LLMs for clinical applications.

Optical coherence tomography imaging biomarkers associated with neovascular age-related macular degeneration: a systematic review

The aim of this systematic literature review is twofold, (1) detail the impact of retinal biomarkers identifiable via optical coherence tomography (OCT) on disease progression and response to treatment in neovascular age-related macular degeneration (nAMD) and (2) establish which biomarkers are currently identifiable by artificial intelligence (AI) models and the utilisation of this technology. Following the PRISMA guidelines, PubMed was searched for peer-reviewed publications dated between January 2016 and January 2022.

POPULATION

Patients diagnosed with nAMD with OCT imaging.

SETTINGS

Comparable settings to NHS hospitals.

STUDY DESIGNS

Randomised controlled trials, prospective/retrospective cohort studies and review articles. From 228 articles, 130 were full-text reviewed, 50 were removed for falling outside the scope of this review with 10 added from the author's inventory, resulting in the inclusion of 90 articles. From 9 biomarkers identified; intraretinal fluid (IRF), subretinal fluid, pigment epithelial detachment, subretinal hyperreflective material (SHRM), retinal pigmental epithelial (RPE) atrophy, drusen, outer retinal tabulation (ORT), hyperreflective foci (HF) and retinal thickness, 5 are considered pertinent to nAMD disease progression; IRF, SHRM, drusen, ORT and HF. A number of these biomarkers can be classified using current AI models. Significant retinal biomarkers pertinent to disease activity and progression in nAMD are identifiable via OCT; IRF being the most important in terms of the significant impact on visual outcome. Incorporating AI into ophthalmology practice is a promising advancement towards automated and reproducible analyses of OCT data with the ability to diagnose disease and predict future disease conversion.

SYSTEMATIC REVIEW REGISTRATION

This review has been registered with PROSPERO (registration ID: CRD42021233200).

The Predictive Capabilities of Artificial Intelligence-Based OCT Analysis for Age-Related Macular Degeneration Progression-A Systematic Review

The era of artificial intelligence (AI) has revolutionized our daily lives and AI has become a powerful force that is gradually transforming the field of medicine. Ophthalmology sits at the forefront of this transformation thanks to the effortless acquisition of an abundance of imaging modalities. There has been tremendous work in the field of AI for retinal diseases, with age-related macular degeneration being at the top of the most studied conditions. The purpose of the current systematic review was to identify and evaluate, in terms of strengths and limitations, the articles that apply AI to optical coherence tomography (OCT) images in order to predict the future evolution of age-related macular degeneration (AMD) during its natural history and after treatment in terms of OCT morphological structure and visual function. After a thorough search through seven databases up to 1 January 2022 using the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines, 1800 records were identified. After screening, 48 articles were selected for full-text retrieval and 19 articles were finally included. From these 19 articles, 4 articles concentrated on predicting the anti-VEGF requirement in neovascular AMD (nAMD), 4 articles focused on predicting anti-VEGF efficacy in nAMD patients, 3 articles predicted the conversion from early or intermediate AMD (iAMD) to nAMD, 1 article predicted the conversion from iAMD to geographic atrophy (GA), 1 article predicted the conversion from iAMD to both nAMD and GA, 3 articles predicted the future growth of GA and 3 articles predicted the future outcome for visual acuity (VA) after anti-VEGF treatment in nAMD patients. Since using AI methods to predict future changes in AMD is only in its initial phase, a systematic review provides the opportunity of setting the context of previous work in this area and can present a starting point for future research.

User-acceptability of an automated telephone call for post-operative follow-up after uncomplicated cataract surgery

BACKGROUND

Innovative technology is recommended to address the current capacity challenges facing the NHS. This study evaluates the patient acceptability of automated telephone follow-up after routine cataract surgery using Dora (Ufonia Limited, Oxford, United Kingdom), which to our knowledge is the first AI-powered clinical assistant to be used in the NHS. Dora has a natural-language, phone conversation with patients about their symptoms after cataract surgery.

METHODS

This is a prospective mixed-methods cohort study that was conducted at Buckinghamshire Healthcare NHS Foundation Trust. All patients who were followed up using Dora were asked to give a Net Promoter Score (NPS), and 24 patients were randomly selected to complete the validated Telephone Usability Questionnaire (TUQ) as well as extended semi-structured interviews that underwent thematic analysis.

RESULTS

A total of 170 autonomous calls were completed. The median NPS score was 9 out of 10. The TUQ (scored out of 5) showed high rates of acceptability, with an overall mean score of 4.0. Simplicity, time saving, and ease of use scored the highest with a median of 5, whilst 'speaking to Dora feels the same as speaking to a clinician' scored a median of 3. The main themes extracted from the qualitative data were 'I can see why you're doing it', 'It went quite well actually', 'I just trust human beings I suppose'.

CONCLUSION

We found high levels of patient acceptability when using Dora across three acceptability measures. Dora provides a potential solution to reduce pressure on hospital capacity whilst also providing a convenient service for patients.

Artificial intelligence to distinguish retinal vein occlusion patients using color fundus photographs

PURPOSE

Our aim is to establish an AI model for distinguishing color fundus photographs (CFP) of RVO patients from normal individuals.

METHODS

The training dataset included 2013 CFP from fellow eyes of RVO patients and 8536 age- and gender-matched normal CFP. Model performance was assessed in two independent testing datasets. We evaluated the performance of the AI model using the area under the receiver operating characteristic curve (AUC), accuracy, precision, specificity, sensitivity, and confusion matrices. We further explained the probable clinical relevance of the AI by extracting and comparing features of the retinal images.

RESULTS

Our model achieved an average AUC was 0.9866 (95% CI: 0.9805-0.9918), accuracy was 0.9534 (95% CI: 0.9421-0.9639), precision was 0.9123 (95% CI: 0.8784-9453), specificity was 0.9810 (95% CI: 0.9729-0.9884), and sensitivity was 0.8367 (95% CI: 0.7953-0.8756) for identifying fundus images of RVO patients in training dataset. In independent external datasets 1, the AUC of the RVO group was 0.8102 (95% CI: 0.7979-0.8226), the accuracy of 0.7752 (95% CI: 0.7633-0.7875), the precision of 0.7041 (95% CI: 0.6873-0.7211), specificity of 0.6499 (95% CI: 0.6305-0.6679) and sensitivity of 0.9124 (95% CI: 0.9004-0.9241) for RVO group. There were significant differences in retinal arteriovenous ratio, optic cup to optic disc ratio, and optic disc tilt angle (p = 0.001, p = 0.0001, and p = 0.0001, respectively) between the two groups in training dataset.

CONCLUSION

We trained an AI model to classify color fundus photographs of RVO patients with stable performance both in internal and external datasets. This may be of great importance for risk prediction in patients with retinal venous occlusion.

Artificial intelligence in ophthalmology: The path to the real-world clinic

Artificial intelligence (AI) has great potential to transform healthcare by enhancing the workflow and productivity of clinicians, enabling existing staff to serve more patients, improving patient outcomes, and reducing health disparities. In the field of ophthalmology, AI systems have shown performance comparable with or even better than experienced ophthalmologists in tasks such as diabetic retinopathy detection and grading. However, despite these quite good results, very few AI systems have been deployed in real-world clinical settings, challenging the true value of these systems. This review provides an overview of the current main AI applications in ophthalmology, describes the challenges that need to be overcome prior to clinical implementation of the AI systems, and discusses the strategies that may pave the way to the clinical translation of these systems.

Deep learning for automated detection of neovascular leakage on ultra-widefield fluorescein angiography in diabetic retinopathy

Diabetic retinopathy is a leading cause of blindness in working-age adults worldwide. Neovascular leakage on fluorescein angiography indicates progression to the proliferative stage of diabetic retinopathy, which is an important distinction that requires timely ophthalmic intervention with laser or intravitreal injection treatment to reduce the risk of severe, permanent vision loss. In this study, we developed a deep learning algorithm to detect neovascular leakage on ultra-widefield fluorescein angiography images obtained from patients with diabetic retinopathy. The algorithm, an ensemble of three convolutional neural networks, was able to accurately classify neovascular leakage and distinguish this disease marker from other angiographic disease features. With additional real-world validation and testing, our algorithm could facilitate identification of neovascular leakage in the clinical setting, allowing timely intervention to reduce the burden of blinding diabetic eye disease.

Deep survival modeling of longitudinal retinal OCT volumes for predicting the onset of atrophy in patients with intermediate AMD

In patients with age-related macular degeneration (AMD), the risk of progression to late stages is highly heterogeneous, and the prognostic imaging biomarkers remain unclear. We propose a deep survival model to predict the progression towards the late atrophic stage of AMD. The model combines the advantages of survival modelling, accounting for time-to-event and censoring, and the advantages of deep learning, generating prediction from raw 3D OCT scans, without the need for extracting a predefined set of quantitative biomarkers. We demonstrate, in an extensive set of evaluations, based on two large longitudinal datasets with 231 eyes from 121 patients for internal evaluation, and 280 eyes from 140 patients for the external evaluation, that this model improves the risk estimation performance over standard deep learning classification models.

Artificial intelligence to estimate the tear film breakup time and diagnose dry eye disease

The use of artificial intelligence (AI) in the diagnosis of dry eye disease (DED) remains limited due to the lack of standardized image formats and analysis models. To overcome these issues, we used the Smart Eye Camera (SEC), a video-recordable slit-lamp device, and collected videos of the anterior segment of the eye. This study aimed to evaluate the accuracy of the AI algorithm in estimating the tear film breakup time and apply this model for the diagnosis of DED according to the Asia Dry Eye Society (ADES) DED diagnostic criteria. Using the retrospectively corrected DED videos of 158 eyes from 79 patients, 22,172 frames were annotated by the DED specialist to label whether or not the frame had breakup. The AI algorithm was developed using the training dataset and machine learning. The DED criteria of the ADES was used to determine the diagnostic performance. The accuracy of tear film breakup time estimation was 0.789 (95% confidence interval (CI) 0.769-0.809), and the area under the receiver operating characteristic curve of this AI model was 0.877 (95% CI 0.861-0.893). The sensitivity and specificity of this AI model for the diagnosis of DED was 0.778 (95% CI 0.572-0.912) and 0.857 (95% CI 0.564-0.866), respectively. We successfully developed a novel AI-based diagnostic model for DED. Our diagnostic model has the potential to enable ophthalmology examination outside hospitals and clinics.

Multitask Learning for Activity Detection in Neovascular Age-Related Macular Degeneration

Purpose

The purpose of this study was to provide a comparison of performance and explainability of a multitask convolutional deep neuronal network to single-task networks for activity detection in neovascular age-related macular degeneration (nAMD).

Methods

From 70 patients (46 women and 24 men) who attended the University Eye Hospital Tübingen, 3762 optical coherence tomography B-scans (right eye = 2011 and left eye = 1751) were acquired with Heidelberg Spectralis, Heidelberg, Germany. B-scans were graded by a retina specialist and an ophthalmology resident, and then used to develop a multitask deep learning model to predict disease activity in neovascular age-related macular degeneration along with the presence of sub- and intraretinal fluid. We used performance metrics for comparison to single-task networks and visualized the deep neural network (DNN)-based decision with t-distributed stochastic neighbor embedding and clinically validated saliency mapping techniques.

Results

The multitask model surpassed single-task networks in accuracy for activity detection (94.2% vs. 91.2%). The area under the curve of the receiver operating curve was 0.984 for the multitask model versus 0.974 for the single-task model. Furthermore, compared to single-task networks, visualizations via t-distributed stochastic neighbor embedding and saliency maps highlighted that multitask networks' decisions for activity detection in neovascular age-related macular degeneration were highly consistent with the presence of both sub- and intraretinal fluid.

Conclusions

Multitask learning increases the performance of neuronal networks for predicting disease activity, while providing clinicians with an easily accessible decision control, which resembles human reasoning.

Translational Relevance

By improving nAMD activity detection performance and transparency of automated decisions, multitask DNNs can support the translation of machine learning research into clinical decision support systems for nAMD activity detection.

Learn Single-horizon Disease Evolution for Predictive Generation of Post-therapeutic Neovascular Age-related Macular Degeneration

BACKGROUND AND OBJECTIVE

Most of the existing disease prediction methods in the field of medical image processing fall into two classes, namely image-to-category predictions and image-to-parameter predictions.Few works have focused on image-to-image predictions. Different from multi-horizon predictions in other fields, ophthalmologists prefer to show more confidence in single-horizon predictions due to the low tolerance of predictive risk.

METHODS

We propose a single-horizon disease evolution network (SHENet) to predictively generate post-therapeutic SD-OCT images by inputting pre-therapeutic SD-OCT images with neovascular age-related macular degeneration (nAMD). In SHENet, a feature encoder converts the input SD-OCT images to deep features, then a graph evolution module predicts the process of disease evolution in high-dimensional latent space and outputs the predicted deep features, and lastly, feature decoder recovers the predicted deep features to SD-OCT images. We further propose an evolution reinforcement module to ensure the effectiveness of disease evolution learning and obtain realistic SD-OCT images by adversarial training.

RESULTS

SHENet is validated on 383 SD-OCT cubes of 22 nAMD patients based on three well-designed schemes (P-0, P-1 and P-M) based on the quantitative and qualitative evaluations. Three metrics (PSNR, SSIM, 1-LPIPS) are used here for quantitative evaluations. Compared with other generative methods, the generative SD-OCT images of SHENet have the highest image quality (P-0: 23.659, P-1: 23.875, P-M: 24.198) by PSNR. Besides, SHENet achieves the best structure protection (P-0: 0.326, P-1: 0.337, P-M: 0.349) by SSIM and content prediction (P-0: 0.609, P-1: 0.626, P-M: 0.642) by 1-LPIPS. Qualitative evaluations also demonstrate that SHENet has a better visual effect than other methods.

CONCLUSIONS

SHENet can generate post-therapeutic SD-OCT images with both high prediction performance and good image quality, which has great potential to help ophthalmologists forecast the therapeutic effect of nAMD.

Deep ensemble learning for automated non-advanced AMD classification using optimized retinal layer segmentation and SD-OCT scans

BACKGROUND

Accurate retinal layer segmentation in optical coherence tomography (OCT) images is crucial for quantitatively analyzing age-related macular degeneration (AMD) and monitoring its progression. However, previous retinal segmentation models depend on experienced experts and manually annotating retinal layers is time-consuming. On the other hand, accuracy of AMD diagnosis is directly related to the segmentation model's performance. To address these issues, we aimed to improve AMD detection using optimized retinal layer segmentation and deep ensemble learning.

METHOD

We integrated a graph-cut algorithm with a cubic spline to automatically annotate 11 retinal boundaries. The refined images were fed into a deep ensemble mechanism that combined a Bagged Tree and end-to-end deep learning classifiers. We tested the developed deep ensemble model on internal and external datasets.

RESULTS

The total error rates for our segmentation model using the boundary refinement approach was significantly lower than OCT Explorer segmentations (1.7% vs. 7.8%, p-value = 0.03). We utilized the refinement approach to quantify 169 imaging features using Zeiss SD-OCT volume scans. The presence of drusen and thickness of total retina, neurosensory retina, and ellipsoid zone to inner-outer segment (EZ-ISOS) thickness had higher contributions to AMD classification compared to other features. The developed ensemble learning model obtained a higher diagnostic accuracy in a shorter time compared with two human graders. The area under the curve (AUC) for normal vs. early AMD was 99.4%.

CONCLUSION

Testing results showed that the developed framework is repeatable and effective as a potentially valuable tool in retinal imaging research.

Uncertainty, Evidence, and the Integration of Machine Learning into Medical Practice

In light of recent advances in machine learning for medical applications, the automation of medical diagnostics is imminent. That said, before machine learning algorithms find their way into clinical practice, various problems at the epistemic level need to be overcome. In this paper, we discuss different sources of uncertainty arising for clinicians trying to evaluate the trustworthiness of algorithmic evidence when making diagnostic judgments. Thereby, we examine many of the limitations of current machine learning algorithms (with deep learning in particular) and highlight their relevance for medical diagnostics. Among the problems we inspect are the theoretical foundations of deep learning (which are not yet adequately understood), the opacity of algorithmic decisions, and the vulnerabilities of machine learning models, as well as concerns regarding the quality of medical data used to train the models. Building on this, we discuss different desiderata for an uncertainty amelioration strategy that ensures that the integration of machine learning into clinical settings proves to be medically beneficial in a meaningful way.

Automated large-scale prediction of exudative AMD progression using machine-read OCT biomarkers

Age-related Macular Degeneration (AMD) is a major cause of irreversible vision loss in individuals over 55 years old in the United States. One of the late-stage manifestations of AMD, and a major cause of vision loss, is the development of exudative macular neovascularization (MNV). Optical Coherence Tomography (OCT) is the gold standard to identify fluid at different levels within the retina. The presence of fluid is considered the hallmark to define the presence of disease activity. Anti-vascular growth factor (anti-VEGF) injections can be used to treat exudative MNV. However, given the limitations of anti-VEGF treatment, as burdensome need for frequent visits and repeated injections to sustain efficacy, limited durability of the treatment, poor or no response, there is a great interest in detecting early biomarkers associated with a higher risk for AMD progression to exudative forms in order to optimize the design of early intervention clinical trials. The annotation of structural biomarkers on optical coherence tomography (OCT) B-scans is a laborious, complex and time-consuming process, and discrepancies between human graders can introduce variability into this assessment. To address this issue, a deep-learning model (SLIVER-net) was proposed, which could identify AMD biomarkers on structural OCT volumes with high precision and without human supervision. However, the validation was performed on a small dataset, and the true predictive power of these detected biomarkers in the context of a large cohort has not been evaluated. In this retrospective cohort study, we perform the largest-scale validation of these biomarkers to date. We also assess how these features combined with other EHR data (demographics, comorbidities, etc) affect and/or improve the prediction performance relative to known factors. Our hypothesis is that these biomarkers can be identified by a machine learning algorithm without human supervision, in a way that they preserve their predictive nature. The way we test this hypothesis is by building several machine learning models utilizing these machine-read biomarkers and assessing their added predictive power. We found that not only can we show that the machine-read OCT B-scan biomarkers are predictive of AMD progression, we also observe that our proposed combined OCT and EHR data-based algorithm outperforms the state-of-the-art solution in clinically relevant metrics and provides actionable information which has the potential to improve patient care. In addition, it provides a framework for automated large-scale processing of OCT volumes, making it possible to analyze vast archives without human supervision.

[Reporting guidelines for clinical trial reports for interventions involving artificial intelligence: the CONSORT-AI extensionDiretrizes para relatórios de ensaios clínicos com intervenções que utilizam inteligência artificial: a extensão CONSORT-AI]

The CONSORT 2010 statement provides minimum guidelines for reporting randomized trials. Its widespread use has been instrumental in ensuring transparency in the evaluation of new interventions. More recently, there has been a growing recognition that interventions involving artificial intelligence (AI) need to undergo rigorous, prospective evaluation to demonstrate impact on health outcomes. The CONSORT-AI (Consolidated Standards of Reporting Trials-Artificial Intelligence) extension is a new reporting guideline for clinical trials evaluating interventions with an AI component. It was developed in parallel with its companion statement for clinical trial protocols: SPIRIT-AI (Standard Protocol Items: Recommendations for Interventional Trials-Artificial Intelligence). Both guidelines were developed through a staged consensus process involving literature review and expert consultation to generate 29 candidate items, which were assessed by an international multi-stakeholder group in a two-stage Delphi survey (103 stakeholders), agreed upon in a two-day consensus meeting (31 stakeholders) and refined through a checklist pilot (34 participants). The CONSORT-AI extension includes 14 new items that were considered sufficiently important for AI interventions that they should be routinely reported in addition to the core CONSORT 2010 items. CONSORT-AI recommends that investigators provide clear descriptions of the AI intervention, including instructions and skills required for use, the setting in which the AI intervention is integrated, the handling of inputs and outputs of the AI intervention, the human-AI interaction and provision of an analysis of error cases. CONSORT-AI will help promote transparency and completeness in reporting clinical trials for AI interventions. It will assist editors and peer reviewers, as well as the general readership, to understand, interpret and critically appraise the quality of clinical trial design and risk of bias in the reported outcomes.

[Guidelines for clinical trial protocols for interventions involving artificial intelligence: the SPIRIT-AI extensionDiretrizes para protocolos de ensaios clínicos com intervenções que utilizam inteligência artificial: a extensão SPIRIT-AI]

The SPIRIT 2013 statement aims to improve the completeness of clinical trial protocol reporting by providing evidence-based recommendations for the minimum set of items to be addressed. This guidance has been instrumental in promoting transparent evaluation of new interventions. More recently, there has been a growing recognition that interventions involving artificial intelligence (AI) need to undergo rigorous, prospective evaluation to demonstrate their impact on health outcomes. The SPIRIT-AI (Standard Protocol Items: Recommendations for Interventional Trials-Artificial Intelligence) extension is a new reporting guideline for clinical trial protocols evaluating interventions with an AI component. It was developed in parallel with its companion statement for trial reports: CONSORT-AI (Consolidated Standards of Reporting Trials-Artificial Intelligence). Both guidelines were developed through a staged consensus process involving literature review and expert consultation to generate 26 candidate items, which were consulted upon by an international multi-stakeholder group in a two-stage Delphi survey (103 stakeholders), agreed upon in a consensus meeting (31 stakeholders) and refined through a checklist pilot (34 participants). The SPIRIT-AI extension includes 15 new items that were considered sufficiently important for clinical trial protocols of AI interventions. These new items should be routinely reported in addition to the core SPIRIT 2013 items. SPIRIT-AI recommends that investigators provide clear descriptions of the AI intervention, including instructions and skills required for use, the setting in which the AI intervention will be integrated, considerations for the handling of input and output data, the human-AI interaction and analysis of error cases. SPIRIT-AI will help promote transparency and completeness for clinical trial protocols for AI interventions. Its use will assist editors and peer reviewers, as well as the general readership, to understand, interpret and critically appraise the design and risk of bias for a planned clinical trial.

Deep Learning in Optical Coherence Tomography Angiography: Current Progress, Challenges, and Future Directions

Optical coherence tomography angiography (OCT-A) provides depth-resolved visualization of the retinal microvasculature without intravenous dye injection. It facilitates investigations of various retinal vascular diseases and glaucoma by assessment of qualitative and quantitative microvascular changes in the different retinal layers and radial peripapillary layer non-invasively, individually, and efficiently. Deep learning (DL), a subset of artificial intelligence (AI) based on deep neural networks, has been applied in OCT-A image analysis in recent years and achieved good performance for different tasks, such as image quality control, segmentation, and classification. DL technologies have further facilitated the potential implementation of OCT-A in eye clinics in an automated and efficient manner and enhanced its clinical values for detecting and evaluating various vascular retinopathies. Nevertheless, the deployment of this combination in real-world clinics is still in the "proof-of-concept" stage due to several limitations, such as small training sample size, lack of standardized data preprocessing, insufficient testing in external datasets, and absence of standardized results interpretation. In this review, we introduce the existing applications of DL in OCT-A, summarize the potential challenges of the clinical deployment, and discuss future research directions.

iERM: An Interpretable Deep Learning System to Classify Epiretinal Membrane for Different Optical Coherence Tomography Devices: A Multi-Center Analysis

Background: Epiretinal membranes (ERM) have been found to be common among individuals >50 years old. However, the severity grading assessment for ERM based on optical coherence tomography (OCT) images has remained a challenge due to lacking reliable and interpretable analysis methods. Thus, this study aimed to develop a two-stage deep learning (DL) system named iERM to provide accurate automatic grading of ERM for clinical practice. Methods: The iERM was trained based on human segmentation of key features to improve classification performance and simultaneously provide interpretability to the classification results. We developed and tested iERM using a total of 4547 OCT B-Scans of four different commercial OCT devices that were collected from nine international medical centers. Results: As per the results, the integrated network effectively improved the grading performance by 1−5.9% compared with the traditional classification DL model and achieved high accuracy scores of 82.9%, 87.0%, and 79.4% in the internal test dataset and two external test datasets, respectively. This is comparable to retinal specialists whose average accuracy scores are 87.8% and 79.4% in two external test datasets. Conclusion: This study proved to be a benchmark method to improve the performance and enhance the interpretability of the traditional DL model with the implementation of segmentation based on prior human knowledge. It may have the potential to provide precise guidance for ERM diagnosis and treatment.

Machine learning-based mortality prediction models using national liver transplantation registries are feasible but have limited utility across countries

Many countries curate national registries of liver transplant (LT) data. These registries are often used to generate predictive models; however, potential performance and transferability of these models remain unclear. We used data from 3 national registries and developed machine learning algorithm (MLA)-based models to predict 90-day post-LT mortality within and across countries. Predictive performance and external validity of each model were assessed. Prospectively collected data of adult patients (aged ≥18 years) who underwent primary LTs between January 2008 and December 2018 from the Canadian Organ Replacement Registry (Canada), National Health Service Blood and Transplantation (United Kingdom), and United Network for Organ Sharing (United States) were used to develop MLA models to predict 90-day post-LT mortality. Models were developed using each registry individually (based on variables inherent to the individual databases) and using all 3 registries combined (variables in common between the registries [harmonized]). The model performance was evaluated using area under the receiver operating characteristic (AUROC) curve. The number of patients included was as follows: Canada, n = 1214; the United Kingdom, n = 5287; and the United States, n = 59,558. The best performing MLA-based model was ridge regression across both individual registries and harmonized data sets. Model performance diminished from individualized to the harmonized registries, especially in Canada (individualized ridge: AUROC, 0.74; range, 0.73-0.74; harmonized: AUROC, 0.68; range, 0.50-0.73) and US (individualized ridge: AUROC, 0.71; range, 0.70-0.71; harmonized: AUROC, 0.66; range, 0.66-0.66) data sets. External model performance across countries was poor overall. MLA-based models yield a fair discriminatory potential when used within individual databases. However, the external validity of these models is poor when applied across countries. Standardization of registry-based variables could facilitate the added value of MLA-based models in informing decision making in future LTs.