Automated Diagnosis of Plus Disease in Retinopathy of Prematurity Using Deep Convolutional Neural Networks

Assessment of Generative Adversarial Networks Model for Synthetic Optical Coherence Tomography Images of Retinal Disorders

PURPOSE

To assess whether a generative adversarial network (GAN) could synthesize realistic optical coherence tomography (OCT) images that satisfactorily serve as the educational images for retinal specialists, and the training datasets for the classification of various retinal disorders using deep learning (DL).

METHODS

The GANs architecture was adopted to synthesize high-resolution OCT images trained on a publicly available OCT dataset, including urgent referrals (37,206 OCT images from eyes with choroidal neovascularization, and 11,349 OCT images from eyes with diabetic macular edema) and nonurgent referrals (8617 OCT images from eyes with drusen, and 51,140 OCT images from normal eyes). Four hundred real and synthetic OCT images were evaluated by two retinal specialists (with over 10 years of clinical retinal experience) to assess image quality. We further trained two DL models on either real or synthetic datasets and compared the performance of urgent versus nonurgent referrals diagnosis tested on a local (1000 images from the public dataset) and clinical validation dataset (278 images from Shanghai Shibei Hospital).

RESULTS

The image quality of real versus synthetic OCT images was similar as assessed by two retinal specialists. The accuracy of discrimination of real versus synthetic OCT images was 59.50% for retinal specialist 1 and 53.67% for retinal specialist 2. For the local dataset, the DL model trained on real (DL_Model_R) and synthetic OCT images (DL_Model_S) had an area under the curve (AUC) of 0.99, and 0.98, respectively. For the clinical dataset, the AUC was 0.94 for DL_Model_R and 0.90 for DL_Model_S.

CONCLUSIONS

The GAN synthetic OCT images can be used by clinicians for educational purposes and for developing DL algorithms.

TRANSLATIONAL RELEVANCE

The medical image synthesis based on GANs is promising in humans and machines to fulfill clinical tasks.

Accounting for data variability in multi-institutional distributed deep learning for medical imaging

OBJECTIVES

Sharing patient data across institutions to train generalizable deep learning models is challenging due to regulatory and technical hurdles. Distributed learning, where model weights are shared instead of patient data, presents an attractive alternative. Cyclical weight transfer (CWT) has recently been demonstrated as an effective distributed learning method for medical imaging with homogeneous data across institutions. In this study, we optimize CWT to overcome performance losses from variability in training sample sizes and label distributions across institutions.

MATERIALS AND METHODS

Optimizations included proportional local training iterations, cyclical learning rate, locally weighted minibatch sampling, and cyclically weighted loss. We evaluated our optimizations on simulated distributed diabetic retinopathy detection and chest radiograph classification.

RESULTS

Proportional local training iteration mitigated performance losses from sample size variability, achieving 98.6% of the accuracy attained by centrally hosting in the diabetic retinopathy dataset split with highest sample size variance across institutions. Locally weighted minibatch sampling and cyclically weighted loss both mitigated performance losses from label distribution variability, achieving 98.6% and 99.1%, respectively, of the accuracy attained by centrally hosting in the diabetic retinopathy dataset split with highest label distribution variability across institutions.

DISCUSSION

Our optimizations to CWT improve its capability of handling data variability across institutions. Compared to CWT without optimizations, CWT with optimizations achieved performance significantly closer to performance from centrally hosting.

CONCLUSION

Our work is the first to identify and address challenges of sample size and label distribution variability in simulated distributed deep learning for medical imaging. Future work is needed to address other sources of real-world data variability.

Different fundus imaging modalities and technical factors in AI screening for diabetic retinopathy: a review

BACKGROUND

Effective screening is a desirable method for the early detection and successful treatment for diabetic retinopathy, and fundus photography is currently the dominant medium for retinal imaging due to its convenience and accessibility. Manual screening using fundus photographs has however involved considerable costs for patients, clinicians and national health systems, which has limited its application particularly in less-developed countries. The advent of artificial intelligence, and in particular deep learning techniques, has however raised the possibility of widespread automated screening.

MAIN TEXT

In this review, we first briefly survey major published advances in retinal analysis using artificial intelligence. We take care to separately describe standard multiple-field fundus photography, and the newer modalities of ultra-wide field photography and smartphone-based photography. Finally, we consider several machine learning concepts that have been particularly relevant to the domain and illustrate their usage with extant works.

CONCLUSIONS

In the ophthalmology field, it was demonstrated that deep learning tools for diabetic retinopathy show clinically acceptable diagnostic performance when using colour retinal fundus images. Artificial intelligence models are among the most promising solutions to tackle the burden of diabetic retinopathy management in a comprehensive manner. However, future research is crucial to assess the potential clinical deployment, evaluate the cost-effectiveness of different DL systems in clinical practice and improve clinical acceptance.

Practice Guidelines for Ocular Telehealth-Diabetic Retinopathy, Third Edition

Contributors The following document and appendices represent the third edition of the Practice Guidelines for Ocular Telehealth-Diabetic Retinopathy. These guidelines were developed by the Diabetic Retinopathy Telehealth Practice Guidelines Working Group. This working group consisted of a large number of subject matter experts in clinical applications for telehealth in ophthalmology. The editorial committee consisted of Mark B. Horton, OD, MD, who served as working group chair and Christopher J. Brady, MD, MHS, and Jerry Cavallerano, OD, PhD, who served as cochairs. The writing committees were separated into seven different categories. They are as follows: 1.Clinical/operational: Jerry Cavallerano, OD, PhD (Chair), Gail Barker, PhD, MBA, Christopher J. Brady, MD, MHS, Yao Liu, MD, MS, Siddarth Rathi, MD, MBA, Veeral Sheth, MD, MBA, Paolo Silva, MD, and Ingrid Zimmer-Galler, MD. 2.Equipment: Veeral Sheth, MD (Chair), Mark B. Horton, OD, MD, Siddarth Rathi, MD, MBA, Paolo Silva, MD, and Kristen Stebbins, MSPH. 3.Quality assurance: Mark B. Horton, OD, MD (Chair), Seema Garg, MD, PhD, Yao Liu, MD, MS, and Ingrid Zimmer-Galler, MD. 4.Glaucoma: Yao Liu, MD, MS (Chair) and Siddarth Rathi, MD, MBA. 5.Retinopathy of prematurity: Christopher J. Brady, MD, MHS (Chair) and Ingrid Zimmer-Galler, MD. 6.Age-related macular degeneration: Christopher J. Brady, MD, MHS (Chair) and Ingrid Zimmer-Galler, MD. 7.Autonomous and computer assisted detection, classification and diagnosis of diabetic retinopathy: Michael Abramoff, MD, PhD (Chair), Michael F. Chiang, MD, and Paolo Silva, MD.

ROPtool analysis of plus and pre-plus disease in narrow-field images: a multi-image quadrant-level approach

BACKGROUND

The presence of plus disease is important in determining when to treat retinopathy of prematurity (ROP), but the diagnosis of plus disease is subjective. Semiautomated computer programs (eg, ROPtool) can objectively measure retinal vascular characteristics in retinal images, but are limited by image quality. The purpose of this study was to evaluate whether ROPtool can accurately identify pre-plus and plus disease in narrow-field images of varying qualities using a new methodology that combines quadrant-level data from multiple images of a single retina.

METHODS

This was a cross-sectional study of previously collected narrow-field retinal images of infants screened for ROP. Using one imaging session per infant, we evaluated the ability of ROPtool to analyze images using our new methodology and the accuracy of ROPtool indices (tortuosity index [TI], maximum tortuosity [T_max], dilation index [DI], maximum dilation [D_max], sum of adjusted indices [SAI], and tortuosity-weighted plus [TWP]) to identify pre-plus and plus disease in images compared to clinical examination findings.

RESULTS

Of 198 eyes (from 99 infants) imaged, 769/792 quadrants (98%) were analyzable. Overall, 98% of eyes had 3-4 analyzable quadrants. For plus disease, area under the curves (AUCs) of receiver operating characteristic curves were: TWP (0.98) > TI (0.97) = T_max (0.97) > SAI (0.96) > DI (0.88) > D_max (0.84). For pre-plus or plus disease, AUCs were: TWP (0.95) > TI (0.94) = T_max (0.94) = SAI (0.94) > DI (0.86) > D_max (0.83).

CONCLUSIONS

Using a novel methodology combining quadrant-level data, ROPtool can analyze narrow-field images of varying quality to identify pre-plus and plus disease with high accuracy.

Siamese neural networks for continuous disease severity evaluation and change detection in medical imaging

Using medical images to evaluate disease severity and change over time is a routine and important task in clinical decision making. Grading systems are often used, but are unreliable as domain experts disagree on disease severity category thresholds. These discrete categories also do not reflect the underlying continuous spectrum of disease severity. To address these issues, we developed a convolutional Siamese neural network approach to evaluate disease severity at single time points and change between longitudinal patient visits on a continuous spectrum. We demonstrate this in two medical imaging domains: retinopathy of prematurity (ROP) in retinal photographs and osteoarthritis in knee radiographs. Our patient cohorts consist of 4861 images from 870 patients in the Imaging and Informatics in Retinopathy of Prematurity (i-ROP) cohort study and 10,012 images from 3021 patients in the Multicenter Osteoarthritis Study (MOST), both of which feature longitudinal imaging data. Multiple expert clinician raters ranked 100 retinal images and 100 knee radiographs from excluded test sets for severity of ROP and osteoarthritis, respectively. The Siamese neural network output for each image in comparison to a pool of normal reference images correlates with disease severity rank (ρ = 0.87 for ROP and ρ = 0.89 for osteoarthritis), both within and between the clinical grading categories. Thus, this output can represent the continuous spectrum of disease severity at any single time point. The difference in these outputs can be used to show change over time. Alternatively, paired images from the same patient at two time points can be directly compared using the Siamese neural network, resulting in an additional continuous measure of change between images. Importantly, our approach does not require manual localization of the pathology of interest and requires only a binary label for training (same versus different). The location of disease and site of change detected by the algorithm can be visualized using an occlusion sensitivity map-based approach. For a longitudinal binary change detection task, our Siamese neural networks achieve test set receiving operator characteristic area under the curves (AUCs) of up to 0.90 in evaluating ROP or knee osteoarthritis change, depending on the change detection strategy. The overall performance on this binary task is similar compared to a conventional convolutional deep-neural network trained for multi-class classification. Our results demonstrate that convolutional Siamese neural networks can be a powerful tool for evaluating the continuous spectrum of disease severity and change in medical imaging.

Telemedicine for Retinopathy of Prematurity

Background: Retinopathy of prematurity (ROP) is a disease of the retinal vasculature that remains a leading cause of childhood blindness worldwide despite improvements in the systemic care of premature newborns. Screening for ROP is effective and cost-effective, but in many areas, access to skilled examiners to conduct dilated examinations is poor. Remote screening with retinal photography is an alternative strategy that may allow for improved ROP care. Methods: The current literature was reviewed to find clinical trials and expert consensus documents on the state-of-the-art of telemedicine for ROP. Results: Several studies have confirmed the utility of telemedicine for ROP. In addition, several clinical studies have reported favorable long-term results. Many investigators have reinforced the need for detailed protocols on image acquisition and image interpretation. Conclusions: Telemedicine for ROP appears to be a viable alternative to live ophthalmoscopic examinations in many circumstances. Standardization and documentation afforded by telemedicine may provide additional benefits to providers and their patients. With continued improvements in image quality and affordability of imaging systems as well as improved automated image interpretation tools anticipated in the near future, telemedicine for ROP is expected to play an expanding role for a uniquely vulnerable patient population.

Artificial Intelligence and Ophthalmology

Artificial intelligence is advancing rapidly and making its way into all areas of our lives. This review discusses developments and potential practices regarding the use of artificial intelligence in the field of ophthalmology, and the related topic of medical ethics. Various artificial intelligence applications related to the diagnosis of eye diseases were researched in books, journals, search engines, print and social media. Resources were cross-checked to verify the information. Artificial intelligence algorithms, some of which were approved by the US Food and Drug Administration, have been adopted in the field of ophthalmology, especially in diagnostic studies. Studies are being conducted that prove that artificial intelligence algorithms can be used in the field of ophthalmology, especially in diabetic retinopathy, age-related macular degeneration, and retinopathy of prematurity. Some of these algorithms have come to the approval stage. The current point in artificial intelligence studies shows that this technology has advanced considerably and shows promise for future work. It is believed that artificial intelligence applications will be effective in identifying patients with preventable vision loss and directing them to physicians, especially in developing countries where there are fewer trained professionals and physicians are difficult to reach. When we consider the possibility that some future artificial intelligence systems may be candidates for moral/ethical status, certain ethical issues arise. Questions about moral/ethical status are important in some areas of applied ethics. Although it is accepted that current intelligence systems do not have moral/ethical status, it has yet to be determined what the exact the characteristics that confer moral/ethical status are or will be.

Introduction to Machine Learning, Neural Networks, and Deep Learning

Purpose

To present an overview of current machine learning methods and their use in medical research, focusing on select machine learning techniques, best practices, and deep learning.

Methods

A systematic literature search in PubMed was performed for articles pertinent to the topic of artificial intelligence methods used in medicine with an emphasis on ophthalmology.

Results

A review of machine learning and deep learning methodology for the audience without an extensive technical computer programming background.

Conclusions

Artificial intelligence has a promising future in medicine; however, many challenges remain.

Translational Relevance

The aim of this review article is to provide the nontechnical readers a layman's explanation of the machine learning methods being used in medicine today. The goal is to provide the reader a better understanding of the potential and challenges of artificial intelligence within the field of medicine.

Plus Disease in Retinopathy of Prematurity: Convolutional Neural Network Performance Using a Combined Neural Network and Feature Extraction Approach

Purpose

Retinopathy of prematurity (ROP), a leading cause of childhood blindness, is diagnosed by clinical ophthalmoscopic examinations or reading retinal images. Plus disease, defined as abnormal tortuosity and dilation of the posterior retinal blood vessels, is the most important feature to determine treatment-requiring ROP. We aimed to create a complete, publicly available and feature-extraction-based pipeline, I-ROP ASSIST, that achieves convolutional neural network (CNN)-like performance when diagnosing plus disease from retinal images.

Methods

We developed two datasets containing 100 and 5512 posterior retinal images, respectively. After segmenting retinal vessels, we detected the vessel centerlines. Then, we extracted features relevant to ROP, including tortuosity and dilation measures, and used these features in the classifiers including logistic regression, support vector machine and neural networks to assess a severity score for the input. We tested our system with fivefold cross-validation and calculated the area under the curve (AUC) metric for each classifier and dataset.

Results

For predicting plus versus not-plus categories, we achieved 99% and 94% AUC on the first and second datasets, respectively. For predicting pre-plus or worse versus normal categories, we achieved 99% and 88% AUC on the first and second datasets, respectively. The CNN method achieved 98% and 94% for predicting two categories on the second dataset.

Conclusions

Our system combining automatic retinal vessel segmentation, tracing, feature extraction and classification is able to diagnose plus disease in ROP with CNN-like performance.

Translational Relevance

The high performance of I-ROP ASSIST suggests potential applications in automated and objective diagnosis of plus disease.

Artificial Intelligence in Retinopathy of Prematurity Diagnosis

Retinopathy of prematurity (ROP) is a leading cause of childhood blindness worldwide. The diagnosis of ROP is subclassified by zone, stage, and plus disease, with each area demonstrating significant intra- and interexpert subjectivity and disagreement. In addition to improved efficiencies for ROP screening, artificial intelligence may lead to automated, quantifiable, and objective diagnosis in ROP. This review focuses on the development of artificial intelligence for automated diagnosis of plus disease in ROP and highlights the clinical and technical challenges of both the development and implementation of artificial intelligence in the real world.

Aggressive Posterior Retinopathy of Prematurity: Clinical and Quantitative Imaging Features in a Large North American Cohort

PURPOSE

Aggressive posterior retinopathy of prematurity (AP-ROP) is a vision-threatening disease with a significant rate of progression to retinal detachment. The purpose of this study was to characterize AP-ROP quantitatively by demographics, rate of disease progression, and a deep learning-based vascular severity score.

DESIGN

Retrospective analysis.

PARTICIPANTS

The Imaging and Informatics in ROP cohort from 8 North American centers, consisting of 947 patients and 5945 clinical eye examinations with fundus images, was used. Pretreatment eyes were categorized by disease severity: none, mild, type 2 or pre-plus, treatment-requiring (TR) without AP-ROP, TR with AP-ROP. Analyses compared TR with AP-ROP and TR without AP-ROP to investigate differences between AP-ROP and other TR disease.

METHODS

A reference standard diagnosis was generated for each eye examination using previously published methods combining 3 independent image-based gradings and 1 ophthalmoscopic grading. All fundus images were analyzed using a previously published deep learning system and were assigned a score from 1 through 9.

MAIN OUTCOME MEASURES

Birth weight, gestational age, postmenstrual age, and vascular severity score.

RESULTS

Infants who demonstrated AP-ROP were more premature by birth weight (617 g vs. 679 g; P = 0.01) and gestational age (24.3 weeks vs. 25.0 weeks; P < 0.01) and reached peak severity at an earlier postmenstrual age (34.7 weeks vs. 36.9 weeks; P < 0.001) compared with infants with TR without AP-ROP. The mean vascular severity score was greatest in TR with AP-ROP infants compared with TR without AP-ROP infants (8.79 vs. 7.19; P < 0.001). Analyzing the severity score over time, the rate of progression was fastest in infants with AP-ROP (P < 0.002 at 30-32 weeks).

CONCLUSIONS

Premature infants in North America with AP-ROP are born younger and demonstrate disease earlier than infants with less severe ROP. Disease severity is quantifiable with a deep learning-based score, which correlates with clinically identified categories of disease, including AP-ROP. The rate of progression to peak disease is greatest in eyes that demonstrate AP-ROP compared with other treatment-requiring eyes. Analysis of quantitative characteristics of AP-ROP may help improve diagnosis and treatment of an aggressive, vision-threatening form of ROP.

Non-contact smartphone-based fundus imaging compared to conventional fundus imaging: a low-cost alternative for retinopathy of prematurity screening and documentation

Retinopathy of prematurity (ROP) is a frequent cause of treatable childhood blindness. The current dependency of telemedicine-based ROP screening on cost-intensive equipment does not meet the needs in economically disadvantaged regions. Smartphone-based fundus imaging (SBFI) allows for affordable and mobile fundus examination and, therefore, could facilitate cost-effective telemedicine-based ROP screening in low-resources settings. We compared non-contact SBFI and conventional contact fundus imaging (CFI) in terms of feasibility for ROP screening and documentation. Twenty-six eyes were imaged with both SBFI and CFI. Field-of-view was smaller (ratio of diameters, 1:2.5), level of detail was equal, and examination time was longer for SBFI as compared to CFI (109.0 ± 57.8 vs. 75.9 ± 36.3 seconds, p < 0.01). Good agreement with clinical evaluation by indirect funduscopy was achieved for assessment of plus disease and ROP stage for both SBFI (squared Cohen's kappa, 0.88 and 0.81, respectively) and CFI (0.86 and 0.93). Likewise, sensitivity/specificity for detection of plus disease and ROP was high for both SBFI (90%/100% and 88%/93%, respectively) and CFI (80%/100% and 100%/96%). SBFI is a non-contact and low-cost alternative to CFI for ROP screening and documentation that has the potential to considerably improve ROP care in middle- and low-resources settings.

Beyond Performance Metrics: Automatic Deep Learning Retinal OCT Analysis Reproduces Clinical Trial Outcome

PURPOSE

To validate the efficacy of a fully automatic, deep learning-based segmentation algorithm beyond conventional performance metrics by measuring the primary outcome of a clinical trial for macular telangiectasia type 2 (MacTel2).

DESIGN

Evaluation of diagnostic test or technology.

PARTICIPANTS

A total of 92 eyes from 62 participants with MacTel2 from a phase 2 clinical trial (NCT01949324) randomized to 1 of 2 treatment groups METHODS: The ellipsoid zone (EZ) defect areas were measured on spectral domain OCT images of each eye at 2 time points (baseline and month 24) by a fully automatic, deep learning-based segmentation algorithm. The change in EZ defect area from baseline to month 24 was calculated and analyzed according to the clinical trial protocol.

MAIN OUTCOME MEASURE

Difference in the change in EZ defect area from baseline to month 24 between the 2 treatment groups.

RESULTS

The difference in the change in EZ defect area from baseline to month 24 between the 2 treatment groups measured by the fully automatic segmentation algorithm was 0.072±0.035 mm² (P = 0.021). This was comparable to the outcome of the clinical trial using semiautomatic measurements by expert readers, 0.065±0.033 mm² (P = 0.025).

CONCLUSIONS

The fully automatic segmentation algorithm was as accurate as semiautomatic expert segmentation to assess EZ defect areas and was able to reliably reproduce the statistically significant primary outcome measure of the clinical trial. This approach, to validate the performance of an automatic segmentation algorithm on the primary clinical trial end point, provides a robust gauge of its clinical applicability.

Assessment of Deep Generative Models for High-Resolution Synthetic Retinal Image Generation of Age-Related Macular Degeneration

Importance

Deep learning (DL) used for discriminative tasks in ophthalmology, such as diagnosing diabetic retinopathy or age-related macular degeneration (AMD), requires large image data sets graded by human experts to train deep convolutional neural networks (DCNNs). In contrast, generative DL techniques could synthesize large new data sets of artificial retina images with different stages of AMD. Such images could enhance existing data sets of common and rare ophthalmic diseases without concern for personally identifying information to assist medical education of students, residents, and retinal specialists, as well as for training new DL diagnostic models for which extensive data sets from large clinical trials of expertly graded images may not exist.

Objective

To develop DL techniques for synthesizing high-resolution realistic fundus images serving as proxy data sets for use by retinal specialists and DL machines.

Design, Setting, and Participants

Generative adversarial networks were trained on 133 821 color fundus images from 4613 study participants from the Age-Related Eye Disease Study (AREDS), generating synthetic fundus images with and without AMD. We compared retinal specialists' ability to diagnose AMD on both real and synthetic images, asking them to assess image gradability and testing their ability to discern real from synthetic images. The performance of AMD diagnostic DCNNs (referable vs not referable AMD) trained on either all-real vs all-synthetic data sets was compared.

Main Outcomes and Measures

Accuracy of 2 retinal specialists (T.Y.A.L. and K.D.P.) for diagnosing and distinguishing AMD on real vs synthetic images and diagnostic performance (area under the curve) of DL algorithms trained on synthetic vs real images.

Results

The diagnostic accuracy of 2 retinal specialists on real vs synthetic images was similar. The accuracy of diagnosis as referable vs nonreferable AMD compared with certified human graders for retinal specialist 1 was 84.54% (error margin, 4.06%) on real images vs 84.12% (error margin, 4.16%) on synthetic images and for retinal specialist 2 was 89.47% (error margin, 3.45%) on real images vs 89.19% (error margin, 3.54%) on synthetic images. Retinal specialists could not distinguish real from synthetic images, with an accuracy of 59.50% (error margin, 3.93%) for retinal specialist 1 and 53.67% (error margin, 3.99%) for retinal specialist 2. The DCNNs trained on real data showed an area under the curve of 0.9706 (error margin, 0.0029), and those trained on synthetic data showed an area under the curve of 0.9235 (error margin, 0.0045).

Conclusions and Relevance

Deep learning-synthesized images appeared to be realistic to retinal specialists, and DCNNs achieved diagnostic performance on synthetic data close to that for real images, suggesting that DL generative techniques hold promise for training humans and machines.

A deep learning system for identifying lattice degeneration and retinal breaks using ultra-widefield fundus images

Background

Lattice degeneration and/or retinal breaks, defined as notable peripheral retinal lesions (NPRLs), are prone to evolving into rhegmatogenous retinal detachment which can cause severe visual loss. However, screening NPRLs is time-consuming and labor-intensive. Therefore, we aimed to develop and evaluate a deep learning (DL) system for automated identifying NPRLs based on ultra-widefield fundus (UWF) images.

Methods

A total of 5,606 UWF images from 2,566 participants were used to train and verify a DL system. All images were classified by 3 experienced ophthalmologists. The reference standard was determined when an agreement was achieved among all 3 ophthalmologists, or adjudicated by another retinal specialist if disagreements existed. An independent test set of 750 images was applied to verify the performance of 12 DL models trained using 4 different DL algorithms (InceptionResNetV2, InceptionV3, ResNet50, and VGG16) with 3 preprocessing techniques (original, augmented, and histogram-equalized images). Heatmaps were generated to visualize the process of the best DL system in the identification of NPRLs.

Results

In the test set, the best DL system for identifying NPRLs achieved an area under the curve (AUC) of 0.999 with a sensitivity and specificity of 98.7% and 99.2%, respectively. The best preprocessing method in each algorithm was the application of original image augmentation (average AUC =0.996). The best algorithm in each preprocessing method was InceptionResNetV2 (average AUC =0.996). In the test set, 150 of 154 true-positive cases (97.4%) displayed heatmap visualization in the NPRL regions.

Conclusions

A DL system has high accuracy in identifying NPRLs based on UWF images. This system may help to prevent the development of rhegmatogenous retinal detachment by early detection of NPRLs.

Deep Learning Algorithm for Automated Diagnosis of Retinopathy of Prematurity Plus Disease

PURPOSE

This study describes the initial development of a deep learning algorithm, ROP.AI, to automatically diagnose retinopathy of prematurity (ROP) plus disease in fundal images.

METHODS

ROP.AI was trained using 6974 fundal images from Australasian image databases. Each image was given a diagnosis as part of real-world routine ROP screening and classified as normal or plus disease. The algorithm was trained using 80% of the images and validated against the remaining 20% within a hold-out test set. Performance in diagnosing plus disease was evaluated against an external set of 90 images. Performance in detecting pre-plus disease was also tested. As a screening tool, the algorithm's operating point was optimized for sensitivity and negative predictive value, and its performance reevaluated.

RESULTS

For plus disease diagnosis within the 20% hold-out test set, the algorithm achieved a 96.6% sensitivity, 98.0% specificity, and 97.3% ± 0.7% accuracy. Area under the receiver operating characteristic curve was 0.993. Within the independent test set, the algorithm achieved a 93.9% sensitivity, 80.7% specificity, and 95.8% negative predictive value. For detection of pre-plus and plus disease, the algorithm achieved 81.4% sensitivity, 80.7% specificity, and 80.7% negative predictive value. Following the identification of an optimized operating point, the algorithm diagnosed plus disease with a 97.0% sensitivity and 97.8% negative predictive value.

CONCLUSIONS

ROP.AI is a deep learning algorithm able to automatically diagnose ROP plus disease with high sensitivity and negative predictive value.

TRANSLATIONAL RELEVANCE

In the context of increasing global disease burden, future development may improve access to ROP diagnosis and care.

Semantic Segmentation of Eye Fundus Images Using Convolutional Neural Networks

This article reviews the problems of eye bottom fundus analysis and semantic segmentation algorithms used to distinguish the eye vessels and the optical disk. Various diseases, such as glaucoma, hypertension, diabetic retinopathy, macular degeneration, etc., can be diagnosed through changes and anomalies of the vesssels and optical disk. Convolutional neural networks, especially the U-Net architecture, are well-suited for semantic segmentation. A number of U-Net modifications have been recently developed that deliver excellent performance results.

Current status and future trends of clinical diagnoses via image-based deep learning

With the recent developments in deep learning technologies, artificial intelligence (AI) has gradually been transformed from cutting-edge technology into practical applications. AI plays an important role in disease diagnosis and treatment, health management, drug research and development, and precision medicine. Interdisciplinary collaborations will be crucial to develop new AI algorithms for medical applications. In this paper, we review the basic workflow for building an AI model, identify publicly available databases of ocular fundus images, and summarize over 60 papers contributing to the field of AI development.

Classification and comparison via neural networks

We consider learning from comparison labels generated as follows: given two samples in a dataset, a labeler produces a label indicating their relative order. Such comparison labels scale quadratically with the dataset size; most importantly, in practice, they often exhibit lower variance compared to class labels. We propose a new neural network architecture based on siamese networks to incorporate both class and comparison labels in the same training pipeline, using Bradley-Terry and Thurstone loss functions. Our architecture leads to a significant improvement in predicting both class and comparison labels, increasing classification AUC by as much as 35% and comparison AUC by as much as 6% on several real-life datasets. We further show that, by incorporating comparisons, training from few samples becomes possible: a deep neural network of 5.9 million parameters trained on 80 images attains a 0.92 AUC when incorporating comparisons.

A comparison of deep learning performance against health-care professionals in detecting diseases from medical imaging: a systematic review and meta-analysis

BACKGROUND

Deep learning offers considerable promise for medical diagnostics. We aimed to evaluate the diagnostic accuracy of deep learning algorithms versus health-care professionals in classifying diseases using medical imaging.

METHODS

In this systematic review and meta-analysis, we searched Ovid-MEDLINE, Embase, Science Citation Index, and Conference Proceedings Citation Index for studies published from Jan 1, 2012, to June 6, 2019. Studies comparing the diagnostic performance of deep learning models and health-care professionals based on medical imaging, for any disease, were included. We excluded studies that used medical waveform data graphics material or investigated the accuracy of image segmentation rather than disease classification. We extracted binary diagnostic accuracy data and constructed contingency tables to derive the outcomes of interest: sensitivity and specificity. Studies undertaking an out-of-sample external validation were included in a meta-analysis, using a unified hierarchical model. This study is registered with PROSPERO, CRD42018091176.

FINDINGS

Our search identified 31 587 studies, of which 82 (describing 147 patient cohorts) were included. 69 studies provided enough data to construct contingency tables, enabling calculation of test accuracy, with sensitivity ranging from 9·7% to 100·0% (mean 79·1%, SD 0·2) and specificity ranging from 38·9% to 100·0% (mean 88·3%, SD 0·1). An out-of-sample external validation was done in 25 studies, of which 14 made the comparison between deep learning models and health-care professionals in the same sample. Comparison of the performance between health-care professionals in these 14 studies, when restricting the analysis to the contingency table for each study reporting the highest accuracy, found a pooled sensitivity of 87·0% (95% CI 83·0-90·2) for deep learning models and 86·4% (79·9-91·0) for health-care professionals, and a pooled specificity of 92·5% (95% CI 85·1-96·4) for deep learning models and 90·5% (80·6-95·7) for health-care professionals.

INTERPRETATION

Our review found the diagnostic performance of deep learning models to be equivalent to that of health-care professionals. However, a major finding of the review is that few studies presented externally validated results or compared the performance of deep learning models and health-care professionals using the same sample. Additionally, poor reporting is prevalent in deep learning studies, which limits reliable interpretation of the reported diagnostic accuracy. New reporting standards that address specific challenges of deep learning could improve future studies, enabling greater confidence in the results of future evaluations of this promising technology.

FUNDING

None.

Smartphone-based fundus photography for screening of plus-disease retinopathy of prematurity

BACKGROUND

Inadequate screening of treatment-warranted retinopathy of prematurity (ROP) can lead to devastating visual outcomes. Especially in resource-poor communities, the use of an affordable, portable, and easy to use smartphone-based non-contact fundus photography device may prove useful for screening for high-risk ROP. This study evaluates the feasibility of screening for high-risk ROP using a novel smartphone-based fundus photography device, RetinaScope.

METHODS

Retinal images were obtained using RetinaScope on a cohort of prematurely born infants during routine examinations for ROP. Images were reviewed by two masked graders who determined the image quality, the presence or absence of plus disease, and whether there was retinopathy that met predefined criteria for referral. The agreement between image-based assessments was compared to the gold standard indirect ophthalmoscopic assessment.

RESULTS

Fifty-four eyes of 27 infants were included. A wide-field fundus photograph was obtained using RetinaScope. Image quality was acceptable or excellent in 98% and 95% of cases. There was substantial agreement between the gold standard and photographic assessment of presence or absence of plus disease (Cohen's κ = 0.85). Intergrader agreement on the presence of any retinopathy in photographs was also high (κ = 0.92).

CONCLUSIONS

RetinaScope can capture digital retinal photographs of prematurely born infants with good image quality for grading of plus disease.

A Quantitative Severity Scale for Retinopathy of Prematurity Using Deep Learning to Monitor Disease Regression After Treatment

Importance

Retinopathy of prematurity (ROP) is a leading cause of childhood blindness worldwide, but treatment failure and disease recurrence are important causes of adverse outcomes in patients with treatment-requiring ROP (TR-ROP).

Objectives

To apply an automated ROP vascular severity score obtained using a deep learning algorithm and to assess its utility for objectively monitoring ROP regression after treatment.

Design, Setting, and Participants

This retrospective cohort study used data from the Imaging and Informatics in ROP consortium, which comprises 9 tertiary referral centers in North America that screen high volumes of at-risk infants for ROP. Images of 5255 clinical eye examinations from 871 infants performed between July 2011 and December 2016 were assessed for eligibility in the present study. The disease course was assessed with time across the numerous examinations for patients with TR-ROP. Infants born prematurely meeting screening criteria for ROP who developed TR-ROP and who had images captured within 4 weeks before and after treatment as well as at the time of treatment were included.

Main Outcomes and Measures

The primary outcome was mean (SD) ROP vascular severity score before, at time of, and after treatment. A deep learning classifier was used to assign a continuous ROP vascular severity score, which ranged from 1 (normal) to 9 (most severe), at each examination. A secondary outcome was the difference in ROP vascular severity score among eyes treated with laser or the vascular endothelial growth factor antagonist bevacizumab. Differences between groups for both outcomes were assessed using unpaired 2-tailed t tests with Bonferroni correction.

Results

Of 5255 examined eyes, 91 developed TR-ROP, of which 46 eyes met the inclusion criteria based on the available images. The mean (SD) birth weight of those patients was 653 (185) g, with a mean (SD) gestational age of 24.9 (1.3) weeks. The mean (SD) ROP vascular severity scores significantly increased 2 weeks prior to treatment (4.19 [1.75]), peaked at treatment (7.43 [1.89]), and decreased for at least 2 weeks after treatment (4.00 [1.88]) (all P < .001). Eyes requiring retreatment with laser had higher ROP vascular severity scores at the time of initial treatment compared with eyes receiving a single treatment (P < .001).

Conclusions and Relevance

This quantitative ROP vascular severity score appears to consistently reflect clinical disease progression and posttreatment regression in eyes with TR-ROP. These study results may have implications for the monitoring of patients with ROP for treatment failure and disease recurrence and for determining the appropriate level of disease severity for primary treatment in eyes with aggressive disease.

Monitoring Disease Progression With a Quantitative Severity Scale for Retinopathy of Prematurity Using Deep Learning

Importance

Retinopathy of prematurity (ROP) is a leading cause of childhood blindness worldwide, but clinical diagnosis is subjective and qualitative.

Objective

To describe a quantitative ROP severity score derived using a deep learning algorithm designed to evaluate plus disease and to assess its utility for objectively monitoring ROP progression.

Design, Setting, and Participants

This retrospective cohort study included images from 5255 clinical examinations of 871 premature infants who met the ROP screening criteria of the Imaging and Informatics in ROP (i-ROP) Consortium, which comprises 9 tertiary care centers in North America, from July 1, 2011, to December 31, 2016. Data analysis was performed from July 2017 to May 2018.

Exposure

A deep learning algorithm was used to assign a continuous ROP vascular severity score from 1 (most normal) to 9 (most severe) at each examination based on a single posterior photograph compared with a reference standard diagnosis (RSD) simplified into 4 categories: no ROP, mild ROP, type 2 ROP or pre-plus disease, or type 1 ROP. Disease course was assessed longitudinally across multiple examinations for all patients.

Main Outcomes and Measures

Mean ROP vascular severity score progression over time compared with the RSD.

Results

A total of 5255 clinical examinations from 871 infants (mean [SD] gestational age, 27.0 [2.0] weeks; 493 [56.6%] male; mean [SD] birth weight, 949 [271] g) were analyzed. The median severity scores for each category were as follows: 1.1 (interquartile range [IQR], 1.0-1.5) (no ROP), 1.5 (IQR, 1.1-3.4) (mild ROP), 4.6 (IQR, 2.4-5.3) (type 2 and pre-plus), and 7.5 (IQR, 5.0-8.7) (treatment-requiring ROP) (P < .001). When the long-term differences in the median severity scores across time between the eyes progressing to treatment and those who did not eventually require treatment were compared, the median score was higher in the treatment group by 0.06 at 30 to 32 weeks, 0.75 at 32 to 34 weeks, 3.56 at 34 to 36 weeks, 3.71 at 36 to 38 weeks, and 3.24 at 38 to 40 weeks postmenstrual age (P < .001 for all comparisons).

Conclusions and Relevance

The findings suggest that the proposed ROP vascular severity score is associated with category of disease at a given point in time and clinical progression of ROP in premature infants. Automated image analysis may be used to quantify clinical disease progression and identify infants at high risk for eventually developing treatment-requiring ROP. This finding has implications for quality and delivery of ROP care and for future approaches to disease classification.

Artificial Intelligence Screening for Diabetic Retinopathy: the Real-World Emerging Application

PURPOSE OF REVIEW

This paper systematically reviews the recent progress in diabetic retinopathy screening. It provides an integrated overview of the current state of knowledge of emerging techniques using artificial intelligence integration in national screening programs around the world. Existing methodological approaches and research insights are evaluated. An understanding of existing gaps and future directions is created.

RECENT FINDINGS

Over the past decades, artificial intelligence has emerged into the scientific consciousness with breakthroughs that are sparking increasing interest among computer science and medical communities. Specifically, machine learning and deep learning (a subtype of machine learning) applications of artificial intelligence are spreading into areas that previously were thought to be only the purview of humans, and a number of applications in ophthalmology field have been explored. Multiple studies all around the world have demonstrated that such systems can behave on par with clinical experts with robust diagnostic performance in diabetic retinopathy diagnosis. However, only few tools have been evaluated in clinical prospective studies. Given the rapid and impressive progress of artificial intelligence technologies, the implementation of deep learning systems into routinely practiced diabetic retinopathy screening could represent a cost-effective alternative to help reduce the incidence of preventable blindness around the world.

Integrating Artificial and Human Intelligence: A Partnership for Responsible Innovation in Biomedical Engineering and Medicine

Historically, the term "artificial intelligence" dates to 1956 when it was first used in a conference at Dartmouth College in the US. Since then, the development of artificial intelligence has in part been shaped by the field of neuroscience. By understanding the human brain, scientists have attempted to build new intelligent machines capable of performing complex tasks akin to humans. Indeed, future research into artificial intelligence will continue to benefit from the study of the human brain. While the development of artificial intelligence algorithms has been fast paced, the actual use of most artificial intelligence (AI) algorithms in biomedical engineering and clinical practice is still markedly below its conceivably broader potentials. This is partly because for any algorithm to be incorporated into existing workflows it has to stand the test of scientific validation, clinical and personal utility, application context, and is equitable as well. In this context, there is much to be gained by combining AI and human intelligence (HI). Harnessing Big Data, computing power and storage capacities, and addressing societal issues emergent from algorithm applications, demand deploying HI in tandem with AI. Very few countries, even economically developed states, lack adequate and critical governance frames to best understand and steer the AI innovation trajectories in health care. Drug discovery and translational pharmaceutical research stand to gain from AI technology provided they are also informed by HI. In this expert review, we analyze the ways in which AI applications are likely to traverse the continuum of life from birth to death, and encompassing not only humans but also all animal, plant, and other living organisms that are increasingly touched by AI. Examples of AI applications include digital health, diagnosis of diseases in newborns, remote monitoring of health by smart devices, real-time Big Data analytics for prompt diagnosis of heart attacks, and facial analysis software with consequences on civil liberties. While we underscore the need for integration of AI and HI, we note that AI technology does not have to replace medical specialists or scientists and rather, is in need of such expert HI. Altogether, AI and HI offer synergy for responsible innovation and veritable prospects for improving health care from prevention to diagnosis to therapeutics while unintended consequences of automation emergent from AI and algorithms should be borne in mind on scientific cultures, work force, and society at large.

Deep Learning Algorithms for Corneal Amyloid Deposition Quantitation in Familial Amyloidosis

Objectives

The aim of this study was to train and validate deep learning algorithms to quantitate relative amyloid deposition (RAD; mean amyloid deposited area per stromal area) in corneal sections from patients with familial amyloidosis, Finnish (FAF), and assess its relationship with visual acuity.

Methods

Corneal specimens were obtained from 42 patients undergoing penetrating keratoplasty, stained with Congo red, and digitally scanned. Areas of amyloid deposits and areas of stromal tissue were labeled on a pixel level for training and validation. The algorithms were used to quantify RAD in each cornea, and the association of RAD with visual acuity was assessed.

Results

In the validation of the amyloid area classification, sensitivity was 86%, specificity 92%, and F-score 81. For corneal stromal area classification, sensitivity was 74%, specificity 82%, and F-score 73. There was insufficient evidence to demonstrate correlation (Spearman's rank correlation, -0.264, p = 0.091) between RAD and visual acuity (logMAR).

Conclusions

Deep learning algorithms can achieve a high sensitivity and specificity in pixel-level classification of amyloid and corneal stromal area. Further modeling and development of algorithms to assess earlier stages of deposition from clinical images is necessary to better assess the correlation between amyloid deposition and visual acuity. The method might be applied to corneal dystrophies as well.

Machine Learning in the Detection of the Glaucomatous Disc and Visual Field

Glaucoma is the leading cause of irreversible blindness worldwide. Early detection is of utmost importance as there is abundant evidence that early treatment prevents disease progression, preserves vision, and improves patients' long-term quality of life. The structure and function thresholds that alert to the diagnosis of glaucoma can be obtained entirely via digital means, and as such, screening is well suited to benefit from artificial intelligence and specifically machine learning. This paper reviews the concepts and current literature on the use of machine learning for detection of the glaucomatous disc and visual field.

Evaluation of a Remote Telemedicine Screening System for Severe Retinopathy of Prematurity

PURPOSE

To evaluate the validity of remote telemedicine screening for retinopathy of prematurity (ROP) in a population of at-risk preterm infants in Iowa and South Dakota.

METHODS

The medical records for all preterm infants screened for ROP at neonatal intensive care units (NICUs) in Sioux City, Iowa, and Sioux Falls, South Dakota, from September 1, 2017, to July 31, 2018, were retrospectively reviewed. The RetCam Shuttle (Natus Medical Inc., Pleasanton, CA) was used to capture retinal images, which were posted on a secure server for evaluation by a pediatric ophthalmologist. Infants with suspected ROP approaching the criteria for treatment with anti-vascular endothelial growth factor (VEGF) medications were transferred to the Children's Hospital and Medical Center NICU in Omaha, Nebraska, where a comprehensive examination was performed and treatment was administered when indicated. The remaining infants received an outpatient comprehensive examination by one of two pediatric ophthalmologists within 2 weeks of discharge.

RESULTS

A total of 124 telemedicine examinations were performed on 35 infants during the study period. Remote telemedicine screening for referral-warranted ROP using the RetCam Shuttle had a sensitivity of 100%, specificity of 97%, positive predictive value of 66.7%, and negative predictive value of 100%. Of the three infants transferred for referral-warranted ROP, two required treatment with anti-VEGF medications. Good outcomes were noted in all cases, and no patients progressed beyond stage 3 ROP.

CONCLUSIONS

Telemedicine screening reliably detected referral-warranted ROP in at-risk premature infants at two remote sites, with no poor outcomes during the 11-month period. These results demonstrate the validity and utility of remote telemedicine screening for ROP. [J Pediatr Ophthalmol Strabismus. 2019;56(3):157-161.].

Automated Detection and Classification of Telemedical Retinopathy of Prematurity Images

Background: Retinopathy of prematurity (ROP) is a retinal disorder of low birth weight infants and it is the leading cause of childhood blindness. The capability of wide field digital imaging systems to capture the clinical features of ROP has greatly helped the physicians to assess the severity of ROP and prevent childhood blindness due to ROP. Currently there is a lack of automated systems to assess the severity of ROP to assist the ROP specialist to make treatment decision. Objective: To present an automated detection and classification approach to assess the severity of ROP using wide field telemedical images. Materials and Methods: A total of 160 telemedical ROP (tele-ROP) images were collected out, of which 36 images were Normal, 79 images were Stage 2, and 45 images were Stage 3. Hessian analysis and support vector machine (SVM) classifier have been used to detect and classify the severity of ROP from tele-ROP images. Results: Classified the Normal, Stage 2, and Stage 3 images using SVM. Achieved accuracy of 91.8%, sensitivity of 90.37%, specificity of 94.65%, false positive rate of 5.35%, and false negative rate of 9.63%. Conclusions: The automated approach of detecting and classifying ROP would support pediatric ophthalmologists for early treatment decisions with optimal care.

Automated Classification of the Tympanic Membrane Using a Convolutional Neural Network

Precise evaluation of the tympanic membrane (TM) is required for accurate diagnosis of middle ear diseases. However, making an accurate assessment is sometimes difficult. Artificial intelligence is often employed for image processing, especially for performing high level analysis such as image classification, segmentation and matching. In particular, convolutional neural networks (CNNs) are increasingly used in medical image recognition. This study demonstrates the usefulness and reliability of CNNs in recognizing the side and perforation of TMs in medical images. CNN was constructed with typically six layers. After random assignment of the available images to the training, validation and test sets, training was performed. The accuracy of the CNN model was consequently evaluated using a new dataset. A class activation map (CAM) was used to evaluate feature extraction. The CNN model accuracy of detecting the TM side in the test dataset was 97.9%, whereas that of detecting the presence of perforation was 91.0%. The side of the TM and the presence of a perforation affect the activation sites. The results show that CNNs can be a useful tool for classifying TM lesions and identifying TM sides. Further research is required to consider real-time analysis and to improve classification accuracy.

A computer-aided diagnosis system for plus disease in retinopathy of prematurity with structure adaptive segmentation and vessel based features

Retinopathy of Prematurity (ROP) is a blinding disease affecting the retina of low birth-weight preterm infants. Accurate diagnosis of ROP is essential to identify treatment-requiring ROP, which would help to prevent childhood blindness. Plus disease, which characterizes abnormal twisting, widening and branching of the blood vessels, is a significant symptom of treatment requiring ROP. In this paper, we have developed and evaluated a computer-based analysis system for objective assessment of plus disease in ROP, which best mimics the clinical method of disease diagnosis by identifying unique vessel based features. The proposed system consists of an initial segmentation stage, which will efficiently extract blood vessels of varying width and length by utilizing structure adaptive filtering, connectivity analysis and image fusion. The paper proposes the usage of additional retinal features namely leaf node count and vessel density, to portray the abnormal growth and branching of the blood vessels and to complement the commonly used features namely tortuosity and width. The test results show a better classification of plus disease in terms of sensitivity (95%) and specificity (93%), emphasizing the superiority of the proposed segmentation algorithm and vessel-based features. An additional advantage of the proposed system is that the process of selection of relevant vessels for feature extraction is fully automated, which makes the system highly useful to the non-physician graders, owing to the unavailability of a sufficient number of ROP specialists.

DeepSeeNet: A Deep Learning Model for Automated Classification of Patient-based Age-related Macular Degeneration Severity from Color Fundus Photographs

PURPOSE

In assessing the severity of age-related macular degeneration (AMD), the Age-Related Eye Disease Study (AREDS) Simplified Severity Scale predicts the risk of progression to late AMD. However, its manual use requires the time-consuming participation of expert practitioners. Although several automated deep learning systems have been developed for classifying color fundus photographs (CFP) of individual eyes by AREDS severity score, none to date has used a patient-based scoring system that uses images from both eyes to assign a severity score.

DESIGN

DeepSeeNet, a deep learning model, was developed to classify patients automatically by the AREDS Simplified Severity Scale (score 0-5) using bilateral CFP.

PARTICIPANTS

DeepSeeNet was trained on 58 402 and tested on 900 images from the longitudinal follow-up of 4549 participants from AREDS. Gold standard labels were obtained using reading center grades.

METHODS

DeepSeeNet simulates the human grading process by first detecting individual AMD risk factors (drusen size, pigmentary abnormalities) for each eye and then calculating a patient-based AMD severity score using the AREDS Simplified Severity Scale.

MAIN OUTCOME MEASURES

Overall accuracy, specificity, sensitivity, Cohen's kappa, and area under the curve (AUC). The performance of DeepSeeNet was compared with that of retinal specialists.

RESULTS

DeepSeeNet performed better on patient-based classification (accuracy = 0.671; kappa = 0.558) than retinal specialists (accuracy = 0.599; kappa = 0.467) with high AUC in the detection of large drusen (0.94), pigmentary abnormalities (0.93), and late AMD (0.97). DeepSeeNet also outperformed retinal specialists in the detection of large drusen (accuracy 0.742 vs. 0.696; kappa 0.601 vs. 0.517) and pigmentary abnormalities (accuracy 0.890 vs. 0.813; kappa 0.723 vs. 0.535) but showed lower performance in the detection of late AMD (accuracy 0.967 vs. 0.973; kappa 0.663 vs. 0.754).

CONCLUSIONS

By simulating the human grading process, DeepSeeNet demonstrated high accuracy with increased transparency in the automated assignment of individual patients to AMD risk categories based on the AREDS Simplified Severity Scale. These results highlight the potential of deep learning to assist and enhance clinical decision-making in patients with AMD, such as early AMD detection and risk prediction for developing late AMD. DeepSeeNet is publicly available on https://github.com/ncbi-nlp/DeepSeeNet.

Improving Health Care with Advanced Analytics: Practical Considerations

Artificial intelligence (AI) is becoming ubiquitous in health care, largely through machine learning and predictive analytics applications. Recent applications of AI to common health care scenarios, such as screening and diagnosing, have fueled optimism about the use of advanced analytics to improve care. Careful and objective considerations need to be made before implementing an advanced analytics solution. Critical evaluation before, during, and after its implementation will ensure safe care, good outcomes, and the elimination of waste. In this commentary we offer basic practical considerations for developing, implementing, and evaluating such solutions based on many years of experience.

Imaging in Retinopathy of Prematurity

Retinopathy of prematurity (ROP) is a leading cause of preventable childhood blindness worldwide. Barriers to ROP screening and difficulties with subsequent evaluation and management include poor access to care, lack of physicians trained in ROP, and issues with objective documentation. Digital retinal imaging can help address these barriers and improve our knowledge of the pathophysiology of the disease. Advancements in technology have led to new, non-mydriatic and mydriatic cameras with wider fields of view as well as devices that can simultaneously incorporate fluorescein angiography, optical coherence tomography (OCT), and OCT angiography. Image analysis in ROP is also being employed through smartphones and computer-based software. Telemedicine programs in the United States and worldwide have utilized imaging to extend ROP screening to infants in remote areas and have shown that digital retinal imaging can be reliable, accurate, and cost-effective. In addition, tele-education programs are also using digital retinal images to increase the number of healthcare providers trained in ROP. Although indirect ophthalmoscopy is still an important skill for screening, digital retinal imaging holds promise for more widespread screening and management of ROP.

Artificial intelligence and deep learning in ophthalmology

Artificial intelligence (AI) based on deep learning (DL) has sparked tremendous global interest in recent years. DL has been widely adopted in image recognition, speech recognition and natural language processing, but is only beginning to impact on healthcare. In ophthalmology, DL has been applied to fundus photographs, optical coherence tomography and visual fields, achieving robust classification performance in the detection of diabetic retinopathy and retinopathy of prematurity, the glaucoma-like disc, macular oedema and age-related macular degeneration. DL in ocular imaging may be used in conjunction with telemedicine as a possible solution to screen, diagnose and monitor major eye diseases for patients in primary care and community settings. Nonetheless, there are also potential challenges with DL application in ophthalmology, including clinical and technical challenges, explainability of the algorithm results, medicolegal issues, and physician and patient acceptance of the AI 'black-box' algorithms. DL could potentially revolutionise how ophthalmology is practised in the future. This review provides a summary of the state-of-the-art DL systems described for ophthalmic applications, potential challenges in clinical deployment and the path forward.