Plus Disease in Retinopathy of Prematurity: A Continuous Spectrum of Vascular Abnormality as a Basis of Diagnostic Variability

Computer-aided detection of retinopathy of prematurity severity assessment via vessel tortuosity measurement in preterm infants' fundus images

OBJECTIVE

To develop a computer-aided diagnostic system for retinopathy of prematurity (ROP) disease using retinal vessel morphological features.

METHODS

A total of 200 fundus images from 136 preterm infants with stage 1 to 3 ROP were analysed. Two methods were developed to measure vessel tortuosity: the peak-and-valley method and the polynomial curve fitting method. Correlations between temporal artery tortuosity (TAT) and temporal vein tortuosity (TVT) with ROP severity were investigated, and vessel tortuosity relationships with vessel angles (TAA and TVA) and vessel widths (TAW and TVW). A separate dataset from Japan containing 126 images from 97 preterm patients was used for verification.

RESULTS

Both methods identified similar tortuosity in images without ROP and mild ROP cases. However, the polynomial curve fit method demonstrated enhanced tortuosity detection in stages 2 and 3 ROP compared to the peak and valley method. A strong positive correlation was revealed between ROP severity and increased arterial and venous tortuosity (P < 0.0001). A significant negative correlation between TAA and TAT (r = -0.485, P < 0.0001) and TVA and TVT (r = -0.281, P < 0.0001), and a significant positive correlation between TAW and TAT (r = 0.204, P value = 0.0040) were identified. Similar results were found in the test dataset from Japan.

CONCLUSIONS

ROP severity was associated with increased retinal tortuosity and retinal vessel width while displaying a decrease in retinal vascular angle. This quantitative analysis of retinal vessels provides crucial insights for advancing ROP diagnosis and understanding its progression.

Use of an Artificial Intelligence-Generated Vascular Severity Score Improved Plus Disease Diagnosis in Retinopathy of Prematurity

PURPOSE

To evaluate whether providing clinicians with an artificial intelligence (AI)-based vascular severity score (VSS) improves consistency in the diagnosis of plus disease in retinopathy of prematurity (ROP).

DESIGN

Multireader diagnostic accuracy imaging study.

PARTICIPANTS

Eleven ROP experts, 9 of whom had been in practice for 10 years or more.

METHODS

RetCam (Natus Medical Incorporated) fundus images were obtained from premature infants during routine ROP screening as part of the Imaging and Informatics in ROP study between January 2012 and July 2020. From all available examinations, a subset of 150 eye examinations from 110 infants were selected for grading. An AI-based VSS was assigned to each set of images using the i-ROP DL system (Siloam Vision). The clinicians were asked to diagnose plus disease for each examination and to assign an estimated VSS (range, 1-9) at baseline, and then again 1 month later with AI-based VSS assistance. A reference standard diagnosis (RSD) was assigned to each eye examination from the Imaging and Informatics in ROP study based on 3 masked expert labels and the ophthalmoscopic diagnosis.

MAIN OUTCOME MEASURES

Mean linearly weighted κ value for plus disease diagnosis compared with RSD. Area under the receiver operating characteristic curve (AUC) and area under the precision-recall curve (AUPR) for labels 1 through 9 compared with RSD for plus disease.

RESULTS

Expert agreement improved significantly, from substantial (κ value, 0.69 [0.59, 0.75]) to near perfect (κ value, 0.81 [0.71, 0.86]), when AI-based VSS was integrated. Additionally, a significant improvement in plus disease discrimination was achieved as measured by mean AUC (from 0.94 [95% confidence interval (CI), 0.92-0.96] to 0.98 [95% CI, 0.96-0.99]; difference, 0.04 [95% CI, 0.01-0.06]) and AUPR (from 0.86 [95% CI, 0.81-0.90] to 0.95 [95% CI, 0.91-0.97]; difference, 0.09 [95% CI, 0.03-0.14]).

CONCLUSIONS

Providing ROP clinicians with an AI-based measurement of vascular severity in ROP was associated with both improved plus disease diagnosis and improved continuous severity labeling as compared with an RSD for plus disease. If implemented in practice, AI-based VSS could reduce interobserver variability and could standardize treatment for infants with ROP.

FINANCIAL DISCLOSURE(S)

Proprietary or commercial disclosure may be found in the Footnotes and Disclosures at the end of this article.

P Score: A Reference Image-Based Clinical Grading Scale for Vascular Change in Retinopathy of Prematurity

PURPOSE

The International Classification of Retinopathy of Prematurity, Third Edition (ICROP3), acknowledged that plus-like retinopathy of prematurity (ROP) vascular changes occurs along a spectrum. Historically, clinician-experts demonstrate variable agreement for plus diagnosis. We developed a 9-photograph reference image set for grading plus-like changes and compared intergrader agreement of the set with standard grading with no plus, preplus, and plus disease.

DESIGN

Retinal photographic grading and expert consensus opinion.

PARTICIPANTS

The development set included 34 international ICROP3 committee members. The validation set included 30 ophthalmologists with ROP expertise (15 ICROP3 committee members and 15 non-ICROP3 members) METHODS: Nine ROP fundus images (P1 through P9) representing increasing degrees of zone I vascular tortuosity and dilation, based on the 34 ICROP3 committee members' gradings and consensus image reviews, were used to establish standard photographs for the plus (P) score. Study participants graded 150 fundus photographs 2 ways, separated by a 1-week washout period: (1) no plus, preplus, or plus disease and (2) choosing the closest P score image.

MAIN OUTCOME MEASURES

Intergrader agreement measured by intraclass correlation coefficient.

RESULTS

Intergrader agreement was higher using the P score (intraclass correlation coefficient, 0.75; 95% confidence interval, 0.71-0.79) than no plus, preplus, or plus disease (intraclass correlation coefficient, 0.67; 95% confidence interval, 0.62-0.72). Mean ± standard deviation P scores for images with mode gradings of no plus, preplus, and plus disease were 2.5 ± 0.7, 4.8 ± 0.8, and 7.4 ± 0.8, respectively.

CONCLUSIONS

Intergrader agreement of plus-like vascular change in ROP using the P score is high. We now incorporate this 9-image reference set into ICROP3 for use in clinician daily practice alongside zone, stage, and plus assessment. P score is not yet meant to replace plus diagnosis for treatment decisions, but its use at our institutions has permitted better comparison between examinations for progression and regression, communication between examiners, and documentation of vascular change without fundus imaging. P score also could provide more detailed ROP classification for clinical trials, consistent with the spectrum of plus-like change that is now formally part of the International Classification of Retinopathy of Prematurity.

FINANCIAL DISCLOSURE(S)

Proprietary or commercial disclosure may be found in the Footnotes and Disclosures at the end of this article.

Implementation of optical coherence tomography in retinopathy of prematurity screening

PURPOSE OF REVIEW

In this review, we explore the investigational applications of optical coherence tomography (OCT) in retinopathy of prematurity (ROP), the insights they have delivered thus far, and key milestones for its integration into the standard of care.

RECENT FINDINGS

While OCT has been widely integrated into clinical management of common retinal diseases, its use in pediatric contexts has been undermined by limitations in ergonomics, image acquisition time, and field of view. Recently, investigational handheld OCT devices have been reported with advancements including ultra-widefield view, noncontact use, and high-speed image capture permitting real-time en face visualization. These developments are compelling for OCT as a more objective alternative with reduced neonatal stress compared to indirect ophthalmoscopy and/or fundus photography as a means of classifying and monitoring ROP.

SUMMARY

OCT may become a viable modality in management of ROP. Ongoing innovation surrounding handheld devices should aim to optimize patient comfort and image resolution in the retinal periphery. Future clinical investigations may seek to objectively characterize features of peripheral stage and explore novel biomarkers of disease activity.

Ocular blood flow in preterm neonates

Retinopathy of prematurity (ROP) is a disorder affecting low birthweight, preterm neonates. In the preterm eye, the retina is not fully developed and neovascularization may occur at the margin between the developed vascular retina and undeveloped avascular retina. Without timely treatment by laser or intravitreal anti-vascular endothelial growth factor (VEGF) therapy, this can lead to tractional retinal detachment and blindness. Visualization of the retina in regular examinations by indirect ophthalmoscopy is hence the current standard of care, but the exams are stressful and interpretation of images is subjective. The upregulation of VEGF in ROP would suggest an increase in ocular blood flow. In this report, we evaluate the potential of ultrafast plane-wave Doppler ultrasound (PWU) to detect increased flow velocities in the orbital vessels supplying the eye in a gentle exam with objective findings. We imaged both eyes of 50 low-birthweight preterm neonates using 18 MHz PWU. Flow velocity in the central retinal artery (CRA) and vein (CRV), and the short posterior ciliary arteries were determined and values at each ROP Stage compared. We found significantly increased velocities in the CRA and CRV in Stage 3 ROP eyes, where intervention would be considered. We compared multivariate models for identifying Stage 3 eyes comprised solely of clinical factors, solely of Doppler parameters, and clinical plus Doppler parameters. The respective models provided areas under their respective ROC curves of 0.760, 0.812, and 0.904. PWU Doppler represents a gentle, objective means for identifying neonates at risk for ROP that could complement ophthalmoscopy.

Automated diagnosis of plus disease in retinopathy of prematurity using quantification of vessels characteristics

The condition known as Plus disease is distinguished by atypical alterations in the retinal vasculature of neonates born prematurely. It has been demonstrated that the diagnosis of Plus disease is subjective and qualitative in nature. The utilization of quantitative methods and computer-based image analysis to enhance the objectivity of Plus disease diagnosis has been extensively established in the literature. This study presents the development of a computer-based image analysis method aimed at automatically distinguishing Plus images from non-Plus images. The proposed methodology conducts a quantitative analysis of the vascular characteristics linked to Plus disease, thereby aiding physicians in making informed judgments. A collection of 76 posterior retinal images from a diverse group of infants who underwent screening for Retinopathy of Prematurity (ROP) was obtained. A reference standard diagnosis was established as the majority of the labeling performed by three experts in ROP during two separate sessions. The process of segmenting retinal vessels was carried out using a semi-automatic methodology. Computer algorithms were developed to compute the tortuosity, dilation, and density of vessels in various retinal regions as potential discriminative characteristics. A classifier was provided with a set of selected features in order to distinguish between Plus images and non-Plus images. This study included 76 infants (49 [64.5%] boys) with mean birth weight of 1305 ± 427 g and mean gestational age of 29.3 ± 3 weeks. The average level of agreement among experts for the diagnosis of plus disease was found to be 79% with a standard deviation of 5.3%. In terms of intra-expert agreement, the average was 85% with a standard deviation of 3%. Furthermore, the average tortuosity of the five most tortuous vessels was significantly higher in Plus images compared to non-Plus images (p ≤ 0.0001). The curvature values based on points were found to be significantly higher in Plus images compared to non-Plus images (p ≤ 0.0001). The maximum diameter of vessels within a region extending 5-disc diameters away from the border of the optic disc (referred to as 5DD) exhibited a statistically significant increase in Plus images compared to non-Plus images (p ≤ 0.0001). The density of vessels in Plus images was found to be significantly higher compared to non-Plus images (p ≤ 0.0001). The classifier's accuracy in distinguishing between Plus and non-Plus images, as determined through tenfold cross-validation, was found to be 0.86 ± 0.01. This accuracy was observed to be higher than the diagnostic accuracy of one out of three experts when compared to the reference standard. The implemented algorithm in the current study demonstrated a commendable level of accuracy in detecting Plus disease in cases of retinopathy of prematurity, exhibiting comparable performance to that of expert diagnoses. By engaging in an objective analysis of the characteristics of vessels, there exists the possibility of conducting a quantitative assessment of the disease progression's features. The utilization of this automated system has the potential to enhance physicians' ability to diagnose Plus disease, thereby offering valuable contributions to the management of ROP through the integration of traditional ophthalmoscopy and image-based telemedicine methodologies.

Expert-centered Evaluation of Deep Learning Algorithms for Brain Tumor Segmentation

Purpose To present results from a literature survey on practices in deep learning segmentation algorithm evaluation and perform a study on expert quality perception of brain tumor segmentation. Materials and Methods A total of 180 articles reporting on brain tumor segmentation algorithms were surveyed for the reported quality evaluation. Additionally, ratings of segmentation quality on a four-point scale were collected from medical professionals for 60 brain tumor segmentation cases. Results Of the surveyed articles, Dice score, sensitivity, and Hausdorff distance were the most popular metrics to report segmentation performance. Notably, only 2.8% of the articles included clinical experts' evaluation of segmentation quality. The experimental results revealed a low interrater agreement (Krippendorff α, 0.34) in experts' segmentation quality perception. Furthermore, the correlations between the ratings and commonly used quantitative quality metrics were low (Kendall tau between Dice score and mean rating, 0.23; Kendall tau between Hausdorff distance and mean rating, 0.51), with large variability among the experts. Conclusion The results demonstrate that quality ratings are prone to variability due to the ambiguity of tumor boundaries and individual perceptual differences, and existing metrics do not capture the clinical perception of segmentation quality. Keywords: Brain Tumor Segmentation, Deep Learning Algorithms, Glioblastoma, Cancer, Machine Learning Clinical trial registration nos. NCT00756106 and NCT00662506 Supplemental material is available for this article. © RSNA, 2023.

Retinal blood flow velocities in infants with retinopathy of prematurity after intravitreal administration of bevacizumab

BACKGROUND/OBJECTIVES

Progression of retinopathy of prematurity (ROP) is associated with increased retinal blood flow velocities. We investigated changes of central retinal arterial and venous blood flow after intravitreal administration of bevacizumab.

SUBJECTS/METHODS

Prospective observational study using serial ultrasound Doppler imaging in preterm infants with bevacizumab-treated ROP. Eyes were examined 1 [0-2] days before injection (median [interquartile range]), and at three time points after injection (1 [1-2] days, 6 [3-8] days, and 17 [9-28] days). Preterm infants with ROP stage 2 displaying spontaneous regression served as controls.

RESULTS

In 21 eyes of 12 infants with bevacizumab-treated ROP, peak arterial systolic velocity declined from 13.6 [11.0-16.3] cm/s prior to intravitreal bevacizumab to 11.2 [9.4-13.9] cm/s, 10.6 [9.2-13.3] cm/s and 9.3 [8.2-11.0] cm/s at discharge (p  =  .002). There was also a decline of the arterial velocity time integral (from 3.1 [2.3-3.9] cm to 2.9 [2.4-3.5], 2.7 [2.3-3.2] cm and 2.2 [2.0-2.7], p  =  .021) and mean velocity in the central retinal vein (from 4.5 [3.6-5.8] cm/s to 3.7 [2.6-4.1] cm/s, 3.5 [3.0-4.3] cm/s, and 3.2 [2.8-4.6] cm/s, p  =  .012). Arterial end-diastolic velocity and resistance index remained unchanged. Blood flow velocities in bevacizumab-treated eyes examined before injection were significantly higher than those measured in untreated eyes that ultimately showed spontaneous regression of ROP. Sequential examinations in these controls did not reveal any declines of retinal blood flow velocities.

CONCLUSION

Increased retinal arterial and venous blood flow velocities in infants with threshold ROP decline following intravitreal bevacizumab injection.

Limited Hyperoxia-Induced Proliferative Retinopathy (LHIPR) as a Model of Retinal Fibrosis, Angiogenesis, and Inflammation

The progression to fibrosis and traction in retinopathy of prematurity (ROP) and other ischemic retinopathies remains an important clinical and surgical challenge, necessitating a comprehensive understanding of its pathogenesis. Fibrosis is an unbalanced deposition of extracellular matrix components responsible for scar tissue formation with consequent tissue and organ impairment. Together with retinal traction, it is among the main causes of retinal detachment and vision loss. We capitalize on the Limited Hyperoxia Induced Retinopathy (LHIPR) model, as it reflects the more advanced pathological phenotypes seen in ROP and other ischemic retinopathies. To model LHIPR, we exposed wild-type C57Bl/6J mouse pups to 65% oxygen from P0 to P7. Then, the pups were returned to room air to recover until later endpoints. We performed histological and molecular analysis to evaluate fibrosis progression, angiogenesis, and inflammation at several time points, from 1.5 months to 9 months. In addition, we performed in vivo retinal imaging by optical coherence tomography (OCT) or OCT Angiography (OCTA) to follow the fibrovascular progression in vivo. Although the retinal morphology was relatively preserved, we found a progressive increase in preretinal fibrogenesis over time, up to 9 months of age. We also detected blood vessels in the preretinal space as well as an active inflammatory process, altogether mimicking advanced preretinal fibrovascular disease in humans.

Development and international validation of custom-engineered and code-free deep-learning models for detection of plus disease in retinopathy of prematurity: a retrospective study

BACKGROUND

Retinopathy of prematurity (ROP), a leading cause of childhood blindness, is diagnosed through interval screening by paediatric ophthalmologists. However, improved survival of premature neonates coupled with a scarcity of available experts has raised concerns about the sustainability of this approach. We aimed to develop bespoke and code-free deep learning-based classifiers for plus disease, a hallmark of ROP, in an ethnically diverse population in London, UK, and externally validate them in ethnically, geographically, and socioeconomically diverse populations in four countries and three continents. Code-free deep learning is not reliant on the availability of expertly trained data scientists, thus being of particular potential benefit for low resource health-care settings.

METHODS

This retrospective cohort study used retinal images from 1370 neonates admitted to a neonatal unit at Homerton University Hospital NHS Foundation Trust, London, UK, between 2008 and 2018. Images were acquired using a Retcam Version 2 device (Natus Medical, Pleasanton, CA, USA) on all babies who were either born at less than 32 weeks gestational age or had a birthweight of less than 1501 g. Each images was graded by two junior ophthalmologists with disagreements adjudicated by a senior paediatric ophthalmologist. Bespoke and code-free deep learning models (CFDL) were developed for the discrimination of healthy, pre-plus disease, and plus disease. Performance was assessed internally on 200 images with the majority vote of three senior paediatric ophthalmologists as the reference standard. External validation was on 338 retinal images from four separate datasets from the USA, Brazil, and Egypt with images derived from Retcam and the 3nethra neo device (Forus Health, Bengaluru, India).

FINDINGS

Of the 7414 retinal images in the original dataset, 6141 images were used in the final development dataset. For the discrimination of healthy versus pre-plus or plus disease, the bespoke model had an area under the curve (AUC) of 0·986 (95% CI 0·973-0·996) and the CFDL model had an AUC of 0·989 (0·979-0·997) on the internal test set. Both models generalised well to external validation test sets acquired using the Retcam for discriminating healthy from pre-plus or plus disease (bespoke range was 0·975-1·000 and CFDL range was 0·969-0·995). The CFDL model was inferior to the bespoke model on discriminating pre-plus disease from healthy or plus disease in the USA dataset (CFDL 0·808 [95% CI 0·671-0·909, bespoke 0·942 [0·892-0·982]], p=0·0070). Performance also reduced when tested on the 3nethra neo imaging device (CFDL 0·865 [0·742-0·965] and bespoke 0·891 [0·783-0·977]).

INTERPRETATION

Both bespoke and CFDL models conferred similar performance to senior paediatric ophthalmologists for discriminating healthy retinal images from ones with features of pre-plus or plus disease; however, CFDL models might generalise less well when considering minority classes. Care should be taken when testing on data acquired using alternative imaging devices from that used for the development dataset. Our study justifies further validation of plus disease classifiers in ROP screening and supports a potential role for code-free approaches to help prevent blindness in vulnerable neonates.

FUNDING

National Institute for Health Research Biomedical Research Centre based at Moorfields Eye Hospital NHS Foundation Trust and the University College London Institute of Ophthalmology.

TRANSLATIONS

For the Portuguese and Arabic translations of the abstract see Supplementary Materials section.

Variability in Plus Disease Diagnosis using Single and Serial Images

PURPOSE

To assess changes in retinopathy of prematurity (ROP) diagnosis in single and serial retinal images.

DESIGN

Cohort study.

PARTICIPANTS

Cases of ROP recruited from the Imaging and Informatics in Retinopathy of Prematurity (i-ROP) consortium evaluated by 7 graders.

METHODS

Seven ophthalmologists reviewed both single and 3 consecutive serial retinal images from 15 cases with ROP, and severity was assigned as plus, preplus, or none. Imaging data were acquired during routine ROP screening from 2011 to 2015, and a reference standard diagnosis was established for each image. A secondary analysis was performed using the i-ROP deep learning system to assign a vascular severity score (VSS) to each image, ranging from 1 to 9, with 9 being the most severe disease. This score has been previously demonstrated to correlate with the International Classification of ROP. Mean plus disease severity was calculated by averaging 14 labels per image in serial and single images to decrease noise.

MAIN OUTCOME MEASURES

Grading severity of ROP as defined by plus, preplus, or no ROP.

RESULTS

Assessment of serial retinal images changed the grading severity for > 50% of the graders, although there was wide variability. Cohen's kappa ranged from 0.29 to 1.0, which showed a wide range of agreement from slight to perfect by each grader. Changes in the grading of serial retinal images were noted more commonly in cases of preplus disease. The mean severity in cases with a diagnosis of plus disease and no disease did not change between single and serial images. The ROP VSS demonstrated good correlation with the range of expert classifications of plus disease and overall agreement with the mode class (P = 0.001). The VSS correlated with mean plus disease severity by expert diagnosis (correlation coefficient, 0.89). The more aggressive graders tended to be influenced by serial images to increase the severity of their grading. The VSS also demonstrated agreement with disease progression across serial images, which progressed to preplus and plus disease.

CONCLUSIONS

Clinicians demonstrated variability in ROP diagnosis when presented with both single and serial images. The use of deep learning as a quantitative assessment of plus disease has the potential to standardize ROP diagnosis and treatment.

Improving the repeatability of deep learning models with Monte Carlo dropout

The integration of artificial intelligence into clinical workflows requires reliable and robust models. Repeatability is a key attribute of model robustness. Ideal repeatable models output predictions without variation during independent tests carried out under similar conditions. However, slight variations, though not ideal, may be unavoidable and acceptable in practice. During model development and evaluation, much attention is given to classification performance while model repeatability is rarely assessed, leading to the development of models that are unusable in clinical practice. In this work, we evaluate the repeatability of four model types (binary classification, multi-class classification, ordinal classification, and regression) on images that were acquired from the same patient during the same visit. We study the each model's performance on four medical image classification tasks from public and private datasets: knee osteoarthritis, cervical cancer screening, breast density estimation, and retinopathy of prematurity. Repeatability is measured and compared on ResNet and DenseNet architectures. Moreover, we assess the impact of sampling Monte Carlo dropout predictions at test time on classification performance and repeatability. Leveraging Monte Carlo predictions significantly increases repeatability, in particular at the class boundaries, for all tasks on the binary, multi-class, and ordinal models leading to an average reduction of the 95% limits of agreement by 16% points and of the class disagreement rate by 7% points. The classification accuracy improves in most settings along with the repeatability. Our results suggest that beyond about 20 Monte Carlo iterations, there is no further gain in repeatability. In addition to the higher test-retest agreement, Monte Carlo predictions are better calibrated which leads to output probabilities reflecting more accurately the true likelihood of being correctly classified.

Association of Optical Coherence Tomography-Measured Fibrovascular Ridge Thickness and Clinical Disease Stage in Retinopathy of Prematurity

Importance

Accurate diagnosis of retinopathy of prematurity (ROP) is essential to provide timely treatment and reduce the risk of blindness. However, the components of an ROP examination are subjective and qualitative.

Objective

To evaluate whether optical coherence tomography (OCT)-derived retinal thickness measurements at the vascular-avascular junction are associated with clinical diagnosis of ROP stage.

Design, Setting, and Participants

This cross-sectional longitudinal study compared OCT-based ridge thickness calculated from OCT B-scans by a masked examiner to the clinical diagnosis of 2 masked examiners using both traditional stage classifications and a more granular continuous scale at the neonatal intensive care unit (NICU) of Oregon Health & Science University (OHSU) Hospital. Infants who met ROP screening criteria in the OHSU NICU between June 2021 and April 2022 and had guardian consent were included. One OCT volume and en face image per patient per eye showing at least 1 to 2 clock hours of ridge were included in the final analysis.

Main Outcomes and Measures

Comparison of OCT-derived ridge thickness to the clinical diagnosis of ROP stage using an ordinal and continuous scale. Repeatability was assessed using 20 repeated examinations from the same visit and compared using intraclass correlation coefficient (ICC) and coefficient of variation (CV). Comparison of ridge thickness with ordinal categories was performed using generalized estimating equations and with continuous stage using Spearman correlation.

Results

A total of 128 separate OCT eye examinations from 50 eyes of 25 patients were analyzed. The ICC was 0.87 with a CV of 7.0%. Higher ordinal disease classification was associated with higher axial ridge thickness on OCT, with mean (SD) thickness measurements of 264.2 (11.2) μm (P < .001), 334.2 (11.4) μm (P < .001), and 495.0 (32.2) μm (P < .001) for stages 1, 2, and 3, respectively and with continuous stage labels (ρ = 0.739, P < .001).

Conclusions and Relevance

These results suggest that OCT-based quantification of peripheral stage in ROP may be an objective and quantitative biomarker that may be useful for clinical diagnosis and longitudinal monitoring and may have implications for disease classification in the future.

ROP screening with the Pictor Plus camera: a telemedicine solution for developing countries

PURPOSE

To assess the Pictor Plus portable noncontact ophthalmic camera for use in diagnosis of severe retinopathy of prematurity (ROP) and its feasibility for telemedicine applications, especially in developing countries.

METHODS

A cross-sectional study was conducted from April 2018 to December 2019 in six public neonatal care centers in the state of Rio de Janeiro. Observer 1 performed the examination with a binocular indirect ophthalmoscope and subsequently captured a digital image using the Pictor Plus camera. These images were analyzed by observers 2 and 3, who were masked to the results recorded by observer 1.

RESULTS

The convenience sample was 200 premature newborns (n = 712 eyes) with a birthweight of ≤1,500 g or gestational age at birth of ≤32 weeks. The sensitivity of Pictor Plus digital images for the diagnosis of plus or pre-plus disease was 65.8% for observer 2 and 63.2% for observer 3; the specificity, 100% and 98,4%, respectively. Interobserver agreement analysis for image reading compared to observer 1 indirect ophthalmoscopy showed kappa coefficients of 0.77 (observer 2) and 0.68 (observer 3).

CONCLUSIONS

In our study cohort, the Pictor Plus, a relatively low-cost camera, was effective in facilitating accurate diagnosis of ROP.

Artificial Intelligence for Retinopathy of Prematurity: Validation of a Vascular Severity Scale against International Expert Diagnosis

PURPOSE

To validate a vascular severity score as an appropriate output for artificial intelligence (AI) Software as a Medical Device (SaMD) for retinopathy of prematurity (ROP) through comparison with ordinal disease severity labels for stage and plus disease assigned by the International Classification of Retinopathy of Prematurity, Third Edition (ICROP3), committee.

DESIGN

Validation study of an AI-based ROP vascular severity score.

PARTICIPANTS

A total of 34 ROP experts from the ICROP3 committee.

METHODS

Two separate datasets of 30 fundus photographs each for stage (0-5) and plus disease (plus, preplus, neither) were labeled by members of the ICROP3 committee using an open-source platform. Averaging these results produced a continuous label for plus (1-9) and stage (1-3) for each image. Experts were also asked to compare each image to each other in terms of relative severity for plus disease. Each image was also labeled with a vascular severity score from the Imaging and Informatics in ROP deep learning system, which was compared with each grader's diagnostic labels for correlation, as well as the ophthalmoscopic diagnosis of stage.

MAIN OUTCOME MEASURES

Weighted kappa and Pearson correlation coefficients (CCs) were calculated between each pair of grader classification labels for stage and plus disease. The Elo algorithm was also used to convert pairwise comparisons for each expert into an ordered set of images from least to most severe.

RESULTS

The mean weighted kappa and CC for all interobserver pairs for plus disease image comparison were 0.67 and 0.88, respectively. The vascular severity score was found to be highly correlated with both the average plus disease classification (CC = 0.90, P < 0.001) and the ophthalmoscopic diagnosis of stage (P < 0.001 by analysis of variance) among all experts.

CONCLUSIONS

The ROP vascular severity score correlates well with the International Classification of Retinopathy of Prematurity committee member's labels for plus disease and stage, which had significant intergrader variability. Generation of a consensus for a validated scoring system for ROP SaMD can facilitate global innovation and regulatory authorization of these technologies.

Improved Training Efficiency for Retinopathy of Prematurity Deep Learning Models Using Comparison versus Class Labels

Purpose

To compare the efficacy and efficiency of training neural networks for medical image classification using comparison labels indicating relative disease severity versus diagnostic class labels from a retinopathy of prematurity (ROP) image dataset.

Design

Evaluation of diagnostic test or technology.

Participants

Deep learning neural networks trained on expert-labeled wide-angle retinal images obtained from patients undergoing diagnostic ROP examinations obtained as part of the Imaging and Informatics in ROP (i-ROP) cohort study.

Methods

Neural networks were trained with either class or comparison labels indicating plus disease severity in ROP retinal fundus images from 2 datasets. After training and validation, all networks underwent evaluation using a separate test dataset in 1 of 2 binary classification tasks: normal versus abnormal or plus versus nonplus.

Main Outcome Measures

Area under the receiver operating characteristic curve (AUC) values were measured to assess network performance.

Results

Given the same number of labels, neural networks learned more efficiently by comparison, generating significantly higher AUCs in both classification tasks across both datasets. Similarly, given the same number of images, comparison learning developed networks with significantly higher AUCs across both classification tasks in 1 of 2 datasets. The difference in efficiency and accuracy between models trained on either label type decreased as the size of the training set increased.

Conclusions

Comparison labels individually are more informative and more abundant per sample than class labels. These findings indicate a potential means of overcoming the common obstacle of data variability and scarcity when training neural networks for medical image classification tasks.

Integrated Visualization Highlighting Retinal Changes in Retinopathy of Prematurity From 3-Dimensional Optical Coherence Tomography Data

Importance

Early diagnosis of plus disease is critical in the management of retinopathy of prematurity (ROP). However, there is substantial interexpert disagreement in the diagnosis of plus disease based on vascular changes alone. Information derived from optical coherence tomography (OCT) may help characterize the severity of vascular and structural abnormalities in ROP.

Objective

To describe integrated visualization of 3-dimensional (3-D) data from investigational swept-source OCT optimized to delineate retinal vascular and microanatomical features in eyes with and without ROP.

Design, Setting, and Participants

This cross-sectional, observational report of OCT was captured in the prospective Study of Eye Imaging in Preterm Infants (BabySTEPS) designed in July 2016 at the Duke Health Intensive Care Nursery. Between December 2018 and August 2019, 2 preterm infants born at 24 and 30 weeks' gestation were enrolled, underwent ROP screening, and were imaged at those screening visits. Data at 36 weeks' postmenstrual age were analyzed via this visualization developed between September 2020 and May 2021.

Main Outcomes and Measures

Superimposed en face retinal vascular shadow view (RVSV) montages and thickness maps were used along with OCT B-scans to evaluate retinal vasculature and cross-section in eyes with and without ROP.

Results

In the right eyes of 2 infants, 3-D data were integrated and visualized from investigational bedside OCT imaging at the posterior pole. In the infant who developed type 1 ROP, RVSV-OCT confirmed presence of dilated and tortuous posterior pole vessels, shunting, and incomplete perifoveal vascular development, resulting in a temporal notch of avascular retina in zone 1. The thickness map revealed irregular pockets of thickening and thinning, and integrated visualization outlined the demarcation between thicker vascularized retina and thinner avascular fovea and presence of extraretinal neovascularization overlying elevated vessels in the superior arcades. In the infant without ROP (stage 0), RVSV-OCT revealed no abnormal vascular findings at the posterior pole. The integrated visualization showed a dome-shaped retinal thickening at the fovea, which was confirmed as macular edema.

Conclusions and Relevance

In 2 preterm infants in BabySTEPS, 3-D visualization of OCT findings during the ongoing ROP disease process demonstrated supplemental information about the retinal vasculature and microanatomy that can be useful to clinicians. These additional details provided by OCT could be integrated into future ROP screening methods with artificial intelligence-based analytics.

Limited hyperoxia-induced proliferative retinopathy: A model of persistent retinal vascular dysfunction, preretinal fibrosis and hyaloidal vascular reprogramming for retinal rescue

Background

Retinopathy of prematurity (ROP) remains the leading cause for blindness in children. Limited hyperoxia induced proliferative retinopathy (L-HIPR) was recently introduced as a potential animal model for ROP and persistent fetal vasculature; however, the detailed pathological changes remain unclear.

Methods

To model L-HIPR, we placed C57BL/6J mice in 65% oxygen from birth to post-natal day 7 (P7). We examined eyes at intervals between P12 and P30. Retinal morphometry, thickness, and preretinal fibrosis were quantified at different time points on histological sections stained with hematoxylin and eosin (H&E) and Masson Trichrome, respectively. Vascular development, angiogenesis, inflammation, and pericyte coverage were analyzed using immunohistochemistry staining in retinal flat mounts and cross sections.

Results

In L-HIPR, the hyaloidal vessels persisted until the latest time point in this study, P30 and began to invaginate the peripheral then central retina starting at P12. Central retinal distortion was noted beginning at P17, while the peripheral retina demonstrated a trend of thinning from P12 to P30. We found that L-HIPR was associated with delayed and abnormal retinal vascular development with subsequent retinal inflammation, pericyte loss and preretinal fibrosis.

Conclusion

Our study presents a detailed analysis of the L-HIPR animal model demonstrating vitreoretinal pathologic changes, preretinal fibrosis and persistent hyaloidal vessels into adulthood. Based on our findings, we suggest that the persistence and peculiar stepwise migration of the hyaloidal vessels into the retina may provide a potential rescue mechanism for inner retinal development that deserves further study.

Newborn Eye Screening as an Application of AI

Artificial intelligence (AI) applications are diverse and serve varied functions in clinical practice. The most successful products today are clinical decision tools used by physicians, but autonomous AI is gaining traction. Widespread use of AI is limited in part because of concerns about bias, fault-tolerance, and specificity. Adoption of AI often depends on removing cost and complexity in clinical workflow integration, providing clear incentives for use, and providing clear demonstration of clinical outcome. Existing wide-angle photographic screening could be integrated into the clinical workflow based on prior implementations for premature babies and linked with AI interpretation with existing technology. Incidence of retinal abnormality, clinical considerations, AI performance, grading variation for AI-augmented human grading, and cost and policy aspects play a significant role. Improved outcomes for newborns and a relatively high estimated incidence of abnormality have been named as benefits to counterweigh costs in the long term. [Ophthalmic Surg Lasers Imaging Retina. 2021;52:S17-S22.].

International Classification of Retinopathy of Prematurity, Third Edition

PURPOSE

The International Classification of Retinopathy of Prematurity is a consensus statement that creates a standard nomenclature for classification of retinopathy of prematurity (ROP). It was initially published in 1984, expanded in 1987, and revisited in 2005. This article presents a third revision, the International Classification of Retinopathy of Prematurity, Third Edition (ICROP3), which is now required because of challenges such as: (1) concerns about subjectivity in critical elements of disease classification; (2) innovations in ophthalmic imaging; (3) novel pharmacologic therapies (e.g., anti-vascular endothelial growth factor agents) with unique regression and reactivation features after treatment compared with ablative therapies; and (4) recognition that patterns of ROP in some regions of the world do not fit neatly into the current classification system.

DESIGN

Review of evidence-based literature, along with expert consensus opinion.

PARTICIPANTS

International ROP expert committee assembled in March 2019 representing 17 countries and comprising 14 pediatric ophthalmologists and 20 retinal specialists, as well as 12 women and 22 men.

METHODS

The committee was initially divided into 3 subcommittees-acute phase, regression or reactivation, and imaging-each of which used iterative videoconferences and an online message board to identify key challenges and approaches. Subsequently, the entire committee used iterative videoconferences, 2 in-person multiday meetings, and an online message board to develop consensus on classification.

MAIN OUTCOME MEASURES

Consensus statement.

RESULTS

The ICROP3 retains current definitions such as zone (location of disease), stage (appearance of disease at the avascular-vascular junction), and circumferential extent of disease. Major updates in the ICROP3 include refined classification metrics (e.g., posterior zone II, notch, subcategorization of stage 5, and recognition that a continuous spectrum of vascular abnormality exists from normal to plus disease). Updates also include the definition of aggressive ROP to replace aggressive-posterior ROP because of increasing recognition that aggressive disease may occur in larger preterm infants and beyond the posterior retina, particularly in regions of the world with limited resources. ROP regression and reactivation are described in detail, with additional description of long-term sequelae.

CONCLUSIONS

These principles may improve the quality and standardization of ROP care worldwide and may provide a foundation to improve research and clinical care.

Evaluation of a Deep Learning-Derived Quantitative Retinopathy of Prematurity Severity Scale

PURPOSE

To evaluate the clinical usefulness of a quantitative deep learning-derived vascular severity score for retinopathy of prematurity (ROP) by assessing its correlation with clinical ROP diagnosis and by measuring clinician agreement in applying a novel scale.

DESIGN

Analysis of existing database of posterior pole fundus images and corresponding ophthalmoscopic examinations using 2 methods of assigning a quantitative scale to vascular severity.

PARTICIPANTS

Images were from clinical examinations of patients in the Imaging and Informatics in ROP Consortium. Four ophthalmologists and 1 study coordinator evaluated vascular severity on a scale from 1 to 9.

METHODS

A quantitative vascular severity score (1-9) was applied to each image using a deep learning algorithm. A database of 499 images was developed for assessment of interobserver agreement.

MAIN OUTCOME MEASURES

Distribution of deep learning-derived vascular severity scores with the clinical assessment of zone (I, II, or III), stage (0, 1, 2, or 3), and extent (<3 clock hours, 3-6 clock hours, and >6 clock hours) of stage 3 evaluated using multivariate linear regression and weighted κ values and Pearson correlation coefficients for interobserver agreement on a 1-to-9 vascular severity scale.

RESULTS

For deep learning analysis, a total of 6344 clinical examinations were analyzed. A higher deep learning-derived vascular severity score was associated with more posterior disease, higher disease stage, and higher extent of stage 3 disease (P < 0.001 for all). For a given ROP stage, the vascular severity score was higher in zone I than zones II or III (P < 0.001). Multivariate regression found zone, stage, and extent all were associated independently with the severity score (P < 0.001 for all). For interobserver agreement, the mean ± standard deviation weighted κ value was 0.67 ± 0.06, and the Pearson correlation coefficient ± standard deviation was 0.88 ± 0.04 on the use of a 1-to-9 vascular severity scale.

CONCLUSIONS

A vascular severity scale for ROP seems feasible for clinical adoption; corresponds with zone, stage, extent of stage 3, and plus disease; and facilitates the use of objective technology such as deep learning to improve the consistency of ROP diagnosis.

Current Application of Digital Diagnosing Systems for Retinopathy of Prematurity

BACKGROUND AND OBJECTIVE

Retinopathy of prematurity (ROP), a proliferative vascular eye disease, is one of the leading causes of blindness in childhood and prevails in premature infants with low-birth-weight. The recent progress in digital image analysis offers novel strategies for ROP diagnosis. This paper provides a comprehensive review on the development of digital diagnosing systems for ROP to software researchers. It may also be adopted as a guide to ophthalmologists for selecting the most suitable diagnostic software in the clinical setting, particularly for the remote ophthalmic support.

METHODS

We review the latest literatures concerning the application of digital diagnosing systems for ROP. The diagnosing systems are analyzed and categorized. Articles published between 1998 and 2020 were screened with the two searching engines Pubmed and Google Scholar.

RESULTS

Telemedicine is a method of remote image interpretation that can provide medical service to remote regions, and yet requires training to local operators. On the basis of image collection in telemedicine, computer-based image analytical systems for ROP were later developed. So far, the aforementioned systems have been mainly developed by virtue of classic machine learning, deep learning (DL) and multiple machine learning. During the past two decades, various computer-aided systems for ROP based on classic machine learning (e.g. RISA, ROPtool, CAIER) became available and have achieved satisfactory performance. Further, automated systems for ROP diagnosis based on DL are developed for clinical applications and exhibit high accuracy. Moreover, multiple instance learning is another method to establish an automated system for ROP detection besides DL, which, however, warrants further investigation in future.

CONCLUSION

At present, the incorporation of computer-based image analysis with telemedicine potentially enables the detection, supervision and in-time treatment of ROP for the preterm babies.

Management of retinopathy of prematurity (ROP) in a Polish cohort of infants

Retinopathy of prematurity (ROP) is potentially blinding, but screening and timely treatment can stop its progression. The data on treatment outcomes of ROP from Central and Eastern Europe are scarce. Therefore, we aimed to analyze the latest results of ROP management in Poznan medical center to update the data from this world region. In the years 2016–2019, 178 patients (350 eyes) received treatment for ROP (6.1% of the screened population). The mean gestational age was 26 weeks (range 22–31 weeks), the mean birth weight was 868 g (range 410–1890 g). The most frequent ROP stage at treatment was zone II, stage 3 + (34.9%). As the first line of treatment, 115 infants (226 eyes, 64.6%) underwent laser photocoagulation (LP); 61 infants (120 eyes, 34.3%) received intravitreal ranibizumab injections (IVR); and 2 infants (4 eyes, 0.6%) were treated simultaneously with LP and IVR. One hundred twenty-six eyes (36%) of 63 patients required retreatment: 20.4% treated with LP and 66.7% treated with IVR. Retinal detachment occurred in 14 eyes (4%). The incidence of ROP, ROP requiring treatment, and reoccurrence rates are higher in the Polish population than in Western Europe and the USA. The identified treatment patterns find increasing use of anti-VEGF agents.

Variability in Plus Disease Identified Using a Deep Learning-Based Retinopathy of Prematurity Severity Scale

PURPOSE

Retinopathy of prematurity is a leading cause of childhood blindness worldwide, but clinical diagnosis is subjective, which leads to treatment differences. Our goal was to determine objective differences in the diagnosis of plus disease between clinicians using an automated retinopathy of prematurity (ROP) vascular severity score.

DESIGN

This retrospective cohort study used data from the Imaging and Informatics in ROP Consortium, which comprises 8 tertiary care centers in North America. Fundus photographs of all infants undergoing ROP screening examinations between July 1, 2011, and December 31, 2016, were obtained.

PARTICIPANTS

Infants meeting ROP screening criteria who were diagnosed with plus disease and treatment initiated by an examining physician based on ophthalmoscopic examination results.

METHODS

An ROP severity score (1-9) was generated for each image using a deep learning (DL) algorithm.

MAIN OUTCOME MEASURES

The mean, median, and range of ROP vascular severity scores overall and for each examiner when the diagnosis of plus disease was made.

RESULTS

A total of 5255 clinical examinations in 871 babies were analyzed. Of these, 168 eyes were diagnosed with plus disease by 11 different examiners and were included in the study. The mean ± standard deviation vascular severity score for patients diagnosed with plus disease was 7.4 ± 1.9, median was 8.5 (interquartile range, 5.8-8.9), and range was 1.1 to 9.0. Within some examiners, variability in the level of vascular severity diagnosed as plus disease was present, and 1 examiner routinely diagnosed plus disease in patients with less severe disease than the other examiners (P < 0.01).

CONCLUSIONS

We observed variability both between and within examiners in the diagnosis of plus disease using DL. Prospective evaluation of clinical trial data using an objective measurement of vascular severity may help to define better the minimum necessary level of vascular severity for the diagnosis of plus disease or how other clinical features such as zone, stage, and extent of peripheral disease ought to be incorporated in treatment decisions.

Auto-Processed Retinal Vessel Shadow View Images From Bedside Optical Coherence Tomography to Evaluate Plus Disease in Retinopathy of Prematurity

Purpose

To describe the creation of en face retinal vessel shadow view (RVSV) optical coherence tomography (OCT) images and assess the feasibility of using these for evaluating vascular disease in preterm infants at risk for retinopathy of prematurity (ROP).

Methods

In this exploratory study, we selected images from eyes with a range of ROP vascular disease, prospectively acquired from preterm infants using an investigational, noncontact, handheld, bedside swept-source OCT. We autosegmented OCT volumes using custom infant-specific software, extracted RVSV-OCT images from volumetric data bracketed around the retinal pigment epithelium, and automontaged the resulting RVSV-OCT images. Three masked ophthalmologists graded the RVSV-OCT montages as plus, pre-plus, or neither and ranked them by relative vascular disease severity.

Results

We selected images from 17 imaging sessions (7 plus, 4 pre-plus, 6 neither on clinical examination). On review, 15/17 (88%) RVSV-OCT montages were gradable for plus, pre-plus, or neither and all 17 montages were rankable for relative severity. Intergrader agreement for plus, pre-plus, or neither grading was good (κ, 0.67; 95% confidence interval, 0.42–0.86) and for relative severity ranking was excellent (intraclass correlation coefficient, 0.98; 95% confidence interval, 0.96–0.99).

Conclusions

Our novel automatic processing method can create RVSV-OCT montages optimized for retinal vessel visualization for ROP screening. Although our data support the feasibility of using RVSV-OCT montages for ranking relative vascular disease severity, there is room for improved OCT image capture and processing methods in preterm infants screened for ROP.

Translational Relevance

Creation and grading of RVSV-OCT images could eventually be integrated into an alternative method for ROP screening.

Convolutional Neural Network Based on Fluorescein Angiography Images for Retinopathy of Prematurity Management

Purpose

The purpose of this study was to explore the use of fluorescein angiography (FA) images in a convolutional neural network (CNN) in the management of retinopathy of prematurity (ROP).

Methods

The dataset involved a total of 835 FA images of 149 eyes (90 patients), where each eye was associated with a binary outcome (57 “untreated” eyes and 92 “treated”; 308 “untreated” images, 527 “treated”). The resolution of the images was 1600 and 1200 px in 20% of cases, whereas the remaining 80% had a resolution of 640 and 480 px. All the images were resized to 640 and 480 px before training and no other preprocessing was applied. A CNN with four convolutional layers was trained on 90% of the images (n = 752) randomly chosen. The accuracy of the prediction was assessed on the remaining 10% of images (n = 83). Keras version 2.2.0 for R with Tensorflow backend version 1.11.0 was used for the analysis.

Results

The validation accuracy after 100 epochs was 0.88, whereas training accuracy was 0.97. The receiver operating characteristic (ROC) presented an area under the curve (AUC) of 0.91.

Conclusions

Our study showed, we believe for the first time, the applicability of artificial intelligence (CNN) technology in the ROP management driven by FA. Further studies are needed to exploit different fields of applications of this technology.

Translational Relevance

This algorithm is the basis for a system that could be applied to both ROP as well as experimental oxygen induced retinopathy.

Prediction of retinopathy of prematurity using the screening algorithm WINROP in a Saudi cohort of preterm infants

OBJECTIVES

To validate the web weight gain-based WINROP (weight, insulin-like growth factor I, neonatal, retinopathy of prematurity [ROP]) algorithm retrospectively to identify type 1 ROP in a Saudi cohort of premature infants.  Methods: The records of preterm infants (greater than 23 and less than 32 weeks gestation) born between August 2013 and October 2018, were reviewed. Birth weight, gestational age, and weekly weight measurements of the premature infants were entered online. Based on weekly weight gain, the WINROP algorithm alerted clinicians whether infants were at high-risk for vision‑threatening type 1 ROP. Sensitivity, specificity, positive and negative predictive values were calculated. Results: The median gestational age of the infants at birth was 28 weeks, with median birth weight at 1085 g. Of the 175 infants included in the study, 13 (7.4%) developed type 1 ROP. WINROP positive alarm was triggered in 70.9% (124/175) of all infants and 100% (13/13) of those treated for type 1 ROP. The specificity of the algorithm was 31.5%. Positive predictive values was 10.5% and negative was 100%. Conclusion: The general WINROP sensitivity in identifying type 1 ROP was 100% similar to that reported in developed countries; however, its specificity was low at 31.5%. Tweaking of the algorithm based on the population may increase the specificity and promote the practical utility of this non-invasive screening tool for ophthalmologists and neonatologists in this population.

Practice Guidelines for Ocular Telehealth-Diabetic Retinopathy, Third Edition

Contributors The following document and appendices represent the third edition of the Practice Guidelines for Ocular Telehealth-Diabetic Retinopathy. These guidelines were developed by the Diabetic Retinopathy Telehealth Practice Guidelines Working Group. This working group consisted of a large number of subject matter experts in clinical applications for telehealth in ophthalmology. The editorial committee consisted of Mark B. Horton, OD, MD, who served as working group chair and Christopher J. Brady, MD, MHS, and Jerry Cavallerano, OD, PhD, who served as cochairs. The writing committees were separated into seven different categories. They are as follows: 1.Clinical/operational: Jerry Cavallerano, OD, PhD (Chair), Gail Barker, PhD, MBA, Christopher J. Brady, MD, MHS, Yao Liu, MD, MS, Siddarth Rathi, MD, MBA, Veeral Sheth, MD, MBA, Paolo Silva, MD, and Ingrid Zimmer-Galler, MD. 2.Equipment: Veeral Sheth, MD (Chair), Mark B. Horton, OD, MD, Siddarth Rathi, MD, MBA, Paolo Silva, MD, and Kristen Stebbins, MSPH. 3.Quality assurance: Mark B. Horton, OD, MD (Chair), Seema Garg, MD, PhD, Yao Liu, MD, MS, and Ingrid Zimmer-Galler, MD. 4.Glaucoma: Yao Liu, MD, MS (Chair) and Siddarth Rathi, MD, MBA. 5.Retinopathy of prematurity: Christopher J. Brady, MD, MHS (Chair) and Ingrid Zimmer-Galler, MD. 6.Age-related macular degeneration: Christopher J. Brady, MD, MHS (Chair) and Ingrid Zimmer-Galler, MD. 7.Autonomous and computer assisted detection, classification and diagnosis of diabetic retinopathy: Michael Abramoff, MD, PhD (Chair), Michael F. Chiang, MD, and Paolo Silva, MD.

Siamese neural networks for continuous disease severity evaluation and change detection in medical imaging

Using medical images to evaluate disease severity and change over time is a routine and important task in clinical decision making. Grading systems are often used, but are unreliable as domain experts disagree on disease severity category thresholds. These discrete categories also do not reflect the underlying continuous spectrum of disease severity. To address these issues, we developed a convolutional Siamese neural network approach to evaluate disease severity at single time points and change between longitudinal patient visits on a continuous spectrum. We demonstrate this in two medical imaging domains: retinopathy of prematurity (ROP) in retinal photographs and osteoarthritis in knee radiographs. Our patient cohorts consist of 4861 images from 870 patients in the Imaging and Informatics in Retinopathy of Prematurity (i-ROP) cohort study and 10,012 images from 3021 patients in the Multicenter Osteoarthritis Study (MOST), both of which feature longitudinal imaging data. Multiple expert clinician raters ranked 100 retinal images and 100 knee radiographs from excluded test sets for severity of ROP and osteoarthritis, respectively. The Siamese neural network output for each image in comparison to a pool of normal reference images correlates with disease severity rank (ρ = 0.87 for ROP and ρ = 0.89 for osteoarthritis), both within and between the clinical grading categories. Thus, this output can represent the continuous spectrum of disease severity at any single time point. The difference in these outputs can be used to show change over time. Alternatively, paired images from the same patient at two time points can be directly compared using the Siamese neural network, resulting in an additional continuous measure of change between images. Importantly, our approach does not require manual localization of the pathology of interest and requires only a binary label for training (same versus different). The location of disease and site of change detected by the algorithm can be visualized using an occlusion sensitivity map-based approach. For a longitudinal binary change detection task, our Siamese neural networks achieve test set receiving operator characteristic area under the curves (AUCs) of up to 0.90 in evaluating ROP or knee osteoarthritis change, depending on the change detection strategy. The overall performance on this binary task is similar compared to a conventional convolutional deep-neural network trained for multi-class classification. Our results demonstrate that convolutional Siamese neural networks can be a powerful tool for evaluating the continuous spectrum of disease severity and change in medical imaging.

Telemedicine for Retinopathy of Prematurity

Background: Retinopathy of prematurity (ROP) is a disease of the retinal vasculature that remains a leading cause of childhood blindness worldwide despite improvements in the systemic care of premature newborns. Screening for ROP is effective and cost-effective, but in many areas, access to skilled examiners to conduct dilated examinations is poor. Remote screening with retinal photography is an alternative strategy that may allow for improved ROP care. Methods: The current literature was reviewed to find clinical trials and expert consensus documents on the state-of-the-art of telemedicine for ROP. Results: Several studies have confirmed the utility of telemedicine for ROP. In addition, several clinical studies have reported favorable long-term results. Many investigators have reinforced the need for detailed protocols on image acquisition and image interpretation. Conclusions: Telemedicine for ROP appears to be a viable alternative to live ophthalmoscopic examinations in many circumstances. Standardization and documentation afforded by telemedicine may provide additional benefits to providers and their patients. With continued improvements in image quality and affordability of imaging systems as well as improved automated image interpretation tools anticipated in the near future, telemedicine for ROP is expected to play an expanding role for a uniquely vulnerable patient population.

Plus Disease in Retinopathy of Prematurity: Convolutional Neural Network Performance Using a Combined Neural Network and Feature Extraction Approach

Purpose

Retinopathy of prematurity (ROP), a leading cause of childhood blindness, is diagnosed by clinical ophthalmoscopic examinations or reading retinal images. Plus disease, defined as abnormal tortuosity and dilation of the posterior retinal blood vessels, is the most important feature to determine treatment-requiring ROP. We aimed to create a complete, publicly available and feature-extraction-based pipeline, I-ROP ASSIST, that achieves convolutional neural network (CNN)-like performance when diagnosing plus disease from retinal images.

Methods

We developed two datasets containing 100 and 5512 posterior retinal images, respectively. After segmenting retinal vessels, we detected the vessel centerlines. Then, we extracted features relevant to ROP, including tortuosity and dilation measures, and used these features in the classifiers including logistic regression, support vector machine and neural networks to assess a severity score for the input. We tested our system with fivefold cross-validation and calculated the area under the curve (AUC) metric for each classifier and dataset.

Results

For predicting plus versus not-plus categories, we achieved 99% and 94% AUC on the first and second datasets, respectively. For predicting pre-plus or worse versus normal categories, we achieved 99% and 88% AUC on the first and second datasets, respectively. The CNN method achieved 98% and 94% for predicting two categories on the second dataset.

Conclusions

Our system combining automatic retinal vessel segmentation, tracing, feature extraction and classification is able to diagnose plus disease in ROP with CNN-like performance.

Translational Relevance

The high performance of I-ROP ASSIST suggests potential applications in automated and objective diagnosis of plus disease.

Aggressive Posterior Retinopathy of Prematurity: Clinical and Quantitative Imaging Features in a Large North American Cohort

PURPOSE

Aggressive posterior retinopathy of prematurity (AP-ROP) is a vision-threatening disease with a significant rate of progression to retinal detachment. The purpose of this study was to characterize AP-ROP quantitatively by demographics, rate of disease progression, and a deep learning-based vascular severity score.

DESIGN

Retrospective analysis.

PARTICIPANTS

The Imaging and Informatics in ROP cohort from 8 North American centers, consisting of 947 patients and 5945 clinical eye examinations with fundus images, was used. Pretreatment eyes were categorized by disease severity: none, mild, type 2 or pre-plus, treatment-requiring (TR) without AP-ROP, TR with AP-ROP. Analyses compared TR with AP-ROP and TR without AP-ROP to investigate differences between AP-ROP and other TR disease.

METHODS

A reference standard diagnosis was generated for each eye examination using previously published methods combining 3 independent image-based gradings and 1 ophthalmoscopic grading. All fundus images were analyzed using a previously published deep learning system and were assigned a score from 1 through 9.

MAIN OUTCOME MEASURES

Birth weight, gestational age, postmenstrual age, and vascular severity score.

RESULTS

Infants who demonstrated AP-ROP were more premature by birth weight (617 g vs. 679 g; P = 0.01) and gestational age (24.3 weeks vs. 25.0 weeks; P < 0.01) and reached peak severity at an earlier postmenstrual age (34.7 weeks vs. 36.9 weeks; P < 0.001) compared with infants with TR without AP-ROP. The mean vascular severity score was greatest in TR with AP-ROP infants compared with TR without AP-ROP infants (8.79 vs. 7.19; P < 0.001). Analyzing the severity score over time, the rate of progression was fastest in infants with AP-ROP (P < 0.002 at 30-32 weeks).

CONCLUSIONS

Premature infants in North America with AP-ROP are born younger and demonstrate disease earlier than infants with less severe ROP. Disease severity is quantifiable with a deep learning-based score, which correlates with clinically identified categories of disease, including AP-ROP. The rate of progression to peak disease is greatest in eyes that demonstrate AP-ROP compared with other treatment-requiring eyes. Analysis of quantitative characteristics of AP-ROP may help improve diagnosis and treatment of an aggressive, vision-threatening form of ROP.

Development of a Retinopathy of Prematurity Activity Scale and Clinical Outcome Measures for Use in Clinical Trials

Importance

To facilitate drug and device development for neonates, the International Neonatal Consortium brings together key stakeholders, including pharmaceutical companies, practitioners, regulators, funding agencies, scientists, and families, to address the need for objective, standardized clinical trial outcome measurements to fulfill regulatory requirements. Retinopathy of prematurity (ROP) is a disease that affects preterm neonates. The current International Classification of Retinopathy of Prematurity does not take into account all of the characteristics of ROP and does not adequately discriminate small changes in disease after treatment. These factors are critical for evaluating outcomes in clinical trials.

Observations

There is need for an updated ROP acute disease activity and structure scale as well as end-stage structure and ophthalmologic outcome measures designed for use at different ages. The scale and measures, based on current diagnostic methods and treatments, could be used as a guideline for clinical intervention trials. The scale is intended to be validated against retrospective data and revised for use in future trials. An iterative revision process can be accomplished if new measures are added to clinical trials and evaluated at the end of each trial for prognostic value. The new measures would then be incorporated into a new version of the activity scale and the outcome measures revised.

Conclusions and Relevance

An ROP activity scale and outcome measures to obtain the most robust and discriminatory data for clinical trials are needed. The scales should be dynamic and modified as knowledge and imaging modalities improve and then validated using data from well-documented clinical trials. This approach is relevant to improving clinical trial data quality.

Deep Learning Algorithm for Automated Diagnosis of Retinopathy of Prematurity Plus Disease

PURPOSE

This study describes the initial development of a deep learning algorithm, ROP.AI, to automatically diagnose retinopathy of prematurity (ROP) plus disease in fundal images.

METHODS

ROP.AI was trained using 6974 fundal images from Australasian image databases. Each image was given a diagnosis as part of real-world routine ROP screening and classified as normal or plus disease. The algorithm was trained using 80% of the images and validated against the remaining 20% within a hold-out test set. Performance in diagnosing plus disease was evaluated against an external set of 90 images. Performance in detecting pre-plus disease was also tested. As a screening tool, the algorithm's operating point was optimized for sensitivity and negative predictive value, and its performance reevaluated.

RESULTS

For plus disease diagnosis within the 20% hold-out test set, the algorithm achieved a 96.6% sensitivity, 98.0% specificity, and 97.3% ± 0.7% accuracy. Area under the receiver operating characteristic curve was 0.993. Within the independent test set, the algorithm achieved a 93.9% sensitivity, 80.7% specificity, and 95.8% negative predictive value. For detection of pre-plus and plus disease, the algorithm achieved 81.4% sensitivity, 80.7% specificity, and 80.7% negative predictive value. Following the identification of an optimized operating point, the algorithm diagnosed plus disease with a 97.0% sensitivity and 97.8% negative predictive value.

CONCLUSIONS

ROP.AI is a deep learning algorithm able to automatically diagnose ROP plus disease with high sensitivity and negative predictive value.

TRANSLATIONAL RELEVANCE

In the context of increasing global disease burden, future development may improve access to ROP diagnosis and care.

Accuracy and Reliability of Eye-Based vs Quadrant-Based Diagnosis of Plus Disease in Retinopathy of Prematurity

Importance

Presence of plus disease in retinopathy of prematurity is the most critical element in identifying treatment-requiring disease. However, there is significant variability in plus disease diagnosis. In particular, plus disease has been defined as 2 or more quadrants of vascular abnormality, and it is not clear whether it is more reliably and accurately diagnosed by eye-based assessment of overall retinal appearance or by quadrant-based assessment combining grades of 4 individual quadrants.

Objective

To compare eye-based vs quadrant-based diagnosis of plus disease and to provide insight for ophthalmologists about the diagnostic process.

Design, Setting, and Participants

In this multicenter cohort study, we developed a database of 197 wide-angle retinal images from 141 preterm infants from neonatal intensive care units at 9 academic institutions (enrolled from July 2011 to December 2016). Each image was assigned a reference standard diagnosis based on consensus image-based and clinical diagnosis. Data analysis was performed from February 2017 to September 2017.

Interventions

Six graders independently diagnosed each of the 4 quadrants (cropped images) of the 197 eyes (quadrant-based diagnosis) as well as the entire image (eye-based diagnosis). Images were displayed individually, in random order. Quadrant-based diagnosis of plus disease was made when 2 or more quadrants were diagnosed as indicating plus disease by combining grades of individual quadrants post hoc.

Main Outcomes and Measures

Intragrader and intergrader reliability (absolute agreement and κ statistic) and accuracy compared with the reference standard diagnosis.

Results

Of the 141 included preterm infants, 65 (46.1%) were female and 116 (82.3%) white, and the mean (SD) gestational age was 27.0 (2.6) weeks. There was variable agreement between eye-based and quadrant-based diagnosis among the 6 graders (Cohen κ range, 0.32-0.75). Four graders showed underdiagnosis of plus disease with quadrant-based diagnosis compared with eye-based diagnosis (by McNemar test). Intergrader agreement of quadrant-based diagnosis was lower than that of eye-based diagnosis (Fleiss κ, 0.75 [95% CI, 0.71-0.78] vs 0.55 [95% CI, 0.51-0.59]). The accuracy of eye-based diagnosis compared with the reference standard diagnosis was substantial to near-perfect, whereas that of quadrant-based plus disease diagnosis was only moderate to substantial for each grader.

Conclusions and Relevance

Graders had lower reliability and accuracy using quadrant-based diagnosis combining grades of individual quadrants than with eye-based diagnosis, suggesting that eye-based diagnosis has advantages over quadrant-based diagnosis. This has implications for more precise definitions of plus disease regarding the criterion of 2 or more quadrants, clinical care, computer-based image analysis, and education for all ophthalmologists who manage retinopathy of prematurity.

Diagnostic Accuracy of Ophthalmoscopy vs Telemedicine in Examinations for Retinopathy of Prematurity

Importance

Examinations for retinopathy of prematurity (ROP) are typically performed using binocular indirect ophthalmoscopy. Telemedicine studies have traditionally assessed the accuracy of telemedicine compared with ophthalmoscopy as a criterion standard. However, it is not known whether ophthalmoscopy is truly more accurate than telemedicine.

Objective

To directly compare the accuracy and sensitivity of ophthalmoscopy vs telemedicine in diagnosing ROP using a consensus reference standard.

Design, Setting, and Participants

This multicenter prospective study conducted between July 1, 2011, and November 30, 2014, at 7 neonatal intensive care units and academic ophthalmology departments in the United States and Mexico included 281 premature infants who met the screening criteria for ROP.

Exposures

Each examination consisted of 1 eye undergoing binocular indirect ophthalmoscopy by an experienced clinician followed by remote image review of wide-angle fundus photographs by 3 independent telemedicine graders.

Main Outcomes and Measures

Results of both examination methods were combined into a consensus reference standard diagnosis. The agreement of both ophthalmoscopy and telemedicine was compared with this standard, using percentage agreement and weighted κ statistics.

Results

Among the 281 infants in the study (127 girls and 154 boys; mean [SD] gestational age, 27.1 [2.4] weeks), a total of 1553 eye examinations were classified using both ophthalmoscopy and telemedicine. Ophthalmoscopy and telemedicine each had similar sensitivity for zone I disease (78% [95% CI, 71%-84%] vs 78% [95% CI, 73%-83%]; P > .99 [n = 165]), plus disease (74% [95% CI, 61%-87%] vs 79% [95% CI, 72%-86%]; P = .41 [n = 50]), and type 2 ROP (stage 3, zone I, or plus disease: 86% [95% CI, 80%-92%] vs 79% [95% CI, 75%-83%]; P = .10 [n = 251]), but ophthalmoscopy was slightly more sensitive in identifying stage 3 disease (85% [95% CI, 79%-91%] vs 73% [95% CI, 67%-78%]; P = .004 [n = 136]).

Conclusions and Relevance

No difference was found in overall accuracy between ophthalmoscopy and telemedicine for the detection of clinically significant ROP, although, on average, ophthalmoscopy had slightly higher accuracy for the diagnosis of zone III and stage 3 ROP. With the caveat that there was variable accuracy between examiners using both modalities, these results support the use of telemedicine for the diagnosis of clinically significant ROP.

A Quantitative Severity Scale for Retinopathy of Prematurity Using Deep Learning to Monitor Disease Regression After Treatment

Importance

Retinopathy of prematurity (ROP) is a leading cause of childhood blindness worldwide, but treatment failure and disease recurrence are important causes of adverse outcomes in patients with treatment-requiring ROP (TR-ROP).

Objectives

To apply an automated ROP vascular severity score obtained using a deep learning algorithm and to assess its utility for objectively monitoring ROP regression after treatment.

Design, Setting, and Participants

This retrospective cohort study used data from the Imaging and Informatics in ROP consortium, which comprises 9 tertiary referral centers in North America that screen high volumes of at-risk infants for ROP. Images of 5255 clinical eye examinations from 871 infants performed between July 2011 and December 2016 were assessed for eligibility in the present study. The disease course was assessed with time across the numerous examinations for patients with TR-ROP. Infants born prematurely meeting screening criteria for ROP who developed TR-ROP and who had images captured within 4 weeks before and after treatment as well as at the time of treatment were included.

Main Outcomes and Measures

The primary outcome was mean (SD) ROP vascular severity score before, at time of, and after treatment. A deep learning classifier was used to assign a continuous ROP vascular severity score, which ranged from 1 (normal) to 9 (most severe), at each examination. A secondary outcome was the difference in ROP vascular severity score among eyes treated with laser or the vascular endothelial growth factor antagonist bevacizumab. Differences between groups for both outcomes were assessed using unpaired 2-tailed t tests with Bonferroni correction.

Results

Of 5255 examined eyes, 91 developed TR-ROP, of which 46 eyes met the inclusion criteria based on the available images. The mean (SD) birth weight of those patients was 653 (185) g, with a mean (SD) gestational age of 24.9 (1.3) weeks. The mean (SD) ROP vascular severity scores significantly increased 2 weeks prior to treatment (4.19 [1.75]), peaked at treatment (7.43 [1.89]), and decreased for at least 2 weeks after treatment (4.00 [1.88]) (all P < .001). Eyes requiring retreatment with laser had higher ROP vascular severity scores at the time of initial treatment compared with eyes receiving a single treatment (P < .001).

Conclusions and Relevance

This quantitative ROP vascular severity score appears to consistently reflect clinical disease progression and posttreatment regression in eyes with TR-ROP. These study results may have implications for the monitoring of patients with ROP for treatment failure and disease recurrence and for determining the appropriate level of disease severity for primary treatment in eyes with aggressive disease.

Monitoring Disease Progression With a Quantitative Severity Scale for Retinopathy of Prematurity Using Deep Learning

Importance

Retinopathy of prematurity (ROP) is a leading cause of childhood blindness worldwide, but clinical diagnosis is subjective and qualitative.

Objective

To describe a quantitative ROP severity score derived using a deep learning algorithm designed to evaluate plus disease and to assess its utility for objectively monitoring ROP progression.

Design, Setting, and Participants

This retrospective cohort study included images from 5255 clinical examinations of 871 premature infants who met the ROP screening criteria of the Imaging and Informatics in ROP (i-ROP) Consortium, which comprises 9 tertiary care centers in North America, from July 1, 2011, to December 31, 2016. Data analysis was performed from July 2017 to May 2018.

Exposure

A deep learning algorithm was used to assign a continuous ROP vascular severity score from 1 (most normal) to 9 (most severe) at each examination based on a single posterior photograph compared with a reference standard diagnosis (RSD) simplified into 4 categories: no ROP, mild ROP, type 2 ROP or pre-plus disease, or type 1 ROP. Disease course was assessed longitudinally across multiple examinations for all patients.

Main Outcomes and Measures

Mean ROP vascular severity score progression over time compared with the RSD.

Results

A total of 5255 clinical examinations from 871 infants (mean [SD] gestational age, 27.0 [2.0] weeks; 493 [56.6%] male; mean [SD] birth weight, 949 [271] g) were analyzed. The median severity scores for each category were as follows: 1.1 (interquartile range [IQR], 1.0-1.5) (no ROP), 1.5 (IQR, 1.1-3.4) (mild ROP), 4.6 (IQR, 2.4-5.3) (type 2 and pre-plus), and 7.5 (IQR, 5.0-8.7) (treatment-requiring ROP) (P < .001). When the long-term differences in the median severity scores across time between the eyes progressing to treatment and those who did not eventually require treatment were compared, the median score was higher in the treatment group by 0.06 at 30 to 32 weeks, 0.75 at 32 to 34 weeks, 3.56 at 34 to 36 weeks, 3.71 at 36 to 38 weeks, and 3.24 at 38 to 40 weeks postmenstrual age (P < .001 for all comparisons).

Conclusions and Relevance

The findings suggest that the proposed ROP vascular severity score is associated with category of disease at a given point in time and clinical progression of ROP in premature infants. Automated image analysis may be used to quantify clinical disease progression and identify infants at high risk for eventually developing treatment-requiring ROP. This finding has implications for quality and delivery of ROP care and for future approaches to disease classification.

Vessel Tortuosity Cutoff Values Using the Modified ROPtool May Predict Need for Treatment in Retinopathy of Prematurity

BACKGROUND AND OBJECTIVE

To quantify vessel tortuosity among infants with retinopathy of prematurity (ROP).

PATIENTS AND METHODS

This was a retrospective study including 61 RetCam images from 33 infants. The laser treatment (LT) group included 17 infants who underwent laser for ROP. The no-treatment (NT) group included 16 infants. The modified ROPtool was used to calculate mean vessel tortuosity (MVT) and highest vessel tortuosity (HVT) for the participants and for the standard plus disease photograph from the Early Treatment for Retinopathy of Prematurity (ETROP) study.

RESULTS

The median MVT was 1.226 versus 1.056 for the LT and NT groups, respectively (P < .001). The median HVT was 1.346 versus 1.088 (P < .001). An MVT of 1.124 was 96.7% sensitive and 100% specific for identifying infants with treatment-requiring ROP. Both MVT and HVT cutoff values correctly captured plus disease in the standard ETROP trial photograph.

CONCLUSION

The modified ROPtool can be used to identify infants who have treatment-requiring ROP with a high level of sensitivity and specificity. [Ophthalmic Surg Lasers Imaging Retina. 2019;50:215-220.].

Automated Analysis for Retinopathy of Prematurity by Deep Neural Networks

Retinopathy of Prematurity (ROP) is a retinal vasproliferative disorder disease principally observed in infants born prematurely with low birth weight. ROP is an important cause of childhood blindness. Although automatic or semi-automatic diagnosis of ROP has been conducted, most previous studies have focused on "plus" disease, which is indicated by abnormalities of retinal vasculature. Few studies have reported methods for identifying the "stage" of the ROP disease. Deep neural networks have achieved impressive results in many computer vision and medical image analysis problems, raising expectations that it might be a promising tool in the automatic diagnosis of ROP. In this paper, convolutional neural networks with a novel architecture are proposed to recognize the existence and severity of ROP disease per-examination. The severity of ROP is divided into mild and severe cases according to the disease progression. The proposed architecture consists of two sub-networks connected by a feature aggregate operator. The first sub-network is designed to extract high-level features from images of the fundus. These features from different images in an examination are fused by the aggregate operator, then used as the input for the second sub-network to predict its class. A large data set imaged by RetCam 3 is used to train and evaluate the model. The high classification accuracy in the experiment demonstrates the effectiveness of the proposed architecture for recognizing the ROP disease.

Evaluation of a deep learning image assessment system for detecting severe retinopathy of prematurity

BACKGROUND

Prior work has demonstrated the near-perfect accuracy of a deep learning retinal image analysis system for diagnosing plus disease in retinopathy of prematurity (ROP). Here we assess the screening potential of this scoring system by determining its ability to detect all components of ROP diagnosis.

METHODS

Clinical examination and fundus photography were performed at seven participating centres. A deep learning system was trained to detect plus disease, generating a quantitative assessment of retinal vascular abnormality (the i-ROP plus score) on a 1-9 scale. Overall ROP disease category was established using a consensus reference standard diagnosis combining clinical and image-based diagnosis. Experts then ranked ordered a second data set of 100 posterior images according to overall ROP severity.

RESULTS

4861 examinations from 870 infants were analysed. 155 examinations (3%) had a reference standard diagnosis of type 1 ROP. The i-ROP deep learning (DL) vascular severity score had an area under the receiver operating curve of 0.960 for detecting type 1 ROP. Establishing a threshold i-ROP DL score of 3 conferred 94% sensitivity, 79% specificity, 13% positive predictive value and 99.7% negative predictive value for type 1 ROP. There was strong correlation between expert rank ordering of overall ROP severity and the i-ROP DL vascular severity score (Spearman correlation coefficient=0.93; p<0.0001).

CONCLUSION

The i-ROP DL system accurately identifies diagnostic categories and overall disease severity in an automated fashion, after being trained only on posterior pole vascular morphology. These data provide proof of concept that a deep learning screening platform could improve objectivity of ROP diagnosis and accessibility of screening.