Analysis of Discrepancy Between Diagnostic Clinical Examination Findings and Corresponding Evaluation of Digital Images in the Telemedicine Approaches to Evaluating Acute-Phase Retinopathy of Prematurity Study

Assessing spectral effectiveness in color fundus photography for deep learning classification of retinopathy of prematurity

Significance

Retinopathy of prematurity (ROP) poses a significant global threat to childhood vision, necessitating effective screening strategies. This study addresses the impact of color channels in fundus imaging on ROP diagnosis, emphasizing the efficacy and safety of utilizing longer wavelengths, such as red or green for enhanced depth information and improved diagnostic capabilities.

Aim

This study aims to assess the spectral effectiveness in color fundus photography for the deep learning classification of ROP.

Approach

A convolutional neural network end-to-end classifier was utilized for deep learning classification of normal, stage 1, stage 2, and stage 3 ROP fundus images. The classification performances with individual-color-channel inputs, i.e., red, green, and blue, and multi-color-channel fusion architectures, including early-fusion, intermediate-fusion, and late-fusion, were quantitatively compared.

Results

For individual-color-channel inputs, similar performance was observed for green channel (88.00% accuracy, 76.00% sensitivity, and 92.00% specificity) and red channel (87.25% accuracy, 74.50% sensitivity, and 91.50% specificity), which is substantially outperforming the blue channel (78.25% accuracy, 56.50% sensitivity, and 85.50% specificity). For multi-color-channel fusion options, the early-fusion and intermediate-fusion architecture showed almost the same performance when compared to the green/red channel input, and they outperformed the late-fusion architecture.

Conclusions

This study reveals that the classification of ROP stages can be effectively achieved using either the green or red image alone. This finding enables the exclusion of blue images, acknowledged for their increased susceptibility to light toxicity.

Multimodal Imaging, Tele-Education, and Telemedicine in Retinopathy of Prematurity

Retinopathy of prematurity (ROP) is a disease that affects retinal vasculature in premature infants and remains one of the leading causes of blindness in childhood worldwide. ROP screening can encounter some difficulties such as the lack of specialists and services in rural areas. The evolution of technology has helped address these issues and led to the emergence of state-of-the-art multimodal digital imaging devices such fundus cameras with its variable properties, optical coherence tomography (OCT), OCT angiography, and fluorescein angiography which has helped immensely in the process of improving ROP care and understanding the disease pathophysiology. Computer-based imaging analysis and deep learning have recently been demonstrating promising outcomes in regard to ROP diagnosis. Telemedicine is considered an acceptable alternative to clinical examination when optimal circumstances for ROP screening in certain areas are lacking, and the expansion of these programs has been reported. Tele-education programs in ROP have the potential to improve the quality of training to physicians to optimize ROP care.

Federated learning for multi-center collaboration in ophthalmology: implications for clinical diagnosis and disease epidemiology

OBJECTIVE OR PURPOSE

To utilize a deep learning (DL) model trained via federated learning (FL), a method of collaborative training without sharing patient data, to delineate institutional differences in clinician diagnostic paradigms and disease epidemiology in retinopathy of prematurity (ROP).

DESIGN

Evaluation of a diagnostic test or technology SUBJECTS, PARTICIPANTS, AND/OR CONTROLS: 5,245 patients with wide-angle retinal imaging from the neonatal intensive care units of 7 institutions as part of the Imaging and Informatics in ROP (i-ROP) study. Images were labeled with the clinical diagnosis of plus disease (plus, pre-plus, no plus) that was documented in the chart, and a reference standard diagnosis (RSD) determined by three image-based ROP graders and the clinical diagnosis.

METHODS, INTERVENTION OR TESTING

Demographics (birthweight [BW], gestational age [GA]), and clinical diagnoses for all eye exams were recorded from each institution. Using a FL approach, a DL model for plus disease classification was trained using only the clinical labels. The three class probabilities were then converted into a vascular severity score (VSS) for each eye exam, as well as an "institutional VSS" in which the average of the VSS values assigned to patients' higher severity ("worse") eyes at each exam was calculated for each institution.

MAIN OUTCOME MEASURES

We compared demographics, clinical diagnosis of plus disease, and institutional VSS between institutions using the McNemar Bowker test, two-proportion Z test and one-way ANOVA with post-hoc analysis by Tukey-Kramer test. Single regression analysis was performed to explore the relationship between demographics and VSS.

RESULTS

We found that the proportion of patients diagnosed with pre-plus disease varied significantly between institutions (p<0.00l). Using the DL-derived VSS trained on the data from all institutions using FL, we observed differences in the institutional VSS, as well as level of vascular severity diagnosed as no plus (p<0.001) across institutions. A significant, inverse relationship between the institutional VSS and the mean GA was found (p=0.049, adjusted R²=0.49).

CONCLUSIONS

A DL-derived ROP VSS developed without sharing data between institutions using FL identified differences in the clinical diagnosis of plus disease, and overall levels of ROP severity between institutions. FL may represent a method to standardize clinical diagnosis and provide objective measurement of disease for image-based diseases.

Extending Hui-Walter framework to correlated outcomes with application to diagnosis tests of an eye disease among premature infants

Diagnostic accuracy, a measure of diagnostic tests for correctly identifying patients with or without a target disease, plays an important role in evidence-based medicine. Diagnostic accuracy of a new test ideally should be evaluated by comparing to a gold standard; however, in many medical applications it may be invasive, costly, or even unethical to obtain a gold standard for particular diseases. When the accuracy of a new candidate test under evaluation is assessed by comparison to an imperfect reference test, bias is expected to occur and result in either overestimates or underestimates of its true accuracy. In addition, diagnostic test studies often involve repeated measurements of the same patient, such as the paired eyes or multiple teeth, and generally lead to correlated and clustered data. Using the conventional statistical methods to estimate diagnostic accuracy can be biased by ignoring the within-cluster correlations. Despite numerous statistical approaches have been proposed to tackle this problem, the methodology to deal with correlated and clustered data in the absence of a gold standard is limited. In this article, we propose a method based on the composite likelihood function to derive simple and intuitive closed-form solutions for estimates of diagnostic accuracy, in terms of sensitivity and specificity. Through simulation studies, we illustrate the relative advantages of the proposed method over the existing methods that simply treat an imperfect reference test as a gold standard in correlated and clustered data. Compared with the existing methods, the proposed method can reduce not only substantial bias, but also the computational burden. Moreover, to demonstrate the utility of this approach, we apply the proposed method to the study of National-Eye-Institute-funded Telemedicine Approaches to Evaluating of Acute-Phase Retinopathy of Prematurity (e-ROP), for estimating accuracies of both the ophthalmologist examination and the image evaluation.

Assessment and management of retinopathy of prematurity in the era of anti-vascular endothelial growth factor (VEGF)

The incidence of retinopathy of prematurity (ROP) continues to rise due to the improved survival of very low birth weight infants in developed countries. This epidemic is also fueled by increased survival of preterm babies with variable use of oxygen and a lack of ROP awareness and screening services in resource-limited regions. Improvements in technology and a basic understanding of the disease pathophysiology have changed the way we screen and manage ROP, educate providers and patients, and improve ROP awareness. Advancements in imaging techniques, expansion of telemedicine services, and the potential for artificial intelligence-assisted ROP screening programs have created opportunities to improve ROP care in areas with a shortage of ophthalmologists trained in ROP. To address the gap in provider knowledge regarding ROP, the Global Education Network for Retinopathy of Prematurity (GEN-ROP) created a web-based tele-education training module that can be used to educate all providers involved in ROP, including non-physician ROP screeners. Over the past 50 years, the treatment of severe ROP has evolved from limited treatment modalities to cryotherapy and laser photocoagulation. More recently, there has been growing evidence to support the use of anti-vascular endothelial growth factor (VEGF) agents for the treatment of severe ROP. However, VEGF is known to be important in organogenesis and microvascular maintenance, and given that intravitreal anti-VEGF treatment can result in systemic VEGF suppression over a period of at least 1-12 weeks, there are concerns regarding adverse effects and long-term ocular and systemic developmental consequences of anti-VEGF therapy. Future research in ophthalmology to address the growing burden of ROP should focus on cost-effective fundus imaging devices, implementation of artificial intelligence platforms, updated treatment algorithms with optimal use of anti-VEGF and careful investigation of its long-term effects, and surgical options in advanced ROP. Addressing these unmet needs will aid the global effort against the ROP epidemic and optimize our understanding and treatment of this blinding disease.

Evaluation of a Deep Learning-Derived Quantitative Retinopathy of Prematurity Severity Scale

PURPOSE

To evaluate the clinical usefulness of a quantitative deep learning-derived vascular severity score for retinopathy of prematurity (ROP) by assessing its correlation with clinical ROP diagnosis and by measuring clinician agreement in applying a novel scale.

DESIGN

Analysis of existing database of posterior pole fundus images and corresponding ophthalmoscopic examinations using 2 methods of assigning a quantitative scale to vascular severity.

PARTICIPANTS

Images were from clinical examinations of patients in the Imaging and Informatics in ROP Consortium. Four ophthalmologists and 1 study coordinator evaluated vascular severity on a scale from 1 to 9.

METHODS

A quantitative vascular severity score (1-9) was applied to each image using a deep learning algorithm. A database of 499 images was developed for assessment of interobserver agreement.

MAIN OUTCOME MEASURES

Distribution of deep learning-derived vascular severity scores with the clinical assessment of zone (I, II, or III), stage (0, 1, 2, or 3), and extent (<3 clock hours, 3-6 clock hours, and >6 clock hours) of stage 3 evaluated using multivariate linear regression and weighted κ values and Pearson correlation coefficients for interobserver agreement on a 1-to-9 vascular severity scale.

RESULTS

For deep learning analysis, a total of 6344 clinical examinations were analyzed. A higher deep learning-derived vascular severity score was associated with more posterior disease, higher disease stage, and higher extent of stage 3 disease (P < 0.001 for all). For a given ROP stage, the vascular severity score was higher in zone I than zones II or III (P < 0.001). Multivariate regression found zone, stage, and extent all were associated independently with the severity score (P < 0.001 for all). For interobserver agreement, the mean ± standard deviation weighted κ value was 0.67 ± 0.06, and the Pearson correlation coefficient ± standard deviation was 0.88 ± 0.04 on the use of a 1-to-9 vascular severity scale.

CONCLUSIONS

A vascular severity scale for ROP seems feasible for clinical adoption; corresponds with zone, stage, extent of stage 3, and plus disease; and facilitates the use of objective technology such as deep learning to improve the consistency of ROP diagnosis.

Evaluation of high-definition video smart glasses for real-time telemedicine strabismus consultations

PURPOSE

To evaluate the use of high-definition, wireless video recording Pivothead glasses in streamed strabismus examinations.

METHODS

A pediatric ophthalmologist wearing Pivothead glasses simultaneously performed and recorded strabismus examinations, which were completed in primary gaze, with and without correction, and at distance and near. Parameters included category of strabismus, angle measurements, and ocular motility. Another pediatric ophthalmologist, masked to clinical findings, reviewed and graded live video feed. At least 3 months later, both pediatric ophthalmologists graded the stored videos. Agreement was determined by unweighted kappa for categorical variables, intraclass coefficient for continuous variables, and percent agreement.

RESULTS

A total of 100 videos were recorded for 37 enrolled patients aged 4-16 years. Agreement between streamed and in-person examinations was perfect for both horizontal and vertical (κ = 1.0) deviations. Agreement for degree manifest (tropia vs intermittent tropia vs phoria) was almost perfect for all deviations (κ = 0.94). Agreement was excellent for angle measurements of both horizontal (ICC = 0.95) and vertical (ICC = 0.91) deviations. Inferior and superior oblique agreement was 93% and 98%, respectively. Similar agreement was also observed between store-and-forward versus gold standard examinations.

CONCLUSIONS

Real-time video feed obtained with video glasses can be read with a high degree of reliability for detecting strabismus category, angle, and extraocular motility.

Key factors in a rigorous longitudinal image-based assessment of retinopathy of prematurity

To describe a database of longitudinally graded telemedicine retinal images to be used as a comparator for future studies assessing grader recall bias and ability to detect typical progression (e.g. International Classification of Retinopathy of Prematurity (ICROP) stages) as well as incremental changes in retinopathy of prematurity (ROP). Cohort comprised of retinal images from 84 eyes of 42 patients who were sequentially screened for ROP over 6 consecutive weeks in a telemedicine program and then followed to vascular maturation or treatment, and then disease stabilization. De-identified retinal images across the 6 weekly exams (2520 total images) were graded by an ROP expert based on whether ROP had improved, worsened, or stayed the same compared to the prior week’s images, corresponding to an overall clinical “gestalt” score. Subsequently, we examined which parameters might have influenced the examiner’s ability to detect longitudinal change; images were graded by the same ROP expert by image view (central, inferior, nasal, superior, temporal) and by retinal components (vascular tortuosity, vascular dilation, stage, hemorrhage, vessel growth), again determining if each particular retinal component or ROP in each image view had improved, worsened, or stayed the same compared to the prior week’s images. Agreement between gestalt scores and view, component, and component by view scores was assessed using percent agreement, absolute agreement, and Cohen’s weighted kappa statistic to determine if any of the hypothesized image features correlated with the ability to predict ROP disease trajectory in patients. The central view showed substantial agreement with gestalt scores (κ = 0.63), with moderate agreement in the remaining views. Of retinal components, vascular tortuosity showed the most overall agreement with gestalt (κ = 0.42–0.61), with only slight to fair agreement for all other components. This is a well-defined ROP database graded by one expert in a real-world setting in a masked fashion that correlated with the actual (remote in time) exams and known outcomes. This provides a foundation for subsequent study of telemedicine’s ability to longitudinally assess ROP disease trajectory, as well as for potential artificial intelligence approaches to retinal image grading, in order to expand patient access to timely, accurate ROP screening.

Current Application of Digital Diagnosing Systems for Retinopathy of Prematurity

BACKGROUND AND OBJECTIVE

Retinopathy of prematurity (ROP), a proliferative vascular eye disease, is one of the leading causes of blindness in childhood and prevails in premature infants with low-birth-weight. The recent progress in digital image analysis offers novel strategies for ROP diagnosis. This paper provides a comprehensive review on the development of digital diagnosing systems for ROP to software researchers. It may also be adopted as a guide to ophthalmologists for selecting the most suitable diagnostic software in the clinical setting, particularly for the remote ophthalmic support.

METHODS

We review the latest literatures concerning the application of digital diagnosing systems for ROP. The diagnosing systems are analyzed and categorized. Articles published between 1998 and 2020 were screened with the two searching engines Pubmed and Google Scholar.

RESULTS

Telemedicine is a method of remote image interpretation that can provide medical service to remote regions, and yet requires training to local operators. On the basis of image collection in telemedicine, computer-based image analytical systems for ROP were later developed. So far, the aforementioned systems have been mainly developed by virtue of classic machine learning, deep learning (DL) and multiple machine learning. During the past two decades, various computer-aided systems for ROP based on classic machine learning (e.g. RISA, ROPtool, CAIER) became available and have achieved satisfactory performance. Further, automated systems for ROP diagnosis based on DL are developed for clinical applications and exhibit high accuracy. Moreover, multiple instance learning is another method to establish an automated system for ROP detection besides DL, which, however, warrants further investigation in future.

CONCLUSION

At present, the incorporation of computer-based image analysis with telemedicine potentially enables the detection, supervision and in-time treatment of ROP for the preterm babies.

Evaluation of real-time video from the digital indirect ophthalmoscope for telemedicine consultations in retinopathy of prematurity

Introduction

This study aimed to evaluate the validity of using real-time video indirect ophthalmoscopy for telemedicine consultations for paediatric retinal findings, using retinopathy of prematurity (ROP) as a model disease.

Methods

An ophthalmologist simultaneously performed and recorded routine ROP screenings on enrolled premature infants (aged ≤30 weeks and weighing <1500 g) using the Keeler digital indirect ophthalmoscope. Examinations were graded as no ROP, mild, type 1 or type 2. Masked to clinical findings, another ophthalmologist reviewed and graded streamed video feed transmitted at 4096 kbps. We compared the sensitivity and specificity of diagnosing ROP via streamed and stored-and-forwarded video clips to the gold-standard in-person examination.

Results

A total of 150 examinations of individual eyes from 34 babies were included. The median postmenstrual age (PMA) at delivery was 24 weeks (range 23–34 weeks), the median birth weight was 630 g (range 455–1530 g) and the median PMA at examination was 37 weeks (range 31–54 weeks). Of those infants with any ROP, the sensitivity and specificity of streamed examinations were 100% and 70.6%, respectively. For type 2 or worse ROP, the sensitivity and specificity were 92.5% and 86.1%, respectively. For type 1 ROP, the sensitivity and specificity were 100% and 99.3%, respectively. Sensitivities (unless already 100%) and specificities were slightly higher for store-and-forward evaluations.

Discussion

Streamed video feed from the digital indirect ophthalmoscope can be utilised to diagnose clinically significant ROP accurately, though store-and-forward video review yielded slightly better results.

Comparison of wide field imaging by nurses with indirect ophthalmoscopy by ophthalmologists for retinopathy of prematurity: a diagnostic accuracy study

OBJECTIVES

Retinopathy of prematurity (ROP) is a vasoproliferative disease of the preterm retina with the potential to cause irreversible blindness. Timely screening and treatment of ROP are critical. Neonatal nurses trained in wide field digital retinal photography (WFDRP) for screening may provide a safe and effective strategy to reduce the burden of ophthalmologists in performing binocular indirect ophthalmoscopy (BIO). The objective of the study was to determine the diagnostic accuracy of WFDRP in the diagnosis of referral warranting ROP (RWROP).

DESIGN

Prospective diagnostic accuracy study.

SETTING

A tertiary neonatal intensive care unit in Perth, Western Australia.

PARTICIPANTS

Preterm infants who fulfilled the Australian ROP screening criteria (gestational age (GA) <31 weeks, birth weight (BW) <1250 g).

INTERVENTION

Sets of 5-6 images per eye (index test) were obtained within 24-48 hours prior to or after the BIO (reference standard), and uploaded onto a secured server. A wide field digital camera (RetCam, Natus, Pleasanton, California, USA) was used for imaging. A paediatric ophthalmologist performed the BIO. The ophthalmologists performing BIO versus reporting the images were masked to each other's findings.

PRIMARY OUTCOME

The area under the receiver operating characteristic (ROC) curve was used as a measure of accuracy of WFDRP to diagnose RWROP.

RESULTS

A total of 85 infants (mean BW; 973.43 g, mean GA; 29 weeks) underwent a median of two sessions of WFDRP. There were 188 episodes of screening with an average of five images per eye. WFDRP identified RWROP in 7.4% (14/188 sessions) of examinations. In one infant, BIO showed bilateral plus disease and WFDRP did not pick up the plus disease. WFDRP image interpretation had a sensitivity of 80%, specificity of 94.5% for the detection of RWROP. The 'area under the ROC curve' was 88% when adjusted for covariates.

CONCLUSIONS

WFDRP by neonatal nurses was feasible and effective for diagnosing RWROP in our set up.

TRIAL REGISTRATION NUMBER

ACTRN12616001386426.

Artificial intelligence for retinopathy of prematurity

PURPOSE OF REVIEW

In this article, we review the current state of artificial intelligence applications in retinopathy of prematurity (ROP) and provide insight on challenges as well as strategies for bringing these algorithms to the bedside.

RECENT FINDINGS

In the past few years, there has been a dramatic shift from machine learning approaches based on feature extraction to 'deep' convolutional neural networks for artificial intelligence applications. Several artificial intelligence for ROP approaches have demonstrated adequate proof-of-concept performance in research studies. The next steps are to determine whether these algorithms are robust to variable clinical and technical parameters in practice. Integration of artificial intelligence into ROP screening and treatment is limited by generalizability of the algorithms to maintain performance on unseen data and integration of artificial intelligence technology into new or existing clinical workflows.

SUMMARY

Real-world implementation of artificial intelligence for ROP diagnosis will require massive efforts targeted at developing standards for data acquisition, true external validation, and demonstration of feasibility. We must now focus on ethical, technical, clinical, regulatory, and financial considerations to bring this technology to the infant bedside to realize the promise offered by this technology to reduce preventable blindness from ROP.

Prediction of retinopathy of prematurity using the screening algorithm WINROP in a Saudi cohort of preterm infants

OBJECTIVES

To validate the web weight gain-based WINROP (weight, insulin-like growth factor I, neonatal, retinopathy of prematurity [ROP]) algorithm retrospectively to identify type 1 ROP in a Saudi cohort of premature infants.  Methods: The records of preterm infants (greater than 23 and less than 32 weeks gestation) born between August 2013 and October 2018, were reviewed. Birth weight, gestational age, and weekly weight measurements of the premature infants were entered online. Based on weekly weight gain, the WINROP algorithm alerted clinicians whether infants were at high-risk for vision‑threatening type 1 ROP. Sensitivity, specificity, positive and negative predictive values were calculated. Results: The median gestational age of the infants at birth was 28 weeks, with median birth weight at 1085 g. Of the 175 infants included in the study, 13 (7.4%) developed type 1 ROP. WINROP positive alarm was triggered in 70.9% (124/175) of all infants and 100% (13/13) of those treated for type 1 ROP. The specificity of the algorithm was 31.5%. Positive predictive values was 10.5% and negative was 100%. Conclusion: The general WINROP sensitivity in identifying type 1 ROP was 100% similar to that reported in developed countries; however, its specificity was low at 31.5%. Tweaking of the algorithm based on the population may increase the specificity and promote the practical utility of this non-invasive screening tool for ophthalmologists and neonatologists in this population.

Discrepancy analysis comparing molecular and histology diagnoses in kidney transplant biopsies

Discrepancy analysis comparing two diagnostic platforms offers potential insights into both without assuming either is always correct. Having optimized the Molecular Microscope Diagnostic System (MMDx) in renal transplant biopsies, we studied discrepancies within MMDx (reports and sign-out comments) and between MMDx and histology. Interpathologist discrepancies have been documented previously and were not assessed. Discrepancy cases were classified as "clear" (eg, antibody-mediated rejection [ABMR] vs T cell-mediated rejection [TCMR]), "boundary" (eg, ABMR vs possible ABMR), or "mixed" (eg, Mixed vs ABMR). MMDx report scores showed 99% correlations; sign-out interpretations showed 7% variation between observers, all located around boundaries. Histology disagreed with MMDx in 37% of biopsies, including 315 clear discrepancies, all with implications for therapy. Discrepancies were distributed widely in all histology diagnoses but increased in some scenarios; for example, histology TCMR contained 14% MMDx ABMR and 20% MMDx no rejection. MMDx usually gave unambiguous diagnoses in cases with ambiguous histology, for example, borderline and transplant glomerulopathy. Histology lesions or features associated with more frequent discrepancies (eg, tubulitis, arteritis, and polyomavirus nephropathy) were not associated with increased MMDx uncertainty, indicating that MMDx can clarify biopsies with histologic ambiguity. The patterns of histology-MMDx discrepancies highlight specific histology diagnoses in which MMDx assessment should be considered for guiding therapy.

Practice Guidelines for Ocular Telehealth-Diabetic Retinopathy, Third Edition

Contributors The following document and appendices represent the third edition of the Practice Guidelines for Ocular Telehealth-Diabetic Retinopathy. These guidelines were developed by the Diabetic Retinopathy Telehealth Practice Guidelines Working Group. This working group consisted of a large number of subject matter experts in clinical applications for telehealth in ophthalmology. The editorial committee consisted of Mark B. Horton, OD, MD, who served as working group chair and Christopher J. Brady, MD, MHS, and Jerry Cavallerano, OD, PhD, who served as cochairs. The writing committees were separated into seven different categories. They are as follows: 1.Clinical/operational: Jerry Cavallerano, OD, PhD (Chair), Gail Barker, PhD, MBA, Christopher J. Brady, MD, MHS, Yao Liu, MD, MS, Siddarth Rathi, MD, MBA, Veeral Sheth, MD, MBA, Paolo Silva, MD, and Ingrid Zimmer-Galler, MD. 2.Equipment: Veeral Sheth, MD (Chair), Mark B. Horton, OD, MD, Siddarth Rathi, MD, MBA, Paolo Silva, MD, and Kristen Stebbins, MSPH. 3.Quality assurance: Mark B. Horton, OD, MD (Chair), Seema Garg, MD, PhD, Yao Liu, MD, MS, and Ingrid Zimmer-Galler, MD. 4.Glaucoma: Yao Liu, MD, MS (Chair) and Siddarth Rathi, MD, MBA. 5.Retinopathy of prematurity: Christopher J. Brady, MD, MHS (Chair) and Ingrid Zimmer-Galler, MD. 6.Age-related macular degeneration: Christopher J. Brady, MD, MHS (Chair) and Ingrid Zimmer-Galler, MD. 7.Autonomous and computer assisted detection, classification and diagnosis of diabetic retinopathy: Michael Abramoff, MD, PhD (Chair), Michael F. Chiang, MD, and Paolo Silva, MD.

Telemedicine for Retinopathy of Prematurity

Background: Retinopathy of prematurity (ROP) is a disease of the retinal vasculature that remains a leading cause of childhood blindness worldwide despite improvements in the systemic care of premature newborns. Screening for ROP is effective and cost-effective, but in many areas, access to skilled examiners to conduct dilated examinations is poor. Remote screening with retinal photography is an alternative strategy that may allow for improved ROP care. Methods: The current literature was reviewed to find clinical trials and expert consensus documents on the state-of-the-art of telemedicine for ROP. Results: Several studies have confirmed the utility of telemedicine for ROP. In addition, several clinical studies have reported favorable long-term results. Many investigators have reinforced the need for detailed protocols on image acquisition and image interpretation. Conclusions: Telemedicine for ROP appears to be a viable alternative to live ophthalmoscopic examinations in many circumstances. Standardization and documentation afforded by telemedicine may provide additional benefits to providers and their patients. With continued improvements in image quality and affordability of imaging systems as well as improved automated image interpretation tools anticipated in the near future, telemedicine for ROP is expected to play an expanding role for a uniquely vulnerable patient population.

Telemedicine for Retinopathy of Prematurity in 2020

Purpose:

Retinopathy of prematurity (ROP) is the leading cause of visual impairment in premature infants, and middle-income nations are currently experiencing the “third epidemic” of ROP. Screening programs are essential to prevent negative visual outcomes, but screening efforts require a great amount of resources from healthcare systems and are difficult to marshal, particularly in geographically isolated or resource-limited settings. Telemedical screening programs using remote digital fundus imaging (RDFI) systems hold the promise of alleviating many of the burdens that currently make screening for ROP logistically challenging.

Methods:

Literature review of the current evidence for RDFI telescreening for ROP, with editorial discussion and recommendations.

Results:

In this review, we summarize the robust body of literature regarding the efficacy of RDFI, the feasibility of telescreening programs, and experiences from current live telescreening programs. We discuss the strengths and limitations of the current evidence and of the screening programs and consider the best practices in developing de novo telemedical screening programs for ROP. The review concludes with a discussion of promising future areas of research and development.

Conclusions:

RDFI ROP screening programs can be accurate and reliable. They show promise in improving many current challenges in screening infants for ROP, may be able to improve some aspects of care, and have been demonstrated to be cost-effective.

The role of retinal photography and telemedicine in ROP screening

Until advances in retinal imaging over recent decades, detection of retinopathy of prematurity requiring treatment depended largely on indirect ophthalmoscopy by ophthalmologists with experience in examination of low birth weight, often fragile infants. Recent studies have added to our understanding of the pros and cons of using this technology to provide timely care for the increasing number of infants at risk for ROP blindness worldwide.

A Quantitative Severity Scale for Retinopathy of Prematurity Using Deep Learning to Monitor Disease Regression After Treatment

Importance

Retinopathy of prematurity (ROP) is a leading cause of childhood blindness worldwide, but treatment failure and disease recurrence are important causes of adverse outcomes in patients with treatment-requiring ROP (TR-ROP).

Objectives

To apply an automated ROP vascular severity score obtained using a deep learning algorithm and to assess its utility for objectively monitoring ROP regression after treatment.

Design, Setting, and Participants

This retrospective cohort study used data from the Imaging and Informatics in ROP consortium, which comprises 9 tertiary referral centers in North America that screen high volumes of at-risk infants for ROP. Images of 5255 clinical eye examinations from 871 infants performed between July 2011 and December 2016 were assessed for eligibility in the present study. The disease course was assessed with time across the numerous examinations for patients with TR-ROP. Infants born prematurely meeting screening criteria for ROP who developed TR-ROP and who had images captured within 4 weeks before and after treatment as well as at the time of treatment were included.

Main Outcomes and Measures

The primary outcome was mean (SD) ROP vascular severity score before, at time of, and after treatment. A deep learning classifier was used to assign a continuous ROP vascular severity score, which ranged from 1 (normal) to 9 (most severe), at each examination. A secondary outcome was the difference in ROP vascular severity score among eyes treated with laser or the vascular endothelial growth factor antagonist bevacizumab. Differences between groups for both outcomes were assessed using unpaired 2-tailed t tests with Bonferroni correction.

Results

Of 5255 examined eyes, 91 developed TR-ROP, of which 46 eyes met the inclusion criteria based on the available images. The mean (SD) birth weight of those patients was 653 (185) g, with a mean (SD) gestational age of 24.9 (1.3) weeks. The mean (SD) ROP vascular severity scores significantly increased 2 weeks prior to treatment (4.19 [1.75]), peaked at treatment (7.43 [1.89]), and decreased for at least 2 weeks after treatment (4.00 [1.88]) (all P < .001). Eyes requiring retreatment with laser had higher ROP vascular severity scores at the time of initial treatment compared with eyes receiving a single treatment (P < .001).

Conclusions and Relevance

This quantitative ROP vascular severity score appears to consistently reflect clinical disease progression and posttreatment regression in eyes with TR-ROP. These study results may have implications for the monitoring of patients with ROP for treatment failure and disease recurrence and for determining the appropriate level of disease severity for primary treatment in eyes with aggressive disease.

Monitoring Disease Progression With a Quantitative Severity Scale for Retinopathy of Prematurity Using Deep Learning

Importance

Retinopathy of prematurity (ROP) is a leading cause of childhood blindness worldwide, but clinical diagnosis is subjective and qualitative.

Objective

To describe a quantitative ROP severity score derived using a deep learning algorithm designed to evaluate plus disease and to assess its utility for objectively monitoring ROP progression.

Design, Setting, and Participants

This retrospective cohort study included images from 5255 clinical examinations of 871 premature infants who met the ROP screening criteria of the Imaging and Informatics in ROP (i-ROP) Consortium, which comprises 9 tertiary care centers in North America, from July 1, 2011, to December 31, 2016. Data analysis was performed from July 2017 to May 2018.

Exposure

A deep learning algorithm was used to assign a continuous ROP vascular severity score from 1 (most normal) to 9 (most severe) at each examination based on a single posterior photograph compared with a reference standard diagnosis (RSD) simplified into 4 categories: no ROP, mild ROP, type 2 ROP or pre-plus disease, or type 1 ROP. Disease course was assessed longitudinally across multiple examinations for all patients.

Main Outcomes and Measures

Mean ROP vascular severity score progression over time compared with the RSD.

Results

A total of 5255 clinical examinations from 871 infants (mean [SD] gestational age, 27.0 [2.0] weeks; 493 [56.6%] male; mean [SD] birth weight, 949 [271] g) were analyzed. The median severity scores for each category were as follows: 1.1 (interquartile range [IQR], 1.0-1.5) (no ROP), 1.5 (IQR, 1.1-3.4) (mild ROP), 4.6 (IQR, 2.4-5.3) (type 2 and pre-plus), and 7.5 (IQR, 5.0-8.7) (treatment-requiring ROP) (P < .001). When the long-term differences in the median severity scores across time between the eyes progressing to treatment and those who did not eventually require treatment were compared, the median score was higher in the treatment group by 0.06 at 30 to 32 weeks, 0.75 at 32 to 34 weeks, 3.56 at 34 to 36 weeks, 3.71 at 36 to 38 weeks, and 3.24 at 38 to 40 weeks postmenstrual age (P < .001 for all comparisons).

Conclusions and Relevance

The findings suggest that the proposed ROP vascular severity score is associated with category of disease at a given point in time and clinical progression of ROP in premature infants. Automated image analysis may be used to quantify clinical disease progression and identify infants at high risk for eventually developing treatment-requiring ROP. This finding has implications for quality and delivery of ROP care and for future approaches to disease classification.

Experimental Evidence Behind Clinical Trial Outcomes in Retinopathy of Prematurity

Treatment of severe retinopathy of prematurity (ROP) has evolved over the last decade. This article reviews recent clinical trials and experimental evidence that supports clinical outcomes and observations, including the efficacy of anti-vascular endothelial growth factor (VEGF) agents in reducing the vascular activity of severe ROP, and the mechanisms behind recurrent stage 3 ROP and plus disease in some infants treated with anti-VEGF agents. Also discussed will be current imaging modalities that link experimental models of ROP with longitudinal human studies and which provide exciting future opportunities to enhance the understanding of pathophysiology of ROP and improve treatments. [Ophthalmic Surg Lasers Imaging Retina. 2019;50:228-234.].

Single grading vs double grading with adjudication in the telemedicine approaches to evaluating acute-phase retinopathy of prematurity (e-ROP) study

PURPOSE

To evaluate the sensitivity and specificity of single, independent, nonphysician trained reader (TR) gradings in the Telemedicine Approaches to Evaluating Acute-phase Retinopathy of Prematurity (e-ROP) study.

METHODS

Secondary analyses of image grading results from 1,235 infants of birth weights <1251 g. Two of three TRs independently graded image sets; discrepancies were adjudicated by the reading center director (an ophthalmologist) to reach final grading. Sensitivity and specificity of each TR grading and final grading was calculated by comparing gradings to clinical examination results.

RESULTS

Of 7,808 double graded image sets, TR1 graded 5,165; TR2, 3,787; and TR3, 6,664. Compared to final grading for referral warranted retinopathy of prematurity (RW-ROP), two TRs had relatively lower sensitivity (TR1, 75% vs 79% [P = 0.03]; TR2, 73% vs 77% [P = 0.02]) and specificity (TR1, 80% vs 83% [P < 0.001]; TR2, 82% vs 83% [P = 0.09]). TR3 had similar sensitivity (83% vs 83% [P = 0.78]) and specificity (83% vs 84% [P = 0.02]). Compared to final grading, TR1 had lower sensitivity for zone I ROP (47% vs 56% [P = 0.04]) and stage ≥3 ROP (71% vs 77% [P = 0.002]); TR2 had lower sensitivity for stage ≥3 ROP (69% vs 77% [P < 0.001]) and lower specificity for all three components (P < 0.001); TR3 had lower sensitivity for detecting plus disease (23% vs 35% [P < 0.001]) and similar sensitivity for zone I ROP and stage ≥3 ROP.

CONCLUSIONS

There is a small but significant decrease in the sensitivity and specificity for RW-ROP when single-reader grading is compared to double adjudicated grading.

Telemedicine for ROP

With the increasing survival of preterm babies in the Asia-Pacific region, there has been an increase in the incidence of retinopathy of prematurity (ROP). There is also a lack of trained ophthalmologists to screen for this disease. New strategies for screening of ROP are required, wherein the load on the ophthalmologist can be reduced. Telemedicine is an excellent tool to address this need. This paper reviews the initial studies comparing digital imaging with indirect ophthalmoscopy, the current active ROP telescreening programs in the world, and ends with studies dealing with automated diagnosis of ROP.

Accuracy and Reliability of a Handheld, Nonmydriatic Fundus Camera for the Remote Detection of Optic Disc Edema

BACKGROUND

Optic disc edema can be an important indicator of serious neurological disease, but is poorly detected using the direct ophthalmoscope. Portable fundus photography may overcome this difficulty.

INTRODUCTION

The purpose of this study was to determine the sensitivity and specificity of a handheld, nonmydriatic fundus camera for the detection of optic disc edema.

MATERIALS AND METHODS

Retrospective review of nonmydriatic optic disc photographs taken with a portable fundus camera (Pictor Plus; Volk Optical, Mentor, OH) from the University of Michigan Neuro-Ophthalmology Clinics. We included 103 consecutive eyes with optic disc edema and 103 consecutive eyes without optic disc edema of 109 patients. Four masked neuro-ophthalmologists graded a single photograph of each optic disc presented in randomized order and documented the presence of optic disc edema. Sensitivity and specificity of graders' photographic interpretation was compared with clinical examinations. Reliability of assessments within and between graders was determined using kappa statistics.

RESULTS

The sensitivity and specificity for detection of optic disc edema were 71.8-92.2% and 81.6-95.2%, respectively. Photos were found to be ungradable in 0-8.3% of cases. The intergrader reliabilities ranged from 0.60 [95% confidence interval (CI): 0.52-0.67] to 0.72 (95% CI: 0.66-0.77). Intragrader reliability ranged from 0.76 (95% CI: 0.63-0.92) to 0.82 (95% CI: 0.69-0.95).

DISCUSSION

Photographs taken with portable, nonmydriatic technology met threshold sensitivity and specificity for remote screening for optic disc edema when performed by most, but not all graders. Reliability between graders was moderate-strong and strong within individual providers.

CONCLUSIONS

Portable photography holds promise for use in remote screening of optic disc edema.

Detection of Potentially Severe Retinopathy of Prematurity by Remote Image Grading

Importance

Telemedicine in retinopathy of prematurity (ROP) has the potential for delivering timely care to premature infants at risk for serious ROP.

Objective

To describe the characteristics of eyes at risk for ROP to provide insights into what types of ROP are most easily detected early by image grading.

Design, Setting, and Participants

Secondary analysis of eyes with referral-warranted (RW) ROP (stage 3 ROP, zone I ROP, plus disease) on diagnostic examination from the Telemedicine Approaches to Evaluating Acute-Phase Retinopathy of Prematurity (e-ROP) study was conducted from May 1, 2011, to October 31, 2013, in 1257 premature infants with birth weights less than 1251 g in 13 neonatal units in North America. Data analysis was performed between February 1, 2016, and June 5, 2017.

Interventions

Serial imaging sessions with concurrent diagnostic examinations for ROP.

Main Outcomes and Measures

Time of detecting RW-ROP on image evaluation compared with clinical examination.

Results

In the e-ROP study, 246 infants (492 eyes) were included in the analysis; 138 (56.1%) were male. A total of 447 eyes had RW-ROP on diagnostic examination. Image grading in 123 infants (mean [SD] gestational age, 24.8 [1.4] weeks) detected RW-ROP earlier than diagnostic examination (early) in 191 (42.7%) eyes by about 15 days and detected RW-ROP in 123 infants (mean [SD] gestational age, 24.6 [1.5] weeks) at the same time (same) in 200 (44.7%) eyes. Most of the early eyes (153 [80.1%]) interpreted as being RW-ROP positive on imaging evaluation agreed with examination findings when the examination subsequently documented RW-ROP. At the sessions in which RW-ROP was first found by examination, stage 3 or more in 123 infants (mean [SD] gestational age, 24.8 [1.4] weeks) ROP was noted earlier on image evaluation in 151 of 191 early eyes (79.1%) and in 172 of 200 of same eyes (86.0%) (P = .08); the presence of zone I ROP was detected in 57 of 191 (29.8%) early eyes vs 64 of 200 (32.0%) same eyes (P = .90); and plus disease was noted in 30 of 191 (15.7%) early eyes and 45 of 200 (22.5%) same eyes (P = .08).

Conclusions and Relevance

In both early and same eyes, zone I and/or stage 3 ROP determined a significant proportion of RW-ROP; plus disease played a relatively minor role. In most early RW-ROP eyes, the findings were consistent with clinical examination and/or image grading at the next session. Because ROP telemedicine is used more widely, development of standard approaches and protocols is essential.

Teleophthalmology image-based navigated retinal laser therapy for diabetic macular edema: a concept of retinal telephotocoagulation

BACKGROUND

To determine the feasibility and efficacy of a retinal telephotocoagulation treatment plan for diabetic macular edema.

METHODS

Prospective, interventional cohort study at two clinical sites. Sixteen eyes of ten subjects with diabetic macular edema underwent navigated focal laser photocoagulation using a novel teleretinal treatment plan. Clinic 1 (King Khaled Eye Specialist Hospital, Riyadh, Saudi Arabia) collected retinal images and fundus fluorescein angiogram. Clinic 2 (Palmetto Retina Center, West Columbia, SC, USA) created image-based treatment plans based on which macular laser photocoagulation was performed back at clinic 1. The primary outcome of the study was feasibility of image transfer and performing navigated laser photocoagulation for subjects with diabetic macular edema between two distant clinics. Secondary measures were change in best-corrected visual acuity (BCVA) and central retinal thickness (CRT) by spectral-domain optical coherence tomography at 3 months after treatment.

RESULTS

The teleretinal treatment plan was able to be successfully completed in all 16 eyes. The mean logMAR BCVA at baseline was 0.49 ± 0.1, which remained stable (0.45 ± 0.1) 3 months after treatment (p = 0.060). The CRT improved from 290.1 ± 37.6 μm at baseline to 270.8 ± 27.7 μm 3 months after treatment (p = 0.005). All eyes demonstrated improvement in the area of retinal edema after laser photocoagulation, and no eyes demonstrated visual acuity loss 3 months after treatment.

CONCLUSION

This study introduces the concept of retinal telephotocoagulation for diabetic macular edema, and demonstrates the feasibility and safety of using telemedicine to perform navigated retinal laser treatments regardless of geographical distance.

Telemedicine Applications in Pediatric Retinal Disease

Teleophthalmology is a developing field that presents diverse opportunities. One of its most successful applications to date has been in pediatric retinal disease, particularly in screening for retinopathy of prematurity (ROP). Many studies have shown that using telemedicine for ROP screening allows a remote ophthalmologist to identify abnormal findings and implement early interventions. Here, we review the literature on uses of telemedicine in pediatric retinal disease and consider future applications.