1
|
Huang YP, Vadloori S, Kang EYC, Fukushima Y, Takahashi R, Wu WC. Computer-aided detection of retinopathy of prematurity severity assessment via vessel tortuosity measurement in preterm infants' fundus images. Eye (Lond) 2024:10.1038/s41433-024-03285-w. [PMID: 39097674 DOI: 10.1038/s41433-024-03285-w] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/20/2024] [Revised: 07/12/2024] [Accepted: 07/23/2024] [Indexed: 08/05/2024] Open
Abstract
OBJECTIVE To develop a computer-aided diagnostic system for retinopathy of prematurity (ROP) disease using retinal vessel morphological features. METHODS A total of 200 fundus images from 136 preterm infants with stage 1 to 3 ROP were analysed. Two methods were developed to measure vessel tortuosity: the peak-and-valley method and the polynomial curve fitting method. Correlations between temporal artery tortuosity (TAT) and temporal vein tortuosity (TVT) with ROP severity were investigated, and vessel tortuosity relationships with vessel angles (TAA and TVA) and vessel widths (TAW and TVW). A separate dataset from Japan containing 126 images from 97 preterm patients was used for verification. RESULTS Both methods identified similar tortuosity in images without ROP and mild ROP cases. However, the polynomial curve fit method demonstrated enhanced tortuosity detection in stages 2 and 3 ROP compared to the peak and valley method. A strong positive correlation was revealed between ROP severity and increased arterial and venous tortuosity (P < 0.0001). A significant negative correlation between TAA and TAT (r = -0.485, P < 0.0001) and TVA and TVT (r = -0.281, P < 0.0001), and a significant positive correlation between TAW and TAT (r = 0.204, P value = 0.0040) were identified. Similar results were found in the test dataset from Japan. CONCLUSIONS ROP severity was associated with increased retinal tortuosity and retinal vessel width while displaying a decrease in retinal vascular angle. This quantitative analysis of retinal vessels provides crucial insights for advancing ROP diagnosis and understanding its progression.
Collapse
Grants
- CORPG3L0131, CMRPG3M0131~2, and CMRPG3L0151~3 Chang Gung Memorial Hospital (CGMH)
- CORPG3L0131, CMRPG3M0131~2, and CMRPG3L0151~3 Chang Gung Memorial Hospital (CGMH)
- NTUT-CGMH-110-01 and NTUT-CGMH-109-01 National Taipei University of Technology (NTUT)
- NTUT-CGMH-110-01 and NTUT-CGMH-109-01 National Taipei University of Technology (NTUT)
- NTUT-CGMH-110-01 and NTUT-CGMH-109-01 National Taipei University of Technology (NTUT)
- NTUT-CGMH-110-01 and NTUT-CGMH-109-01 National Taipei University of Technology (NTUT)
- NTUT-CGMH-110-01 and NTUT-CGMH-109-01 National Taipei University of Technology (NTUT)
- NTUT-CGMH-110-01 and NTUT-CGMH-109-01 National Taipei University of Technology (NTUT)
- MOST 109-2314-B-182A-019-MY3 Ministry of Science and Technology, Taiwan (Ministry of Science and Technology of Taiwan)
- MOST 109-2314-B-182A-019-MY3 Ministry of Science and Technology, Taiwan (Ministry of Science and Technology of Taiwan)
- MOST 109-2314-B-182A-019-MY3 Ministry of Science and Technology, Taiwan (Ministry of Science and Technology of Taiwan)
- MOST 109-2314-B-182A-019-MY3 Ministry of Science and Technology, Taiwan (Ministry of Science and Technology of Taiwan)
- MOST 109-2314-B-182A-019-MY3 Ministry of Science and Technology, Taiwan (Ministry of Science and Technology of Taiwan)
- MOST 109-2314-B-182A-019-MY3 Ministry of Science and Technology, Taiwan (Ministry of Science and Technology of Taiwan)
- CORPG3L0131, CMRPG3M0131~2, and CMRPG3L0151~3 Chang Gung Memorial Hospital, Linkou (Linkou Chang Gung Memorial Hospital)
- CORPG3L0131, CMRPG3M0131~2, and CMRPG3L0151~3 Chang Gung Memorial Hospital, Linkou (Linkou Chang Gung Memorial Hospital)
- CORPG3L0131, CMRPG3M0131~2, and CMRPG3L0151~3 Chang Gung Memorial Hospital, Linkou (Linkou Chang Gung Memorial Hospital)
- CORPG3L0131, CMRPG3M0131~2, and CMRPG3L0151~3 Chang Gung Memorial Hospital, Linkou (Linkou Chang Gung Memorial Hospital)
Collapse
Affiliation(s)
- Yo-Ping Huang
- Department of Electrical Engineering, National Penghu University of Science and Technology, Penghu, 88046, Taiwan.
- Department of Electrical Engineering, National Taipei University of Technology, Taipei, 10608, Taiwan.
- Department of Information and Communication Engineering, Chaoyang University of Technology, Taichung, 41349, Taiwan.
| | - Spandana Vadloori
- Department of Electrical Engineering, National Penghu University of Science and Technology, Penghu, 88046, Taiwan
| | - Eugene Yu-Chuan Kang
- Department of Ophthalmology, Chang Gung Memorial Hospital, Linkou, 33305, Taiwan
- College of Medicine, Chang Gung University, Taoyuan, 33305, Taiwan
| | - Yoko Fukushima
- Department of Ophthalmology, Osaka University, Osaka, 565-0871, Japan
| | - Rie Takahashi
- Department of Ophthalmology, Fukuoka University, Fukuoka, 814-0180, Japan
| | - Wei-Chi Wu
- Department of Ophthalmology, Chang Gung Memorial Hospital, Linkou, 33305, Taiwan.
- College of Medicine, Chang Gung University, Taoyuan, 33305, Taiwan.
| |
Collapse
|
2
|
Coyner AS, Young BK, Ostmo SR, Grigorian F, Ells A, Hubbard B, Rodriguez SH, Rishi P, Miller AM, Bhatt AR, Agarwal-Sinha S, Sears J, Chan RVP, Chiang MF, Kalpathy-Cramer J, Binenbaum G, Campbell JP. Use of an Artificial Intelligence-Generated Vascular Severity Score Improved Plus Disease Diagnosis in Retinopathy of Prematurity. Ophthalmology 2024:S0161-6420(24)00339-7. [PMID: 38866367 DOI: 10.1016/j.ophtha.2024.06.006] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/12/2024] [Revised: 06/04/2024] [Accepted: 06/04/2024] [Indexed: 06/14/2024] Open
Abstract
PURPOSE To evaluate whether providing clinicians with an artificial intelligence (AI)-based vascular severity score (VSS) improves consistency in the diagnosis of plus disease in retinopathy of prematurity (ROP). DESIGN Multireader diagnostic accuracy imaging study. PARTICIPANTS Eleven ROP experts, 9 of whom had been in practice for 10 years or more. METHODS RetCam (Natus Medical Incorporated) fundus images were obtained from premature infants during routine ROP screening as part of the Imaging and Informatics in ROP study between January 2012 and July 2020. From all available examinations, a subset of 150 eye examinations from 110 infants were selected for grading. An AI-based VSS was assigned to each set of images using the i-ROP DL system (Siloam Vision). The clinicians were asked to diagnose plus disease for each examination and to assign an estimated VSS (range, 1-9) at baseline, and then again 1 month later with AI-based VSS assistance. A reference standard diagnosis (RSD) was assigned to each eye examination from the Imaging and Informatics in ROP study based on 3 masked expert labels and the ophthalmoscopic diagnosis. MAIN OUTCOME MEASURES Mean linearly weighted κ value for plus disease diagnosis compared with RSD. Area under the receiver operating characteristic curve (AUC) and area under the precision-recall curve (AUPR) for labels 1 through 9 compared with RSD for plus disease. RESULTS Expert agreement improved significantly, from substantial (κ value, 0.69 [0.59, 0.75]) to near perfect (κ value, 0.81 [0.71, 0.86]), when AI-based VSS was integrated. Additionally, a significant improvement in plus disease discrimination was achieved as measured by mean AUC (from 0.94 [95% confidence interval (CI), 0.92-0.96] to 0.98 [95% CI, 0.96-0.99]; difference, 0.04 [95% CI, 0.01-0.06]) and AUPR (from 0.86 [95% CI, 0.81-0.90] to 0.95 [95% CI, 0.91-0.97]; difference, 0.09 [95% CI, 0.03-0.14]). CONCLUSIONS Providing ROP clinicians with an AI-based measurement of vascular severity in ROP was associated with both improved plus disease diagnosis and improved continuous severity labeling as compared with an RSD for plus disease. If implemented in practice, AI-based VSS could reduce interobserver variability and could standardize treatment for infants with ROP. FINANCIAL DISCLOSURE(S) Proprietary or commercial disclosure may be found in the Footnotes and Disclosures at the end of this article.
Collapse
Affiliation(s)
- Aaron S Coyner
- Casey Eye Institute, Oregon Health & Science University, Portland, Oregon
| | - Benjamin K Young
- Casey Eye Institute, Oregon Health & Science University, Portland, Oregon
| | - Susan R Ostmo
- Casey Eye Institute, Oregon Health & Science University, Portland, Oregon
| | - Florin Grigorian
- Arkansas Children's Hospital, University of Arkansas for Medical Sciences, Little Rock, Arkansas
| | - Anna Ells
- Calgary Retina Consultants, University of Calgary, Calgary, Alberta, Canada
| | - Baker Hubbard
- Emory Eye Center, Emory University School of Medicine, Atlanta, Georgia
| | - Sarah H Rodriguez
- Department of Ophthalmology and Visual Science, University of Chicago, Chicago, Illinois
| | - Pukhraj Rishi
- Truhlsen Eye Institute, University of Nebraska Medical Centre, Omaha, Nebraska
| | - Aaron M Miller
- Department of Ophthalmology, Blanton Eye Institute, Houston Methodist Hospital, Houston, Texas
| | - Amit R Bhatt
- Department of Ophthalmology, Texas Children's Hospital, Houston, Texas
| | | | - Jonathan Sears
- Cole Eye Institute, The Cleveland Clinic, Cleveland, Ohio
| | - R V Paul Chan
- Illinois Eye and Ear Infirmary, University of Illinois at Chicago, Chicago, Illinois
| | - Michael F Chiang
- National Eye Institute, National Institutes of Health, Bethesda, Maryland
| | - Jayashree Kalpathy-Cramer
- National Eye Institute, National Institutes of Health, Bethesda, Maryland; Department of Ophthalmology, University of Colorado School of Medicine, Aurora, Colorado
| | - Gil Binenbaum
- Children's Hospital of Philadelphia, Philadelphia, Pennsylvania
| | - J Peter Campbell
- Casey Eye Institute, Oregon Health & Science University, Portland, Oregon.
| |
Collapse
|
3
|
Binenbaum G, Stahl A, Coyner AS, He J, Ying GS, Ostmo S, Chan RVP, Toth C, Vinekar A, Campbell JP. P Score: A Reference Image-Based Clinical Grading Scale for Vascular Change in Retinopathy of Prematurity. Ophthalmology 2024:S0161-6420(24)00314-2. [PMID: 38795976 DOI: 10.1016/j.ophtha.2024.05.019] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/12/2024] [Revised: 05/16/2024] [Accepted: 05/17/2024] [Indexed: 05/28/2024] Open
Abstract
PURPOSE The International Classification of Retinopathy of Prematurity, Third Edition (ICROP3), acknowledged that plus-like retinopathy of prematurity (ROP) vascular changes occurs along a spectrum. Historically, clinician-experts demonstrate variable agreement for plus diagnosis. We developed a 9-photograph reference image set for grading plus-like changes and compared intergrader agreement of the set with standard grading with no plus, preplus, and plus disease. DESIGN Retinal photographic grading and expert consensus opinion. PARTICIPANTS The development set included 34 international ICROP3 committee members. The validation set included 30 ophthalmologists with ROP expertise (15 ICROP3 committee members and 15 non-ICROP3 members) METHODS: Nine ROP fundus images (P1 through P9) representing increasing degrees of zone I vascular tortuosity and dilation, based on the 34 ICROP3 committee members' gradings and consensus image reviews, were used to establish standard photographs for the plus (P) score. Study participants graded 150 fundus photographs 2 ways, separated by a 1-week washout period: (1) no plus, preplus, or plus disease and (2) choosing the closest P score image. MAIN OUTCOME MEASURES Intergrader agreement measured by intraclass correlation coefficient. RESULTS Intergrader agreement was higher using the P score (intraclass correlation coefficient, 0.75; 95% confidence interval, 0.71-0.79) than no plus, preplus, or plus disease (intraclass correlation coefficient, 0.67; 95% confidence interval, 0.62-0.72). Mean ± standard deviation P scores for images with mode gradings of no plus, preplus, and plus disease were 2.5 ± 0.7, 4.8 ± 0.8, and 7.4 ± 0.8, respectively. CONCLUSIONS Intergrader agreement of plus-like vascular change in ROP using the P score is high. We now incorporate this 9-image reference set into ICROP3 for use in clinician daily practice alongside zone, stage, and plus assessment. P score is not yet meant to replace plus diagnosis for treatment decisions, but its use at our institutions has permitted better comparison between examinations for progression and regression, communication between examiners, and documentation of vascular change without fundus imaging. P score also could provide more detailed ROP classification for clinical trials, consistent with the spectrum of plus-like change that is now formally part of the International Classification of Retinopathy of Prematurity. FINANCIAL DISCLOSURE(S) Proprietary or commercial disclosure may be found in the Footnotes and Disclosures at the end of this article.
Collapse
Affiliation(s)
- Gil Binenbaum
- Division of Ophthalmology, Children's Hospital of Philadelphia, Philadelphia, Pennsylvania; Scheie Eye Institute, University of Pennsylvania, Philadelphia, Pennsylvania.
| | - Andreas Stahl
- Department of Ophthalmology, University Medicine Greifswald, Greifswald, Germany
| | - Aaron S Coyner
- Casey Eye Institute, Oregon Health and Science University, Portland, Oregon
| | - Jocelyn He
- Scheie Eye Institute, University of Pennsylvania, Philadelphia, Pennsylvania
| | - Gui-Shuang Ying
- Scheie Eye Institute, University of Pennsylvania, Philadelphia, Pennsylvania
| | - Susan Ostmo
- Casey Eye Institute, Oregon Health and Science University, Portland, Oregon
| | - R V Paul Chan
- Department of Ophthalmology and Visual Sciences, Illinois Eye and Ear Infirmary, University of Illinois at Chicago, Chicago, Illinois
| | - Cynthia Toth
- Department of Ophthalmology, Duke University, Durham North Carolina
| | - Anand Vinekar
- Department of Pediatric Retina, Narayana Nethralaya Eye Institute, Bangalore, Karnataka, India
| | - J Peter Campbell
- Casey Eye Institute, Oregon Health and Science University, Portland, Oregon
| |
Collapse
|
4
|
Hanif AM, Jian Y, Young BK, Campbell JP. Implementation of optical coherence tomography in retinopathy of prematurity screening. Curr Opin Ophthalmol 2024; 35:252-259. [PMID: 38205941 PMCID: PMC11034813 DOI: 10.1097/icu.0000000000001030] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/12/2024]
Abstract
PURPOSE OF REVIEW In this review, we explore the investigational applications of optical coherence tomography (OCT) in retinopathy of prematurity (ROP), the insights they have delivered thus far, and key milestones for its integration into the standard of care. RECENT FINDINGS While OCT has been widely integrated into clinical management of common retinal diseases, its use in pediatric contexts has been undermined by limitations in ergonomics, image acquisition time, and field of view. Recently, investigational handheld OCT devices have been reported with advancements including ultra-widefield view, noncontact use, and high-speed image capture permitting real-time en face visualization. These developments are compelling for OCT as a more objective alternative with reduced neonatal stress compared to indirect ophthalmoscopy and/or fundus photography as a means of classifying and monitoring ROP. SUMMARY OCT may become a viable modality in management of ROP. Ongoing innovation surrounding handheld devices should aim to optimize patient comfort and image resolution in the retinal periphery. Future clinical investigations may seek to objectively characterize features of peripheral stage and explore novel biomarkers of disease activity.
Collapse
Affiliation(s)
- Adam M. Hanif
- Ophthalmology, Oregon Health & Science University, Portland, Oregon
| | - Yifan Jian
- Ophthalmology, Oregon Health & Science University, Portland, Oregon
| | | | | |
Collapse
|
5
|
Silverman RH, Urs R, Horowitz JD, Coki O, Pinto L. Ocular blood flow in preterm neonates. Sci Rep 2024; 14:7722. [PMID: 38565630 PMCID: PMC10987658 DOI: 10.1038/s41598-024-58523-8] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/15/2023] [Accepted: 03/31/2024] [Indexed: 04/04/2024] Open
Abstract
Retinopathy of prematurity (ROP) is a disorder affecting low birthweight, preterm neonates. In the preterm eye, the retina is not fully developed and neovascularization may occur at the margin between the developed vascular retina and undeveloped avascular retina. Without timely treatment by laser or intravitreal anti-vascular endothelial growth factor (VEGF) therapy, this can lead to tractional retinal detachment and blindness. Visualization of the retina in regular examinations by indirect ophthalmoscopy is hence the current standard of care, but the exams are stressful and interpretation of images is subjective. The upregulation of VEGF in ROP would suggest an increase in ocular blood flow. In this report, we evaluate the potential of ultrafast plane-wave Doppler ultrasound (PWU) to detect increased flow velocities in the orbital vessels supplying the eye in a gentle exam with objective findings. We imaged both eyes of 50 low-birthweight preterm neonates using 18 MHz PWU. Flow velocity in the central retinal artery (CRA) and vein (CRV), and the short posterior ciliary arteries were determined and values at each ROP Stage compared. We found significantly increased velocities in the CRA and CRV in Stage 3 ROP eyes, where intervention would be considered. We compared multivariate models for identifying Stage 3 eyes comprised solely of clinical factors, solely of Doppler parameters, and clinical plus Doppler parameters. The respective models provided areas under their respective ROC curves of 0.760, 0.812, and 0.904. PWU Doppler represents a gentle, objective means for identifying neonates at risk for ROP that could complement ophthalmoscopy.
Collapse
Affiliation(s)
- Ronald H Silverman
- Department of Ophthalmology, Columbia University Irving Medical Center, New York, NY, USA.
- Department of Ophthalmology, Columbia University Irving Medical Center, 701 West 168th St., Room 609B, New York, NY, 10032, USA.
| | - Raksha Urs
- Department of Ophthalmology, Columbia University Irving Medical Center, New York, NY, USA
| | - Jason D Horowitz
- Department of Ophthalmology, Columbia University Irving Medical Center, New York, NY, USA
| | - Osode Coki
- Department of Ophthalmology, Columbia University Irving Medical Center, New York, NY, USA
| | - Leora Pinto
- Department of Ophthalmology, Columbia University Irving Medical Center, New York, NY, USA
| |
Collapse
|
6
|
Sharafi SM, Ebrahimiadib N, Roohipourmoallai R, Farahani AD, Fooladi MI, Khalili Pour E. Automated diagnosis of plus disease in retinopathy of prematurity using quantification of vessels characteristics. Sci Rep 2024; 14:6375. [PMID: 38493272 PMCID: PMC10944526 DOI: 10.1038/s41598-024-57072-4] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/27/2023] [Accepted: 03/14/2024] [Indexed: 03/18/2024] Open
Abstract
The condition known as Plus disease is distinguished by atypical alterations in the retinal vasculature of neonates born prematurely. It has been demonstrated that the diagnosis of Plus disease is subjective and qualitative in nature. The utilization of quantitative methods and computer-based image analysis to enhance the objectivity of Plus disease diagnosis has been extensively established in the literature. This study presents the development of a computer-based image analysis method aimed at automatically distinguishing Plus images from non-Plus images. The proposed methodology conducts a quantitative analysis of the vascular characteristics linked to Plus disease, thereby aiding physicians in making informed judgments. A collection of 76 posterior retinal images from a diverse group of infants who underwent screening for Retinopathy of Prematurity (ROP) was obtained. A reference standard diagnosis was established as the majority of the labeling performed by three experts in ROP during two separate sessions. The process of segmenting retinal vessels was carried out using a semi-automatic methodology. Computer algorithms were developed to compute the tortuosity, dilation, and density of vessels in various retinal regions as potential discriminative characteristics. A classifier was provided with a set of selected features in order to distinguish between Plus images and non-Plus images. This study included 76 infants (49 [64.5%] boys) with mean birth weight of 1305 ± 427 g and mean gestational age of 29.3 ± 3 weeks. The average level of agreement among experts for the diagnosis of plus disease was found to be 79% with a standard deviation of 5.3%. In terms of intra-expert agreement, the average was 85% with a standard deviation of 3%. Furthermore, the average tortuosity of the five most tortuous vessels was significantly higher in Plus images compared to non-Plus images (p ≤ 0.0001). The curvature values based on points were found to be significantly higher in Plus images compared to non-Plus images (p ≤ 0.0001). The maximum diameter of vessels within a region extending 5-disc diameters away from the border of the optic disc (referred to as 5DD) exhibited a statistically significant increase in Plus images compared to non-Plus images (p ≤ 0.0001). The density of vessels in Plus images was found to be significantly higher compared to non-Plus images (p ≤ 0.0001). The classifier's accuracy in distinguishing between Plus and non-Plus images, as determined through tenfold cross-validation, was found to be 0.86 ± 0.01. This accuracy was observed to be higher than the diagnostic accuracy of one out of three experts when compared to the reference standard. The implemented algorithm in the current study demonstrated a commendable level of accuracy in detecting Plus disease in cases of retinopathy of prematurity, exhibiting comparable performance to that of expert diagnoses. By engaging in an objective analysis of the characteristics of vessels, there exists the possibility of conducting a quantitative assessment of the disease progression's features. The utilization of this automated system has the potential to enhance physicians' ability to diagnose Plus disease, thereby offering valuable contributions to the management of ROP through the integration of traditional ophthalmoscopy and image-based telemedicine methodologies.
Collapse
Affiliation(s)
- Sayed Mehran Sharafi
- Retinopathy of Prematurity Department, Retina Ward, Farabi Eye Hospital, Tehran University of Medical Sciences, South Kargar Street, Qazvin Square, Tehran, Iran
| | - Nazanin Ebrahimiadib
- Ophthalmology Department, College of Medicine, University of Florida, Gainesville, FL, USA
| | - Ramak Roohipourmoallai
- Department of Ophthalmology, Morsani College of Medicine, University of South Florida, Tempa, FL, USA
| | - Afsar Dastjani Farahani
- Retinopathy of Prematurity Department, Retina Ward, Farabi Eye Hospital, Tehran University of Medical Sciences, South Kargar Street, Qazvin Square, Tehran, Iran
| | - Marjan Imani Fooladi
- Clinical Pediatric Ophthalmology Department, UPMC, Children's Hospital of Pittsburgh, Pittsburgh, PA, USA
| | - Elias Khalili Pour
- Retinopathy of Prematurity Department, Retina Ward, Farabi Eye Hospital, Tehran University of Medical Sciences, South Kargar Street, Qazvin Square, Tehran, Iran.
| |
Collapse
|
7
|
Hoebel KV, Bridge CP, Ahmed S, Akintola O, Chung C, Huang RY, Johnson JM, Kim A, Ly KI, Chang K, Patel J, Pinho M, Batchelor TT, Rosen BR, Gerstner ER, Kalpathy-Cramer J. Expert-centered Evaluation of Deep Learning Algorithms for Brain Tumor Segmentation. Radiol Artif Intell 2024; 6:e220231. [PMID: 38197800 PMCID: PMC10831514 DOI: 10.1148/ryai.220231] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/31/2022] [Revised: 09/13/2023] [Accepted: 11/01/2023] [Indexed: 01/11/2024]
Abstract
Purpose To present results from a literature survey on practices in deep learning segmentation algorithm evaluation and perform a study on expert quality perception of brain tumor segmentation. Materials and Methods A total of 180 articles reporting on brain tumor segmentation algorithms were surveyed for the reported quality evaluation. Additionally, ratings of segmentation quality on a four-point scale were collected from medical professionals for 60 brain tumor segmentation cases. Results Of the surveyed articles, Dice score, sensitivity, and Hausdorff distance were the most popular metrics to report segmentation performance. Notably, only 2.8% of the articles included clinical experts' evaluation of segmentation quality. The experimental results revealed a low interrater agreement (Krippendorff α, 0.34) in experts' segmentation quality perception. Furthermore, the correlations between the ratings and commonly used quantitative quality metrics were low (Kendall tau between Dice score and mean rating, 0.23; Kendall tau between Hausdorff distance and mean rating, 0.51), with large variability among the experts. Conclusion The results demonstrate that quality ratings are prone to variability due to the ambiguity of tumor boundaries and individual perceptual differences, and existing metrics do not capture the clinical perception of segmentation quality. Keywords: Brain Tumor Segmentation, Deep Learning Algorithms, Glioblastoma, Cancer, Machine Learning Clinical trial registration nos. NCT00756106 and NCT00662506 Supplemental material is available for this article. © RSNA, 2023.
Collapse
Affiliation(s)
- Katharina V. Hoebel
- From the Athinoula A. Martinos Center for Biomedical Imaging, Department of Radiology (K.V.H., C.P.B., A.K., K.I.L., K.C., J.P., B.R.R., E.R.G., J.K.C.), and Stephen E. and Catherine Pappas Center for Neuro-Oncology (O.A., A.K., K.I.L., E.R.G.), Massachusetts General Hospital, 149 13th St, Charlestown, MA 02129; Harvard-MIT Division of Health Sciences and Technology, Massachusetts Institute of Technology, Cambridge, Mass (K.V.H., K.C., J.P.); MGH and BWH Center for Clinical Data Science, Boston, Mass (C.P.B., J.K.C.); Department of Radiation Oncology, Division of Radiation Oncology (S.A., C.C.); Department of Diagnostic Radiology, Division of Diagnostic Imaging (C.C.), and Department of Neuroradiology (J.M.J.), Division of Diagnostic Imaging, The University of Texas MD Anderson Cancer Center, Houston, Tex; Departments of Radiology (R.Y.H.) and Neurology (T.T.B.), Brigham and Women’s Hospital, Boston, Mass; Department of Radiology and Advanced Imaging Research Center, University of Texas Southwestern Medical Center, Dallas, Tex (M.P.); and Department of Ophthalmology, University of Colorado Anschutz Medical Campus, Aurora, Colo (J.K.C.)
| | - Christopher P. Bridge
- From the Athinoula A. Martinos Center for Biomedical Imaging, Department of Radiology (K.V.H., C.P.B., A.K., K.I.L., K.C., J.P., B.R.R., E.R.G., J.K.C.), and Stephen E. and Catherine Pappas Center for Neuro-Oncology (O.A., A.K., K.I.L., E.R.G.), Massachusetts General Hospital, 149 13th St, Charlestown, MA 02129; Harvard-MIT Division of Health Sciences and Technology, Massachusetts Institute of Technology, Cambridge, Mass (K.V.H., K.C., J.P.); MGH and BWH Center for Clinical Data Science, Boston, Mass (C.P.B., J.K.C.); Department of Radiation Oncology, Division of Radiation Oncology (S.A., C.C.); Department of Diagnostic Radiology, Division of Diagnostic Imaging (C.C.), and Department of Neuroradiology (J.M.J.), Division of Diagnostic Imaging, The University of Texas MD Anderson Cancer Center, Houston, Tex; Departments of Radiology (R.Y.H.) and Neurology (T.T.B.), Brigham and Women’s Hospital, Boston, Mass; Department of Radiology and Advanced Imaging Research Center, University of Texas Southwestern Medical Center, Dallas, Tex (M.P.); and Department of Ophthalmology, University of Colorado Anschutz Medical Campus, Aurora, Colo (J.K.C.)
| | - Sara Ahmed
- From the Athinoula A. Martinos Center for Biomedical Imaging, Department of Radiology (K.V.H., C.P.B., A.K., K.I.L., K.C., J.P., B.R.R., E.R.G., J.K.C.), and Stephen E. and Catherine Pappas Center for Neuro-Oncology (O.A., A.K., K.I.L., E.R.G.), Massachusetts General Hospital, 149 13th St, Charlestown, MA 02129; Harvard-MIT Division of Health Sciences and Technology, Massachusetts Institute of Technology, Cambridge, Mass (K.V.H., K.C., J.P.); MGH and BWH Center for Clinical Data Science, Boston, Mass (C.P.B., J.K.C.); Department of Radiation Oncology, Division of Radiation Oncology (S.A., C.C.); Department of Diagnostic Radiology, Division of Diagnostic Imaging (C.C.), and Department of Neuroradiology (J.M.J.), Division of Diagnostic Imaging, The University of Texas MD Anderson Cancer Center, Houston, Tex; Departments of Radiology (R.Y.H.) and Neurology (T.T.B.), Brigham and Women’s Hospital, Boston, Mass; Department of Radiology and Advanced Imaging Research Center, University of Texas Southwestern Medical Center, Dallas, Tex (M.P.); and Department of Ophthalmology, University of Colorado Anschutz Medical Campus, Aurora, Colo (J.K.C.)
| | - Oluwatosin Akintola
- From the Athinoula A. Martinos Center for Biomedical Imaging, Department of Radiology (K.V.H., C.P.B., A.K., K.I.L., K.C., J.P., B.R.R., E.R.G., J.K.C.), and Stephen E. and Catherine Pappas Center for Neuro-Oncology (O.A., A.K., K.I.L., E.R.G.), Massachusetts General Hospital, 149 13th St, Charlestown, MA 02129; Harvard-MIT Division of Health Sciences and Technology, Massachusetts Institute of Technology, Cambridge, Mass (K.V.H., K.C., J.P.); MGH and BWH Center for Clinical Data Science, Boston, Mass (C.P.B., J.K.C.); Department of Radiation Oncology, Division of Radiation Oncology (S.A., C.C.); Department of Diagnostic Radiology, Division of Diagnostic Imaging (C.C.), and Department of Neuroradiology (J.M.J.), Division of Diagnostic Imaging, The University of Texas MD Anderson Cancer Center, Houston, Tex; Departments of Radiology (R.Y.H.) and Neurology (T.T.B.), Brigham and Women’s Hospital, Boston, Mass; Department of Radiology and Advanced Imaging Research Center, University of Texas Southwestern Medical Center, Dallas, Tex (M.P.); and Department of Ophthalmology, University of Colorado Anschutz Medical Campus, Aurora, Colo (J.K.C.)
| | - Caroline Chung
- From the Athinoula A. Martinos Center for Biomedical Imaging, Department of Radiology (K.V.H., C.P.B., A.K., K.I.L., K.C., J.P., B.R.R., E.R.G., J.K.C.), and Stephen E. and Catherine Pappas Center for Neuro-Oncology (O.A., A.K., K.I.L., E.R.G.), Massachusetts General Hospital, 149 13th St, Charlestown, MA 02129; Harvard-MIT Division of Health Sciences and Technology, Massachusetts Institute of Technology, Cambridge, Mass (K.V.H., K.C., J.P.); MGH and BWH Center for Clinical Data Science, Boston, Mass (C.P.B., J.K.C.); Department of Radiation Oncology, Division of Radiation Oncology (S.A., C.C.); Department of Diagnostic Radiology, Division of Diagnostic Imaging (C.C.), and Department of Neuroradiology (J.M.J.), Division of Diagnostic Imaging, The University of Texas MD Anderson Cancer Center, Houston, Tex; Departments of Radiology (R.Y.H.) and Neurology (T.T.B.), Brigham and Women’s Hospital, Boston, Mass; Department of Radiology and Advanced Imaging Research Center, University of Texas Southwestern Medical Center, Dallas, Tex (M.P.); and Department of Ophthalmology, University of Colorado Anschutz Medical Campus, Aurora, Colo (J.K.C.)
| | - Raymond Y. Huang
- From the Athinoula A. Martinos Center for Biomedical Imaging, Department of Radiology (K.V.H., C.P.B., A.K., K.I.L., K.C., J.P., B.R.R., E.R.G., J.K.C.), and Stephen E. and Catherine Pappas Center for Neuro-Oncology (O.A., A.K., K.I.L., E.R.G.), Massachusetts General Hospital, 149 13th St, Charlestown, MA 02129; Harvard-MIT Division of Health Sciences and Technology, Massachusetts Institute of Technology, Cambridge, Mass (K.V.H., K.C., J.P.); MGH and BWH Center for Clinical Data Science, Boston, Mass (C.P.B., J.K.C.); Department of Radiation Oncology, Division of Radiation Oncology (S.A., C.C.); Department of Diagnostic Radiology, Division of Diagnostic Imaging (C.C.), and Department of Neuroradiology (J.M.J.), Division of Diagnostic Imaging, The University of Texas MD Anderson Cancer Center, Houston, Tex; Departments of Radiology (R.Y.H.) and Neurology (T.T.B.), Brigham and Women’s Hospital, Boston, Mass; Department of Radiology and Advanced Imaging Research Center, University of Texas Southwestern Medical Center, Dallas, Tex (M.P.); and Department of Ophthalmology, University of Colorado Anschutz Medical Campus, Aurora, Colo (J.K.C.)
| | - Jason M. Johnson
- From the Athinoula A. Martinos Center for Biomedical Imaging, Department of Radiology (K.V.H., C.P.B., A.K., K.I.L., K.C., J.P., B.R.R., E.R.G., J.K.C.), and Stephen E. and Catherine Pappas Center for Neuro-Oncology (O.A., A.K., K.I.L., E.R.G.), Massachusetts General Hospital, 149 13th St, Charlestown, MA 02129; Harvard-MIT Division of Health Sciences and Technology, Massachusetts Institute of Technology, Cambridge, Mass (K.V.H., K.C., J.P.); MGH and BWH Center for Clinical Data Science, Boston, Mass (C.P.B., J.K.C.); Department of Radiation Oncology, Division of Radiation Oncology (S.A., C.C.); Department of Diagnostic Radiology, Division of Diagnostic Imaging (C.C.), and Department of Neuroradiology (J.M.J.), Division of Diagnostic Imaging, The University of Texas MD Anderson Cancer Center, Houston, Tex; Departments of Radiology (R.Y.H.) and Neurology (T.T.B.), Brigham and Women’s Hospital, Boston, Mass; Department of Radiology and Advanced Imaging Research Center, University of Texas Southwestern Medical Center, Dallas, Tex (M.P.); and Department of Ophthalmology, University of Colorado Anschutz Medical Campus, Aurora, Colo (J.K.C.)
| | - Albert Kim
- From the Athinoula A. Martinos Center for Biomedical Imaging, Department of Radiology (K.V.H., C.P.B., A.K., K.I.L., K.C., J.P., B.R.R., E.R.G., J.K.C.), and Stephen E. and Catherine Pappas Center for Neuro-Oncology (O.A., A.K., K.I.L., E.R.G.), Massachusetts General Hospital, 149 13th St, Charlestown, MA 02129; Harvard-MIT Division of Health Sciences and Technology, Massachusetts Institute of Technology, Cambridge, Mass (K.V.H., K.C., J.P.); MGH and BWH Center for Clinical Data Science, Boston, Mass (C.P.B., J.K.C.); Department of Radiation Oncology, Division of Radiation Oncology (S.A., C.C.); Department of Diagnostic Radiology, Division of Diagnostic Imaging (C.C.), and Department of Neuroradiology (J.M.J.), Division of Diagnostic Imaging, The University of Texas MD Anderson Cancer Center, Houston, Tex; Departments of Radiology (R.Y.H.) and Neurology (T.T.B.), Brigham and Women’s Hospital, Boston, Mass; Department of Radiology and Advanced Imaging Research Center, University of Texas Southwestern Medical Center, Dallas, Tex (M.P.); and Department of Ophthalmology, University of Colorado Anschutz Medical Campus, Aurora, Colo (J.K.C.)
| | - K. Ina Ly
- From the Athinoula A. Martinos Center for Biomedical Imaging, Department of Radiology (K.V.H., C.P.B., A.K., K.I.L., K.C., J.P., B.R.R., E.R.G., J.K.C.), and Stephen E. and Catherine Pappas Center for Neuro-Oncology (O.A., A.K., K.I.L., E.R.G.), Massachusetts General Hospital, 149 13th St, Charlestown, MA 02129; Harvard-MIT Division of Health Sciences and Technology, Massachusetts Institute of Technology, Cambridge, Mass (K.V.H., K.C., J.P.); MGH and BWH Center for Clinical Data Science, Boston, Mass (C.P.B., J.K.C.); Department of Radiation Oncology, Division of Radiation Oncology (S.A., C.C.); Department of Diagnostic Radiology, Division of Diagnostic Imaging (C.C.), and Department of Neuroradiology (J.M.J.), Division of Diagnostic Imaging, The University of Texas MD Anderson Cancer Center, Houston, Tex; Departments of Radiology (R.Y.H.) and Neurology (T.T.B.), Brigham and Women’s Hospital, Boston, Mass; Department of Radiology and Advanced Imaging Research Center, University of Texas Southwestern Medical Center, Dallas, Tex (M.P.); and Department of Ophthalmology, University of Colorado Anschutz Medical Campus, Aurora, Colo (J.K.C.)
| | - Ken Chang
- From the Athinoula A. Martinos Center for Biomedical Imaging, Department of Radiology (K.V.H., C.P.B., A.K., K.I.L., K.C., J.P., B.R.R., E.R.G., J.K.C.), and Stephen E. and Catherine Pappas Center for Neuro-Oncology (O.A., A.K., K.I.L., E.R.G.), Massachusetts General Hospital, 149 13th St, Charlestown, MA 02129; Harvard-MIT Division of Health Sciences and Technology, Massachusetts Institute of Technology, Cambridge, Mass (K.V.H., K.C., J.P.); MGH and BWH Center for Clinical Data Science, Boston, Mass (C.P.B., J.K.C.); Department of Radiation Oncology, Division of Radiation Oncology (S.A., C.C.); Department of Diagnostic Radiology, Division of Diagnostic Imaging (C.C.), and Department of Neuroradiology (J.M.J.), Division of Diagnostic Imaging, The University of Texas MD Anderson Cancer Center, Houston, Tex; Departments of Radiology (R.Y.H.) and Neurology (T.T.B.), Brigham and Women’s Hospital, Boston, Mass; Department of Radiology and Advanced Imaging Research Center, University of Texas Southwestern Medical Center, Dallas, Tex (M.P.); and Department of Ophthalmology, University of Colorado Anschutz Medical Campus, Aurora, Colo (J.K.C.)
| | - Jay Patel
- From the Athinoula A. Martinos Center for Biomedical Imaging, Department of Radiology (K.V.H., C.P.B., A.K., K.I.L., K.C., J.P., B.R.R., E.R.G., J.K.C.), and Stephen E. and Catherine Pappas Center for Neuro-Oncology (O.A., A.K., K.I.L., E.R.G.), Massachusetts General Hospital, 149 13th St, Charlestown, MA 02129; Harvard-MIT Division of Health Sciences and Technology, Massachusetts Institute of Technology, Cambridge, Mass (K.V.H., K.C., J.P.); MGH and BWH Center for Clinical Data Science, Boston, Mass (C.P.B., J.K.C.); Department of Radiation Oncology, Division of Radiation Oncology (S.A., C.C.); Department of Diagnostic Radiology, Division of Diagnostic Imaging (C.C.), and Department of Neuroradiology (J.M.J.), Division of Diagnostic Imaging, The University of Texas MD Anderson Cancer Center, Houston, Tex; Departments of Radiology (R.Y.H.) and Neurology (T.T.B.), Brigham and Women’s Hospital, Boston, Mass; Department of Radiology and Advanced Imaging Research Center, University of Texas Southwestern Medical Center, Dallas, Tex (M.P.); and Department of Ophthalmology, University of Colorado Anschutz Medical Campus, Aurora, Colo (J.K.C.)
| | - Marco Pinho
- From the Athinoula A. Martinos Center for Biomedical Imaging, Department of Radiology (K.V.H., C.P.B., A.K., K.I.L., K.C., J.P., B.R.R., E.R.G., J.K.C.), and Stephen E. and Catherine Pappas Center for Neuro-Oncology (O.A., A.K., K.I.L., E.R.G.), Massachusetts General Hospital, 149 13th St, Charlestown, MA 02129; Harvard-MIT Division of Health Sciences and Technology, Massachusetts Institute of Technology, Cambridge, Mass (K.V.H., K.C., J.P.); MGH and BWH Center for Clinical Data Science, Boston, Mass (C.P.B., J.K.C.); Department of Radiation Oncology, Division of Radiation Oncology (S.A., C.C.); Department of Diagnostic Radiology, Division of Diagnostic Imaging (C.C.), and Department of Neuroradiology (J.M.J.), Division of Diagnostic Imaging, The University of Texas MD Anderson Cancer Center, Houston, Tex; Departments of Radiology (R.Y.H.) and Neurology (T.T.B.), Brigham and Women’s Hospital, Boston, Mass; Department of Radiology and Advanced Imaging Research Center, University of Texas Southwestern Medical Center, Dallas, Tex (M.P.); and Department of Ophthalmology, University of Colorado Anschutz Medical Campus, Aurora, Colo (J.K.C.)
| | - Tracy T. Batchelor
- From the Athinoula A. Martinos Center for Biomedical Imaging, Department of Radiology (K.V.H., C.P.B., A.K., K.I.L., K.C., J.P., B.R.R., E.R.G., J.K.C.), and Stephen E. and Catherine Pappas Center for Neuro-Oncology (O.A., A.K., K.I.L., E.R.G.), Massachusetts General Hospital, 149 13th St, Charlestown, MA 02129; Harvard-MIT Division of Health Sciences and Technology, Massachusetts Institute of Technology, Cambridge, Mass (K.V.H., K.C., J.P.); MGH and BWH Center for Clinical Data Science, Boston, Mass (C.P.B., J.K.C.); Department of Radiation Oncology, Division of Radiation Oncology (S.A., C.C.); Department of Diagnostic Radiology, Division of Diagnostic Imaging (C.C.), and Department of Neuroradiology (J.M.J.), Division of Diagnostic Imaging, The University of Texas MD Anderson Cancer Center, Houston, Tex; Departments of Radiology (R.Y.H.) and Neurology (T.T.B.), Brigham and Women’s Hospital, Boston, Mass; Department of Radiology and Advanced Imaging Research Center, University of Texas Southwestern Medical Center, Dallas, Tex (M.P.); and Department of Ophthalmology, University of Colorado Anschutz Medical Campus, Aurora, Colo (J.K.C.)
| | - Bruce R. Rosen
- From the Athinoula A. Martinos Center for Biomedical Imaging, Department of Radiology (K.V.H., C.P.B., A.K., K.I.L., K.C., J.P., B.R.R., E.R.G., J.K.C.), and Stephen E. and Catherine Pappas Center for Neuro-Oncology (O.A., A.K., K.I.L., E.R.G.), Massachusetts General Hospital, 149 13th St, Charlestown, MA 02129; Harvard-MIT Division of Health Sciences and Technology, Massachusetts Institute of Technology, Cambridge, Mass (K.V.H., K.C., J.P.); MGH and BWH Center for Clinical Data Science, Boston, Mass (C.P.B., J.K.C.); Department of Radiation Oncology, Division of Radiation Oncology (S.A., C.C.); Department of Diagnostic Radiology, Division of Diagnostic Imaging (C.C.), and Department of Neuroradiology (J.M.J.), Division of Diagnostic Imaging, The University of Texas MD Anderson Cancer Center, Houston, Tex; Departments of Radiology (R.Y.H.) and Neurology (T.T.B.), Brigham and Women’s Hospital, Boston, Mass; Department of Radiology and Advanced Imaging Research Center, University of Texas Southwestern Medical Center, Dallas, Tex (M.P.); and Department of Ophthalmology, University of Colorado Anschutz Medical Campus, Aurora, Colo (J.K.C.)
| | - Elizabeth R. Gerstner
- From the Athinoula A. Martinos Center for Biomedical Imaging, Department of Radiology (K.V.H., C.P.B., A.K., K.I.L., K.C., J.P., B.R.R., E.R.G., J.K.C.), and Stephen E. and Catherine Pappas Center for Neuro-Oncology (O.A., A.K., K.I.L., E.R.G.), Massachusetts General Hospital, 149 13th St, Charlestown, MA 02129; Harvard-MIT Division of Health Sciences and Technology, Massachusetts Institute of Technology, Cambridge, Mass (K.V.H., K.C., J.P.); MGH and BWH Center for Clinical Data Science, Boston, Mass (C.P.B., J.K.C.); Department of Radiation Oncology, Division of Radiation Oncology (S.A., C.C.); Department of Diagnostic Radiology, Division of Diagnostic Imaging (C.C.), and Department of Neuroradiology (J.M.J.), Division of Diagnostic Imaging, The University of Texas MD Anderson Cancer Center, Houston, Tex; Departments of Radiology (R.Y.H.) and Neurology (T.T.B.), Brigham and Women’s Hospital, Boston, Mass; Department of Radiology and Advanced Imaging Research Center, University of Texas Southwestern Medical Center, Dallas, Tex (M.P.); and Department of Ophthalmology, University of Colorado Anschutz Medical Campus, Aurora, Colo (J.K.C.)
| | - Jayashree Kalpathy-Cramer
- From the Athinoula A. Martinos Center for Biomedical Imaging, Department of Radiology (K.V.H., C.P.B., A.K., K.I.L., K.C., J.P., B.R.R., E.R.G., J.K.C.), and Stephen E. and Catherine Pappas Center for Neuro-Oncology (O.A., A.K., K.I.L., E.R.G.), Massachusetts General Hospital, 149 13th St, Charlestown, MA 02129; Harvard-MIT Division of Health Sciences and Technology, Massachusetts Institute of Technology, Cambridge, Mass (K.V.H., K.C., J.P.); MGH and BWH Center for Clinical Data Science, Boston, Mass (C.P.B., J.K.C.); Department of Radiation Oncology, Division of Radiation Oncology (S.A., C.C.); Department of Diagnostic Radiology, Division of Diagnostic Imaging (C.C.), and Department of Neuroradiology (J.M.J.), Division of Diagnostic Imaging, The University of Texas MD Anderson Cancer Center, Houston, Tex; Departments of Radiology (R.Y.H.) and Neurology (T.T.B.), Brigham and Women’s Hospital, Boston, Mass; Department of Radiology and Advanced Imaging Research Center, University of Texas Southwestern Medical Center, Dallas, Tex (M.P.); and Department of Ophthalmology, University of Colorado Anschutz Medical Campus, Aurora, Colo (J.K.C.)
| |
Collapse
|
8
|
Mohr FH, Fischer HS, Czernik C, Müller B, Bührer C. Retinal blood flow velocities in infants with retinopathy of prematurity after intravitreal administration of bevacizumab. Eur J Ophthalmol 2024; 34:95-101. [PMID: 37218176 PMCID: PMC10757389 DOI: 10.1177/11206721231178062] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/30/2022] [Accepted: 05/08/2023] [Indexed: 05/24/2023]
Abstract
BACKGROUND/OBJECTIVES Progression of retinopathy of prematurity (ROP) is associated with increased retinal blood flow velocities. We investigated changes of central retinal arterial and venous blood flow after intravitreal administration of bevacizumab. SUBJECTS/METHODS Prospective observational study using serial ultrasound Doppler imaging in preterm infants with bevacizumab-treated ROP. Eyes were examined 1 [0-2] days before injection (median [interquartile range]), and at three time points after injection (1 [1-2] days, 6 [3-8] days, and 17 [9-28] days). Preterm infants with ROP stage 2 displaying spontaneous regression served as controls. RESULTS In 21 eyes of 12 infants with bevacizumab-treated ROP, peak arterial systolic velocity declined from 13.6 [11.0-16.3] cm/s prior to intravitreal bevacizumab to 11.2 [9.4-13.9] cm/s, 10.6 [9.2-13.3] cm/s and 9.3 [8.2-11.0] cm/s at discharge (p = .002). There was also a decline of the arterial velocity time integral (from 3.1 [2.3-3.9] cm to 2.9 [2.4-3.5], 2.7 [2.3-3.2] cm and 2.2 [2.0-2.7], p = .021) and mean velocity in the central retinal vein (from 4.5 [3.6-5.8] cm/s to 3.7 [2.6-4.1] cm/s, 3.5 [3.0-4.3] cm/s, and 3.2 [2.8-4.6] cm/s, p = .012). Arterial end-diastolic velocity and resistance index remained unchanged. Blood flow velocities in bevacizumab-treated eyes examined before injection were significantly higher than those measured in untreated eyes that ultimately showed spontaneous regression of ROP. Sequential examinations in these controls did not reveal any declines of retinal blood flow velocities. CONCLUSION Increased retinal arterial and venous blood flow velocities in infants with threshold ROP decline following intravitreal bevacizumab injection.
Collapse
Affiliation(s)
- Franziska H Mohr
- Department of Neonatology, Charité – Universitätsmedizin Berlin, Berlin, Germany
- Department of Pediatric Pulmonology, Immunology, and Intensive Care Medicine, Charité - Universitätsmedizin Berlin, Berlin, Germany
| | - Hendrik S Fischer
- Department of Neonatology, Charité – Universitätsmedizin Berlin, Berlin, Germany
| | - Christoph Czernik
- Department of Neonatology, Charité – Universitätsmedizin Berlin, Berlin, Germany
| | - Bert Müller
- Department of Ophthalmology, Charité – Universitätsmedizin Berlin, Berlin, Germany
| | - Christoph Bührer
- Department of Neonatology, Charité – Universitätsmedizin Berlin, Berlin, Germany
| |
Collapse
|
9
|
Corano Scheri K, Hsieh YW, Jeong E, Fawzi AA. Limited Hyperoxia-Induced Proliferative Retinopathy (LHIPR) as a Model of Retinal Fibrosis, Angiogenesis, and Inflammation. Cells 2023; 12:2468. [PMID: 37887312 PMCID: PMC10605514 DOI: 10.3390/cells12202468] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/26/2023] [Revised: 09/26/2023] [Accepted: 10/13/2023] [Indexed: 10/28/2023] Open
Abstract
The progression to fibrosis and traction in retinopathy of prematurity (ROP) and other ischemic retinopathies remains an important clinical and surgical challenge, necessitating a comprehensive understanding of its pathogenesis. Fibrosis is an unbalanced deposition of extracellular matrix components responsible for scar tissue formation with consequent tissue and organ impairment. Together with retinal traction, it is among the main causes of retinal detachment and vision loss. We capitalize on the Limited Hyperoxia Induced Retinopathy (LHIPR) model, as it reflects the more advanced pathological phenotypes seen in ROP and other ischemic retinopathies. To model LHIPR, we exposed wild-type C57Bl/6J mouse pups to 65% oxygen from P0 to P7. Then, the pups were returned to room air to recover until later endpoints. We performed histological and molecular analysis to evaluate fibrosis progression, angiogenesis, and inflammation at several time points, from 1.5 months to 9 months. In addition, we performed in vivo retinal imaging by optical coherence tomography (OCT) or OCT Angiography (OCTA) to follow the fibrovascular progression in vivo. Although the retinal morphology was relatively preserved, we found a progressive increase in preretinal fibrogenesis over time, up to 9 months of age. We also detected blood vessels in the preretinal space as well as an active inflammatory process, altogether mimicking advanced preretinal fibrovascular disease in humans.
Collapse
Affiliation(s)
| | | | | | - Amani A. Fawzi
- Department of Ophthalmology, Feinberg School of Medicine, Northwestern University, Chicago, IL 60611, USA; (K.C.S.); (Y.-W.H.); (E.J.)
| |
Collapse
|
10
|
Wagner SK, Liefers B, Radia M, Zhang G, Struyven R, Faes L, Than J, Balal S, Hennings C, Kilduff C, Pooprasert P, Glinton S, Arunakirinathan M, Giannakis P, Braimah IZ, Ahmed ISH, Al-Feky M, Khalid H, Ferraz D, Vieira J, Jorge R, Husain S, Ravelo J, Hinds AM, Henderson R, Patel HI, Ostmo S, Campbell JP, Pontikos N, Patel PJ, Keane PA, Adams G, Balaskas K. Development and international validation of custom-engineered and code-free deep-learning models for detection of plus disease in retinopathy of prematurity: a retrospective study. Lancet Digit Health 2023; 5:e340-e349. [PMID: 37088692 PMCID: PMC10279502 DOI: 10.1016/s2589-7500(23)00050-x] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/20/2022] [Revised: 01/08/2023] [Accepted: 02/14/2023] [Indexed: 04/25/2023]
Abstract
BACKGROUND Retinopathy of prematurity (ROP), a leading cause of childhood blindness, is diagnosed through interval screening by paediatric ophthalmologists. However, improved survival of premature neonates coupled with a scarcity of available experts has raised concerns about the sustainability of this approach. We aimed to develop bespoke and code-free deep learning-based classifiers for plus disease, a hallmark of ROP, in an ethnically diverse population in London, UK, and externally validate them in ethnically, geographically, and socioeconomically diverse populations in four countries and three continents. Code-free deep learning is not reliant on the availability of expertly trained data scientists, thus being of particular potential benefit for low resource health-care settings. METHODS This retrospective cohort study used retinal images from 1370 neonates admitted to a neonatal unit at Homerton University Hospital NHS Foundation Trust, London, UK, between 2008 and 2018. Images were acquired using a Retcam Version 2 device (Natus Medical, Pleasanton, CA, USA) on all babies who were either born at less than 32 weeks gestational age or had a birthweight of less than 1501 g. Each images was graded by two junior ophthalmologists with disagreements adjudicated by a senior paediatric ophthalmologist. Bespoke and code-free deep learning models (CFDL) were developed for the discrimination of healthy, pre-plus disease, and plus disease. Performance was assessed internally on 200 images with the majority vote of three senior paediatric ophthalmologists as the reference standard. External validation was on 338 retinal images from four separate datasets from the USA, Brazil, and Egypt with images derived from Retcam and the 3nethra neo device (Forus Health, Bengaluru, India). FINDINGS Of the 7414 retinal images in the original dataset, 6141 images were used in the final development dataset. For the discrimination of healthy versus pre-plus or plus disease, the bespoke model had an area under the curve (AUC) of 0·986 (95% CI 0·973-0·996) and the CFDL model had an AUC of 0·989 (0·979-0·997) on the internal test set. Both models generalised well to external validation test sets acquired using the Retcam for discriminating healthy from pre-plus or plus disease (bespoke range was 0·975-1·000 and CFDL range was 0·969-0·995). The CFDL model was inferior to the bespoke model on discriminating pre-plus disease from healthy or plus disease in the USA dataset (CFDL 0·808 [95% CI 0·671-0·909, bespoke 0·942 [0·892-0·982]], p=0·0070). Performance also reduced when tested on the 3nethra neo imaging device (CFDL 0·865 [0·742-0·965] and bespoke 0·891 [0·783-0·977]). INTERPRETATION Both bespoke and CFDL models conferred similar performance to senior paediatric ophthalmologists for discriminating healthy retinal images from ones with features of pre-plus or plus disease; however, CFDL models might generalise less well when considering minority classes. Care should be taken when testing on data acquired using alternative imaging devices from that used for the development dataset. Our study justifies further validation of plus disease classifiers in ROP screening and supports a potential role for code-free approaches to help prevent blindness in vulnerable neonates. FUNDING National Institute for Health Research Biomedical Research Centre based at Moorfields Eye Hospital NHS Foundation Trust and the University College London Institute of Ophthalmology. TRANSLATIONS For the Portuguese and Arabic translations of the abstract see Supplementary Materials section.
Collapse
Affiliation(s)
- Siegfried K Wagner
- NIHR Moorfields Biomedical Research Centre, London, UK; Institute of Ophthalmology, University College London, London, UK; Moorfields Eye Hospital NHS Foundation Trust, London, UK
| | - Bart Liefers
- NIHR Moorfields Biomedical Research Centre, London, UK
| | - Meera Radia
- Moorfields Eye Hospital NHS Foundation Trust, London, UK
| | - Gongyu Zhang
- NIHR Moorfields Biomedical Research Centre, London, UK
| | - Robbert Struyven
- NIHR Moorfields Biomedical Research Centre, London, UK; Institute of Ophthalmology, University College London, London, UK; Moorfields Eye Hospital NHS Foundation Trust, London, UK
| | - Livia Faes
- NIHR Moorfields Biomedical Research Centre, London, UK; Moorfields Eye Hospital NHS Foundation Trust, London, UK
| | - Jonathan Than
- Moorfields Eye Hospital NHS Foundation Trust, London, UK
| | - Shafi Balal
- Moorfields Eye Hospital NHS Foundation Trust, London, UK
| | | | | | | | | | | | - Periklis Giannakis
- Institute of Health Sciences Education, Queen Mary University of London, London, UK
| | - Imoro Zeba Braimah
- Lions International Eye Centre, Korle-Bu Teaching Hospital, Accra, Ghana
| | - Islam S H Ahmed
- Faculty of Medicine, Alexandria University, Alexandria, Egypt; Alexandria University Hospital, Alexandria, Egypt
| | - Mariam Al-Feky
- Department of Ophthalmology, Ain Shams University Hospitals, Cairo, Egypt; Watany Eye Hospital, Cairo, Egypt
| | - Hagar Khalid
- Moorfields Eye Hospital NHS Foundation Trust, London, UK; Department of Ophthalmology, Tanta University, Tanta, Egypt
| | - Daniel Ferraz
- Institute of Ophthalmology, University College London, London, UK; D'Or Institute for Research and Education, São Paulo, Brazil
| | - Juliana Vieira
- Department of Ophthalmology, Ribeirão Preto Medical School, University of São Paulo, Ribeirão Preto, Brazil
| | - Rodrigo Jorge
- Department of Ophthalmology, Ribeirão Preto Medical School, University of São Paulo, Ribeirão Preto, Brazil
| | - Shahid Husain
- The Blizard Institute, Queen Mary University of London, London, UK; Neonatology Department, Homerton University Hospital NHS Foundation Trust, London, UK
| | - Janette Ravelo
- Neonatology Department, Homerton University Hospital NHS Foundation Trust, London, UK
| | | | - Robert Henderson
- UCL Great Ormond Street Institute of Child Health, University College London, London, UK; Clinical and Academic Department of Ophthalmology, Great Ormond Street Hospital for Children, London, UK
| | - Himanshu I Patel
- Moorfields Eye Hospital NHS Foundation Trust, London, UK; The Royal London Hospital, Barts Health NHS Trust, London, UK
| | - Susan Ostmo
- Department of Ophthalmology, Oregon Health & Science University, Portland, OR, USA
| | - J Peter Campbell
- Department of Ophthalmology, Oregon Health & Science University, Portland, OR, USA
| | - Nikolas Pontikos
- NIHR Moorfields Biomedical Research Centre, London, UK; Institute of Ophthalmology, University College London, London, UK; Moorfields Eye Hospital NHS Foundation Trust, London, UK
| | - Praveen J Patel
- NIHR Moorfields Biomedical Research Centre, London, UK; Institute of Ophthalmology, University College London, London, UK; Moorfields Eye Hospital NHS Foundation Trust, London, UK
| | - Pearse A Keane
- NIHR Moorfields Biomedical Research Centre, London, UK; Institute of Ophthalmology, University College London, London, UK; Moorfields Eye Hospital NHS Foundation Trust, London, UK
| | - Gill Adams
- NIHR Moorfields Biomedical Research Centre, London, UK; Moorfields Eye Hospital NHS Foundation Trust, London, UK
| | - Konstantinos Balaskas
- NIHR Moorfields Biomedical Research Centre, London, UK; Institute of Ophthalmology, University College London, London, UK; Moorfields Eye Hospital NHS Foundation Trust, London, UK.
| |
Collapse
|
11
|
Cole ED, Park SH, Kim SJ, Kang KB, Valikodath NG, Al-Khaled T, Patel SN, Jonas KE, Ostmo S, Coyner A, Berrocal A, Drenser KA, Nagiel A, Horowitz JD, Lee TC, Kalpathy-Cramer J, Chiang MF, Campbell JP, Chan RVP. Variability in Plus Disease Diagnosis using Single and Serial Images. Ophthalmol Retina 2022; 6:1122-1129. [PMID: 35659941 DOI: 10.1016/j.oret.2022.05.024] [Citation(s) in RCA: 2] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/28/2022] [Revised: 05/21/2022] [Accepted: 05/23/2022] [Indexed: 01/06/2023]
Abstract
PURPOSE To assess changes in retinopathy of prematurity (ROP) diagnosis in single and serial retinal images. DESIGN Cohort study. PARTICIPANTS Cases of ROP recruited from the Imaging and Informatics in Retinopathy of Prematurity (i-ROP) consortium evaluated by 7 graders. METHODS Seven ophthalmologists reviewed both single and 3 consecutive serial retinal images from 15 cases with ROP, and severity was assigned as plus, preplus, or none. Imaging data were acquired during routine ROP screening from 2011 to 2015, and a reference standard diagnosis was established for each image. A secondary analysis was performed using the i-ROP deep learning system to assign a vascular severity score (VSS) to each image, ranging from 1 to 9, with 9 being the most severe disease. This score has been previously demonstrated to correlate with the International Classification of ROP. Mean plus disease severity was calculated by averaging 14 labels per image in serial and single images to decrease noise. MAIN OUTCOME MEASURES Grading severity of ROP as defined by plus, preplus, or no ROP. RESULTS Assessment of serial retinal images changed the grading severity for > 50% of the graders, although there was wide variability. Cohen's kappa ranged from 0.29 to 1.0, which showed a wide range of agreement from slight to perfect by each grader. Changes in the grading of serial retinal images were noted more commonly in cases of preplus disease. The mean severity in cases with a diagnosis of plus disease and no disease did not change between single and serial images. The ROP VSS demonstrated good correlation with the range of expert classifications of plus disease and overall agreement with the mode class (P = 0.001). The VSS correlated with mean plus disease severity by expert diagnosis (correlation coefficient, 0.89). The more aggressive graders tended to be influenced by serial images to increase the severity of their grading. The VSS also demonstrated agreement with disease progression across serial images, which progressed to preplus and plus disease. CONCLUSIONS Clinicians demonstrated variability in ROP diagnosis when presented with both single and serial images. The use of deep learning as a quantitative assessment of plus disease has the potential to standardize ROP diagnosis and treatment.
Collapse
Affiliation(s)
- Emily D Cole
- Department of Ophthalmology and Visual Sciences, Illinois Eye and Ear Infirmary, University of Illinois at Chicago, Chicago, Illinois
| | - Shin Hae Park
- Department of Ophthalmology and Visual Sciences, Illinois Eye and Ear Infirmary, University of Illinois at Chicago, Chicago, Illinois; Department of Ophthalmology, Seoul St. Mary's Hospital, College of Medicine, The Catholic University of Korea, Seoul, Republic of Korea
| | - Sang Jin Kim
- Department of Ophthalmology, Casey Eye Institute, Oregon Health & Science University, Portland, Oregon; Department of Ophthalmology, Samsung Medical Center, Sungkyunkwan University School of Medicine, Seoul, Republic of Korea
| | - Kai B Kang
- Department of Ophthalmology and Visual Sciences, Illinois Eye and Ear Infirmary, University of Illinois at Chicago, Chicago, Illinois
| | - Nita G Valikodath
- Department of Ophthalmology and Visual Sciences, Illinois Eye and Ear Infirmary, University of Illinois at Chicago, Chicago, Illinois
| | - Tala Al-Khaled
- Department of Ophthalmology and Visual Sciences, Illinois Eye and Ear Infirmary, University of Illinois at Chicago, Chicago, Illinois
| | | | - Karyn E Jonas
- Department of Ophthalmology and Visual Sciences, Illinois Eye and Ear Infirmary, University of Illinois at Chicago, Chicago, Illinois
| | - Susan Ostmo
- Department of Ophthalmology, Casey Eye Institute, Oregon Health & Science University, Portland, Oregon
| | - Aaron Coyner
- Department of Ophthalmology, Casey Eye Institute, Oregon Health & Science University, Portland, Oregon
| | - Audina Berrocal
- Bascom Palmer Eye Institute, University of Miami, Miami, Florida
| | - Kimberly A Drenser
- Department of Ophthalmology, Beaumont Eye Institute, Royal Oak, Michigan
| | - Aaron Nagiel
- Stein Eye Institute, University of California Los Angeles, Los Angeles, California
| | - Jason D Horowitz
- Department of Ophthalmology, Columbia University, New York, New York
| | - Thomas C Lee
- Roski Eye Institute, Department of Ophthalmology, Keck School of Medicine of the University of Southern California, Los Angeles, California
| | | | - Michael F Chiang
- National Eye Institute, National Institutes of Health, Bethesda, Maryland
| | - J Peter Campbell
- Department of Ophthalmology, Casey Eye Institute, Oregon Health & Science University, Portland, Oregon
| | - R V Paul Chan
- Department of Ophthalmology and Visual Sciences, Illinois Eye and Ear Infirmary, University of Illinois at Chicago, Chicago, Illinois.
| | | |
Collapse
|
12
|
Lemay A, Hoebel K, Bridge CP, Befano B, De Sanjosé S, Egemen D, Rodriguez AC, Schiffman M, Campbell JP, Kalpathy-Cramer J. Improving the repeatability of deep learning models with Monte Carlo dropout. NPJ Digit Med 2022; 5:174. [PMID: 36400939 PMCID: PMC9674698 DOI: 10.1038/s41746-022-00709-3] [Citation(s) in RCA: 9] [Impact Index Per Article: 4.5] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/14/2022] [Accepted: 10/10/2022] [Indexed: 11/19/2022] Open
Abstract
The integration of artificial intelligence into clinical workflows requires reliable and robust models. Repeatability is a key attribute of model robustness. Ideal repeatable models output predictions without variation during independent tests carried out under similar conditions. However, slight variations, though not ideal, may be unavoidable and acceptable in practice. During model development and evaluation, much attention is given to classification performance while model repeatability is rarely assessed, leading to the development of models that are unusable in clinical practice. In this work, we evaluate the repeatability of four model types (binary classification, multi-class classification, ordinal classification, and regression) on images that were acquired from the same patient during the same visit. We study the each model's performance on four medical image classification tasks from public and private datasets: knee osteoarthritis, cervical cancer screening, breast density estimation, and retinopathy of prematurity. Repeatability is measured and compared on ResNet and DenseNet architectures. Moreover, we assess the impact of sampling Monte Carlo dropout predictions at test time on classification performance and repeatability. Leveraging Monte Carlo predictions significantly increases repeatability, in particular at the class boundaries, for all tasks on the binary, multi-class, and ordinal models leading to an average reduction of the 95% limits of agreement by 16% points and of the class disagreement rate by 7% points. The classification accuracy improves in most settings along with the repeatability. Our results suggest that beyond about 20 Monte Carlo iterations, there is no further gain in repeatability. In addition to the higher test-retest agreement, Monte Carlo predictions are better calibrated which leads to output probabilities reflecting more accurately the true likelihood of being correctly classified.
Collapse
Affiliation(s)
- Andreanne Lemay
- Martinos Center for Biomedical Imaging, Boston, MA, USA
- NeuroPoly, Polytechnique Montreal, Montreal, QC, Canada
| | - Katharina Hoebel
- Martinos Center for Biomedical Imaging, Boston, MA, USA
- Massachusetts Institute of Technology, Cambridge, MA, USA
| | - Christopher P Bridge
- Martinos Center for Biomedical Imaging, Boston, MA, USA
- MGH & BWH Center for Clinical Data Science, Boston, MA, USA
| | - Brian Befano
- Department of Epidemiology, University of Washington School of Public Health, Seattle, WA, USA
| | - Silvia De Sanjosé
- Division of Cancer Epidemiology & Genetics, National Cancer Institute, Rockville, MD, USA
| | - Didem Egemen
- Division of Cancer Epidemiology & Genetics, National Cancer Institute, Rockville, MD, USA
| | - Ana Cecilia Rodriguez
- Division of Cancer Epidemiology & Genetics, National Cancer Institute, Rockville, MD, USA
| | - Mark Schiffman
- Division of Cancer Epidemiology & Genetics, National Cancer Institute, Rockville, MD, USA
| | | | | |
Collapse
|
13
|
Nguyen TTP, Ni S, Ostmo S, Rajagopalan A, Coyner AS, Woodward M, Chiang MF, Jia Y, Huang D, Campbell JP, Jian Y. Association of Optical Coherence Tomography-Measured Fibrovascular Ridge Thickness and Clinical Disease Stage in Retinopathy of Prematurity. JAMA Ophthalmol 2022; 140:2797385. [PMID: 36227622 PMCID: PMC9562098 DOI: 10.1001/jamaophthalmol.2022.4173] [Citation(s) in RCA: 10] [Impact Index Per Article: 5.0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/07/2022] [Accepted: 08/18/2022] [Indexed: 11/30/2022]
Abstract
Importance Accurate diagnosis of retinopathy of prematurity (ROP) is essential to provide timely treatment and reduce the risk of blindness. However, the components of an ROP examination are subjective and qualitative. Objective To evaluate whether optical coherence tomography (OCT)-derived retinal thickness measurements at the vascular-avascular junction are associated with clinical diagnosis of ROP stage. Design, Setting, and Participants This cross-sectional longitudinal study compared OCT-based ridge thickness calculated from OCT B-scans by a masked examiner to the clinical diagnosis of 2 masked examiners using both traditional stage classifications and a more granular continuous scale at the neonatal intensive care unit (NICU) of Oregon Health & Science University (OHSU) Hospital. Infants who met ROP screening criteria in the OHSU NICU between June 2021 and April 2022 and had guardian consent were included. One OCT volume and en face image per patient per eye showing at least 1 to 2 clock hours of ridge were included in the final analysis. Main Outcomes and Measures Comparison of OCT-derived ridge thickness to the clinical diagnosis of ROP stage using an ordinal and continuous scale. Repeatability was assessed using 20 repeated examinations from the same visit and compared using intraclass correlation coefficient (ICC) and coefficient of variation (CV). Comparison of ridge thickness with ordinal categories was performed using generalized estimating equations and with continuous stage using Spearman correlation. Results A total of 128 separate OCT eye examinations from 50 eyes of 25 patients were analyzed. The ICC was 0.87 with a CV of 7.0%. Higher ordinal disease classification was associated with higher axial ridge thickness on OCT, with mean (SD) thickness measurements of 264.2 (11.2) μm (P < .001), 334.2 (11.4) μm (P < .001), and 495.0 (32.2) μm (P < .001) for stages 1, 2, and 3, respectively and with continuous stage labels (ρ = 0.739, P < .001). Conclusions and Relevance These results suggest that OCT-based quantification of peripheral stage in ROP may be an objective and quantitative biomarker that may be useful for clinical diagnosis and longitudinal monitoring and may have implications for disease classification in the future.
Collapse
Affiliation(s)
| | - Shuibin Ni
- Casey Eye Institute, Oregon Health & Science University, Portland
- Department of Biomedical Engineering, Oregon Health & Science University, Portland
| | - Susan Ostmo
- Casey Eye Institute, Oregon Health & Science University, Portland
| | | | - Aaron S. Coyner
- Casey Eye Institute, Oregon Health & Science University, Portland
| | - Mani Woodward
- Casey Eye Institute, Oregon Health & Science University, Portland
| | - Michael F. Chiang
- National Eye Institute, National Institutes of Health, Bethesda, Maryland
| | - Yali Jia
- Casey Eye Institute, Oregon Health & Science University, Portland
- Department of Biomedical Engineering, Oregon Health & Science University, Portland
| | - David Huang
- Casey Eye Institute, Oregon Health & Science University, Portland
- Department of Biomedical Engineering, Oregon Health & Science University, Portland
| | | | - Yifan Jian
- Casey Eye Institute, Oregon Health & Science University, Portland
- Department of Biomedical Engineering, Oregon Health & Science University, Portland
| |
Collapse
|
14
|
Lanzelotte V, Vieira AA, Monteiro Fonseca AB, Eduardo da Silva J, Gonet B, Bueno AC. ROP screening with the Pictor Plus camera: a telemedicine solution for developing countries. J AAPOS 2022; 26:244.e1-244.e6. [PMID: 36126881 DOI: 10.1016/j.jaapos.2022.05.019] [Citation(s) in RCA: 2] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 11/29/2021] [Revised: 05/24/2022] [Accepted: 05/24/2022] [Indexed: 11/27/2022]
Abstract
PURPOSE To assess the Pictor Plus portable noncontact ophthalmic camera for use in diagnosis of severe retinopathy of prematurity (ROP) and its feasibility for telemedicine applications, especially in developing countries. METHODS A cross-sectional study was conducted from April 2018 to December 2019 in six public neonatal care centers in the state of Rio de Janeiro. Observer 1 performed the examination with a binocular indirect ophthalmoscope and subsequently captured a digital image using the Pictor Plus camera. These images were analyzed by observers 2 and 3, who were masked to the results recorded by observer 1. RESULTS The convenience sample was 200 premature newborns (n = 712 eyes) with a birthweight of ≤1,500 g or gestational age at birth of ≤32 weeks. The sensitivity of Pictor Plus digital images for the diagnosis of plus or pre-plus disease was 65.8% for observer 2 and 63.2% for observer 3; the specificity, 100% and 98,4%, respectively. Interobserver agreement analysis for image reading compared to observer 1 indirect ophthalmoscopy showed kappa coefficients of 0.77 (observer 2) and 0.68 (observer 3). CONCLUSIONS In our study cohort, the Pictor Plus, a relatively low-cost camera, was effective in facilitating accurate diagnosis of ROP.
Collapse
Affiliation(s)
- Viviane Lanzelotte
- Municipal Health Secretariat, Rio de Janeiro, Brazil; MESP-MI Department, Faculty of Medicine, Fluminense Federal University, Niterói, Brazil.
| | - Alan Araujo Vieira
- Maternal and Child Department of the Faculty of Medicine of the Fluminense Federal University, Niterói, Brazil
| | - Ana Beatriz Monteiro Fonseca
- Department of Statistics, Institute of Mathematics and Statistics, Fluminense Federal University, Niterói, Brazil
| | | | - Bárbara Gonet
- Municipal Health Secretariat, Rio de Janeiro, Brazil
| | - Arnaldo Costa Bueno
- Maternal and Child Department of the Faculty of Medicine of the Fluminense Federal University, Niterói, Brazil
| |
Collapse
|
15
|
National guideline for ophthalmological screening of premature infants in Germany (S2k level, AWMF guidelines register no. 024/010, March 2020) : Joint recommendation of the German Ophthalmological Society (DOG), German Retina Society (RG), Professional Association of Ophthalmologists in Germany (BVA), German Society of Pediatrics and Adolescent Medicine (DGKJ), Professional Association of Pediatricians (BVKJ), Federal Association "The Premature Infant", Society for Neonatology and Pediatric Intensive Care Medicine (GNPI). DIE OPHTHALMOLOGIE 2022; 119:123-136. [PMID: 35507084 DOI: 10.1007/s00347-022-01632-4] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Track Full Text] [Subscribe] [Scholar Register] [Accepted: 03/25/2022] [Indexed: 01/25/2023]
|
16
|
Campbell JP, Chiang MF, Chen JS, Moshfeghi DM, Nudleman E, Ruambivoonsuk P, Cherwek H, Cheung CY, Singh P, Kalpathy-Cramer J, Ostmo S, Eydelman M, Chan RP, Capone A. Artificial Intelligence for Retinopathy of Prematurity: Validation of a Vascular Severity Scale against International Expert Diagnosis. Ophthalmology 2022; 129:e69-e76. [PMID: 35157950 PMCID: PMC9232863 DOI: 10.1016/j.ophtha.2022.02.008] [Citation(s) in RCA: 27] [Impact Index Per Article: 13.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/08/2021] [Revised: 01/31/2022] [Accepted: 02/03/2022] [Indexed: 01/07/2023] Open
Abstract
PURPOSE To validate a vascular severity score as an appropriate output for artificial intelligence (AI) Software as a Medical Device (SaMD) for retinopathy of prematurity (ROP) through comparison with ordinal disease severity labels for stage and plus disease assigned by the International Classification of Retinopathy of Prematurity, Third Edition (ICROP3), committee. DESIGN Validation study of an AI-based ROP vascular severity score. PARTICIPANTS A total of 34 ROP experts from the ICROP3 committee. METHODS Two separate datasets of 30 fundus photographs each for stage (0-5) and plus disease (plus, preplus, neither) were labeled by members of the ICROP3 committee using an open-source platform. Averaging these results produced a continuous label for plus (1-9) and stage (1-3) for each image. Experts were also asked to compare each image to each other in terms of relative severity for plus disease. Each image was also labeled with a vascular severity score from the Imaging and Informatics in ROP deep learning system, which was compared with each grader's diagnostic labels for correlation, as well as the ophthalmoscopic diagnosis of stage. MAIN OUTCOME MEASURES Weighted kappa and Pearson correlation coefficients (CCs) were calculated between each pair of grader classification labels for stage and plus disease. The Elo algorithm was also used to convert pairwise comparisons for each expert into an ordered set of images from least to most severe. RESULTS The mean weighted kappa and CC for all interobserver pairs for plus disease image comparison were 0.67 and 0.88, respectively. The vascular severity score was found to be highly correlated with both the average plus disease classification (CC = 0.90, P < 0.001) and the ophthalmoscopic diagnosis of stage (P < 0.001 by analysis of variance) among all experts. CONCLUSIONS The ROP vascular severity score correlates well with the International Classification of Retinopathy of Prematurity committee member's labels for plus disease and stage, which had significant intergrader variability. Generation of a consensus for a validated scoring system for ROP SaMD can facilitate global innovation and regulatory authorization of these technologies.
Collapse
Affiliation(s)
- J. Peter Campbell
- Casey Eye Institute, Department of Ophthalmology, Oregon Health & Science University, Portland, OR
| | | | - Jimmy S. Chen
- Casey Eye Institute, Department of Ophthalmology, Oregon Health & Science University, Portland, OR
| | - Darius M. Moshfeghi
- Byers Eye Institute, Horngren Family Vitreoretinal Center,Department of Ophthalmology, Stanford University, Palo Alto, CA
| | - Eric Nudleman
- Department of Ophthalmology, University of California, San Diego
| | | | | | - Carol Y. Cheung
- Department of Ophthalmology and Visual Sciences, Faculty of Medicine, The Chinese University of Hong
| | - Praveer Singh
- Department of Radiology, MGH/Harvard Medical School, Charlestown, MA;,Massachusetts General Hospital & Brigham and Women’s Hospital Center for Clinical Data Science, Boston, MA
| | - Jayashree Kalpathy-Cramer
- Department of Radiology, MGH/Harvard Medical School, Charlestown, MA;,Massachusetts General Hospital & Brigham and Women’s Hospital Center for Clinical Data Science, Boston, MA
| | - Susan Ostmo
- Casey Eye Institute, Department of Ophthalmology, Oregon Health & Science University, Portland, OR
| | - Malvina Eydelman
- Center for Devices and Radiological Health, US Food and Drug Administration, Silver Spring, Maryland
| | - R.V. Paul Chan
- Department of Ophthalmology and Visual Sciences, University of Illinois at Chicago, Chicago, IL
| | - Antonio Capone
- Associated Retinal Consultants, Oakland University William Beaumont School of Medicine, Royal Oak, Michigan, USA
| | | | | |
Collapse
|
17
|
Fielder AR, Quinn GE, Paul Chan RV, Holmström GE, Chiang MF. Retinopathy of prematurity classification updates: possible implications for treatment. J AAPOS 2022; 26:109-112. [PMID: 35472596 DOI: 10.1016/j.jaapos.2022.03.003] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 03/16/2022] [Accepted: 03/25/2022] [Indexed: 01/07/2023]
Affiliation(s)
- Alistair R Fielder
- Department of Optometry & Visual Science, City, University of London, England.
| | - Graham E Quinn
- Division of Ophthalmology, Children's Hospital of Philadelphia, Scheie Eye Institute, Raymond and Ruth Perelman School of Medicine at the University of Pennsylvania, Philadelphia
| | - R V Paul Chan
- Department of Ophthalmology and Visual Sciences, Illinois Eye and Ear Infirmary, University of Illinois at Chicago
| | - Gerd E Holmström
- Department Surgical Sciences/Ophthalmology, Uppsala University, Uppsala, Sweden
| | - Michael F Chiang
- National Eye Institute, National Institutes of Health, Bethesda, Maryland
| |
Collapse
|
18
|
Hanif A, Yıldız İ, Tian P, Kalkanlı B, Erdoğmuş D, Ioannidis S, Dy J, Kalpathy-Cramer J, Ostmo S, Jonas K, Chan RVP, Chiang MF, Campbell JP. Improved Training Efficiency for Retinopathy of Prematurity Deep Learning Models Using Comparison versus Class Labels. OPHTHALMOLOGY SCIENCE 2022; 2:100122. [PMID: 36249702 PMCID: PMC9560533 DOI: 10.1016/j.xops.2022.100122] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 08/26/2021] [Revised: 12/13/2021] [Accepted: 01/26/2022] [Indexed: 11/24/2022]
Abstract
Purpose To compare the efficacy and efficiency of training neural networks for medical image classification using comparison labels indicating relative disease severity versus diagnostic class labels from a retinopathy of prematurity (ROP) image dataset. Design Evaluation of diagnostic test or technology. Participants Deep learning neural networks trained on expert-labeled wide-angle retinal images obtained from patients undergoing diagnostic ROP examinations obtained as part of the Imaging and Informatics in ROP (i-ROP) cohort study. Methods Neural networks were trained with either class or comparison labels indicating plus disease severity in ROP retinal fundus images from 2 datasets. After training and validation, all networks underwent evaluation using a separate test dataset in 1 of 2 binary classification tasks: normal versus abnormal or plus versus nonplus. Main Outcome Measures Area under the receiver operating characteristic curve (AUC) values were measured to assess network performance. Results Given the same number of labels, neural networks learned more efficiently by comparison, generating significantly higher AUCs in both classification tasks across both datasets. Similarly, given the same number of images, comparison learning developed networks with significantly higher AUCs across both classification tasks in 1 of 2 datasets. The difference in efficiency and accuracy between models trained on either label type decreased as the size of the training set increased. Conclusions Comparison labels individually are more informative and more abundant per sample than class labels. These findings indicate a potential means of overcoming the common obstacle of data variability and scarcity when training neural networks for medical image classification tasks.
Collapse
|
19
|
Mangalesh S, Seely KR, Tran-Viet D, Tai V, Chen X, Prakalapakorn SG, Freedman SF, Toth CA. Integrated Visualization Highlighting Retinal Changes in Retinopathy of Prematurity From 3-Dimensional Optical Coherence Tomography Data. JAMA Ophthalmol 2022; 140:725-729. [PMID: 35616956 DOI: 10.1001/jamaophthalmol.2022.1344] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/14/2022]
Abstract
Importance Early diagnosis of plus disease is critical in the management of retinopathy of prematurity (ROP). However, there is substantial interexpert disagreement in the diagnosis of plus disease based on vascular changes alone. Information derived from optical coherence tomography (OCT) may help characterize the severity of vascular and structural abnormalities in ROP. Objective To describe integrated visualization of 3-dimensional (3-D) data from investigational swept-source OCT optimized to delineate retinal vascular and microanatomical features in eyes with and without ROP. Design, Setting, and Participants This cross-sectional, observational report of OCT was captured in the prospective Study of Eye Imaging in Preterm Infants (BabySTEPS) designed in July 2016 at the Duke Health Intensive Care Nursery. Between December 2018 and August 2019, 2 preterm infants born at 24 and 30 weeks' gestation were enrolled, underwent ROP screening, and were imaged at those screening visits. Data at 36 weeks' postmenstrual age were analyzed via this visualization developed between September 2020 and May 2021. Main Outcomes and Measures Superimposed en face retinal vascular shadow view (RVSV) montages and thickness maps were used along with OCT B-scans to evaluate retinal vasculature and cross-section in eyes with and without ROP. Results In the right eyes of 2 infants, 3-D data were integrated and visualized from investigational bedside OCT imaging at the posterior pole. In the infant who developed type 1 ROP, RVSV-OCT confirmed presence of dilated and tortuous posterior pole vessels, shunting, and incomplete perifoveal vascular development, resulting in a temporal notch of avascular retina in zone 1. The thickness map revealed irregular pockets of thickening and thinning, and integrated visualization outlined the demarcation between thicker vascularized retina and thinner avascular fovea and presence of extraretinal neovascularization overlying elevated vessels in the superior arcades. In the infant without ROP (stage 0), RVSV-OCT revealed no abnormal vascular findings at the posterior pole. The integrated visualization showed a dome-shaped retinal thickening at the fovea, which was confirmed as macular edema. Conclusions and Relevance In 2 preterm infants in BabySTEPS, 3-D visualization of OCT findings during the ongoing ROP disease process demonstrated supplemental information about the retinal vasculature and microanatomy that can be useful to clinicians. These additional details provided by OCT could be integrated into future ROP screening methods with artificial intelligence-based analytics.
Collapse
Affiliation(s)
- Shwetha Mangalesh
- Department of Ophthalmology, Duke University, Durham, North Carolina
| | - Kai R Seely
- Department of Ophthalmology, Duke University, Durham, North Carolina
| | - Du Tran-Viet
- Department of Ophthalmology, Duke University, Durham, North Carolina
| | - Vincent Tai
- Department of Ophthalmology, Duke University, Durham, North Carolina
| | - Xi Chen
- Department of Ophthalmology, Duke University, Durham, North Carolina
| | - S Grace Prakalapakorn
- Department of Ophthalmology, Duke University, Durham, North Carolina.,Department of Pediatrics, Duke University, Durham, North Carolina
| | - Sharon F Freedman
- Department of Ophthalmology, Duke University, Durham, North Carolina.,Department of Pediatrics, Duke University, Durham, North Carolina
| | - Cynthia A Toth
- Department of Ophthalmology, Duke University, Durham, North Carolina.,Department of Biomedical Engineering, Duke University, Durham, North Carolina
| | | |
Collapse
|
20
|
Tedeschi T, Lee K, Zhu W, Fawzi AA. Limited hyperoxia-induced proliferative retinopathy: A model of persistent retinal vascular dysfunction, preretinal fibrosis and hyaloidal vascular reprogramming for retinal rescue. PLoS One 2022; 17:e0267576. [PMID: 35476813 PMCID: PMC9045626 DOI: 10.1371/journal.pone.0267576] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/27/2022] [Accepted: 04/11/2022] [Indexed: 11/21/2022] Open
Abstract
Background Retinopathy of prematurity (ROP) remains the leading cause for blindness in children. Limited hyperoxia induced proliferative retinopathy (L-HIPR) was recently introduced as a potential animal model for ROP and persistent fetal vasculature; however, the detailed pathological changes remain unclear. Methods To model L-HIPR, we placed C57BL/6J mice in 65% oxygen from birth to post-natal day 7 (P7). We examined eyes at intervals between P12 and P30. Retinal morphometry, thickness, and preretinal fibrosis were quantified at different time points on histological sections stained with hematoxylin and eosin (H&E) and Masson Trichrome, respectively. Vascular development, angiogenesis, inflammation, and pericyte coverage were analyzed using immunohistochemistry staining in retinal flat mounts and cross sections. Results In L-HIPR, the hyaloidal vessels persisted until the latest time point in this study, P30 and began to invaginate the peripheral then central retina starting at P12. Central retinal distortion was noted beginning at P17, while the peripheral retina demonstrated a trend of thinning from P12 to P30. We found that L-HIPR was associated with delayed and abnormal retinal vascular development with subsequent retinal inflammation, pericyte loss and preretinal fibrosis. Conclusion Our study presents a detailed analysis of the L-HIPR animal model demonstrating vitreoretinal pathologic changes, preretinal fibrosis and persistent hyaloidal vessels into adulthood. Based on our findings, we suggest that the persistence and peculiar stepwise migration of the hyaloidal vessels into the retina may provide a potential rescue mechanism for inner retinal development that deserves further study.
Collapse
Affiliation(s)
- Thomas Tedeschi
- Department of Ophthalmology, Feinberg School of Medicine, Northwestern University, Chicago, IL, United States of America
| | - Kendal Lee
- Department of Ophthalmology, Feinberg School of Medicine, Northwestern University, Chicago, IL, United States of America
| | - Wei Zhu
- Department of Ophthalmology, Feinberg School of Medicine, Northwestern University, Chicago, IL, United States of America
- Department of Ophthalmology, Changshu No. 2 People’s Hospital, Changshu, China
| | - Amani A. Fawzi
- Department of Ophthalmology, Feinberg School of Medicine, Northwestern University, Chicago, IL, United States of America
- * E-mail:
| |
Collapse
|
21
|
Kumm J. Newborn Eye Screening as an Application of AI. Ophthalmic Surg Lasers Imaging Retina 2021; 52:S17-S22. [PMID: 34908492 DOI: 10.3928/23258160-20211115-03] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/20/2022]
Abstract
Artificial intelligence (AI) applications are diverse and serve varied functions in clinical practice. The most successful products today are clinical decision tools used by physicians, but autonomous AI is gaining traction. Widespread use of AI is limited in part because of concerns about bias, fault-tolerance, and specificity. Adoption of AI often depends on removing cost and complexity in clinical workflow integration, providing clear incentives for use, and providing clear demonstration of clinical outcome. Existing wide-angle photographic screening could be integrated into the clinical workflow based on prior implementations for premature babies and linked with AI interpretation with existing technology. Incidence of retinal abnormality, clinical considerations, AI performance, grading variation for AI-augmented human grading, and cost and policy aspects play a significant role. Improved outcomes for newborns and a relatively high estimated incidence of abnormality have been named as benefits to counterweigh costs in the long term. [Ophthalmic Surg Lasers Imaging Retina. 2021;52:S17-S22.].
Collapse
|
22
|
Chiang MF, Quinn GE, Fielder AR, Ostmo SR, Paul Chan RV, Berrocal A, Binenbaum G, Blair M, Peter Campbell J, Capone A, Chen Y, Dai S, Ells A, Fleck BW, Good WV, Elizabeth Hartnett M, Holmstrom G, Kusaka S, Kychenthal A, Lepore D, Lorenz B, Martinez-Castellanos MA, Özdek Ş, Ademola-Popoola D, Reynolds JD, Shah PK, Shapiro M, Stahl A, Toth C, Vinekar A, Visser L, Wallace DK, Wu WC, Zhao P, Zin A. International Classification of Retinopathy of Prematurity, Third Edition. Ophthalmology 2021; 128:e51-e68. [PMID: 34247850 PMCID: PMC10979521 DOI: 10.1016/j.ophtha.2021.05.031] [Citation(s) in RCA: 246] [Impact Index Per Article: 82.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/29/2021] [Revised: 05/27/2021] [Accepted: 05/28/2021] [Indexed: 12/17/2022] Open
Abstract
PURPOSE The International Classification of Retinopathy of Prematurity is a consensus statement that creates a standard nomenclature for classification of retinopathy of prematurity (ROP). It was initially published in 1984, expanded in 1987, and revisited in 2005. This article presents a third revision, the International Classification of Retinopathy of Prematurity, Third Edition (ICROP3), which is now required because of challenges such as: (1) concerns about subjectivity in critical elements of disease classification; (2) innovations in ophthalmic imaging; (3) novel pharmacologic therapies (e.g., anti-vascular endothelial growth factor agents) with unique regression and reactivation features after treatment compared with ablative therapies; and (4) recognition that patterns of ROP in some regions of the world do not fit neatly into the current classification system. DESIGN Review of evidence-based literature, along with expert consensus opinion. PARTICIPANTS International ROP expert committee assembled in March 2019 representing 17 countries and comprising 14 pediatric ophthalmologists and 20 retinal specialists, as well as 12 women and 22 men. METHODS The committee was initially divided into 3 subcommittees-acute phase, regression or reactivation, and imaging-each of which used iterative videoconferences and an online message board to identify key challenges and approaches. Subsequently, the entire committee used iterative videoconferences, 2 in-person multiday meetings, and an online message board to develop consensus on classification. MAIN OUTCOME MEASURES Consensus statement. RESULTS The ICROP3 retains current definitions such as zone (location of disease), stage (appearance of disease at the avascular-vascular junction), and circumferential extent of disease. Major updates in the ICROP3 include refined classification metrics (e.g., posterior zone II, notch, subcategorization of stage 5, and recognition that a continuous spectrum of vascular abnormality exists from normal to plus disease). Updates also include the definition of aggressive ROP to replace aggressive-posterior ROP because of increasing recognition that aggressive disease may occur in larger preterm infants and beyond the posterior retina, particularly in regions of the world with limited resources. ROP regression and reactivation are described in detail, with additional description of long-term sequelae. CONCLUSIONS These principles may improve the quality and standardization of ROP care worldwide and may provide a foundation to improve research and clinical care.
Collapse
Affiliation(s)
- Michael F Chiang
- National Eye Institute, National Institutes of Health, Bethesda, Maryland.
| | - Graham E Quinn
- Division of Ophthalmology, Children's Hospital of Philadelphia, Scheie Eye Institute, Raymond and Ruth Perelman School of Medicine at the University of Pennsylvania, Philadelphia, Pennsylvania
| | - Alistair R Fielder
- Department of Optometry and Visual Science, University of London, London, United Kingdom
| | - Susan R Ostmo
- Department of Ophthalmology, Casey Eye Institute, Oregon Health & Science University, Portland, Oregon
| | - R V Paul Chan
- Department of Ophthalmology and Visual Sciences, Illinois Eye and Ear Infirmary, University of Illinois at Chicago, Chicago, Illinois
| | - Audina Berrocal
- Department of Ophthalmology, Bascom Palmer Eye Institute, University of Miami Miller School of Medicine, Miami, Florida
| | - Gil Binenbaum
- Division of Ophthalmology, Children's Hospital of Philadelphia, Scheie Eye Institute, Raymond and Ruth Perelman School of Medicine at the University of Pennsylvania, Philadelphia, Pennsylvania
| | - Michael Blair
- Retina Consultants, Ltd., Des Plaines, Illinois; Department of Ophthalmology, University of Chicago, Chicago, Illinois
| | - J Peter Campbell
- Department of Ophthalmology, Casey Eye Institute, Oregon Health & Science University, Portland, Oregon
| | - Antonio Capone
- Associated Retinal Consultants, PC, Royal Oak, Michigan, and Department of Ophthalmology, Oakland University, William Beaumont Hospital School of Medicine, Auburn Hills, Michigan
| | - Yi Chen
- Department of Ophthalmology, China-Japan Friendship Hospital, Beijing, China
| | - Shuan Dai
- Ophthalmology Department, Queensland Children's Hospital, Brisbane, Australia
| | - Anna Ells
- Calgary Retina Consultants, Calgary, Canada
| | - Brian W Fleck
- Department of Ophthalmology, University of Edinburgh, Edinburgh, United Kingdom
| | - William V Good
- Smith-Kettlewell Eye Research Institute, San Francisco, California
| | - M Elizabeth Hartnett
- Department of Ophthalmology and Visual Sciences, John A. Moran Eye Center, University of Utah, Salt Lake City, Utah
| | - Gerd Holmstrom
- Department Neuroscience/Ophthalmology, Uppsala University, Uppsala, Sweden
| | - Shunji Kusaka
- Department of Ophthalmology, Kindai University, Osakasayama, Japan
| | | | - Domenico Lepore
- A. Gemelli Foundation IRCSS, Department of Ageing and Neuroscience, Catholic University of the Sacred Heart, Rome, Italy
| | - Birgit Lorenz
- Department of Ophthalmology, Justus-Liebig-University Giessen, Giessen, Germany; Department of Ophthalmology, Universitaetsklinikum Bonn, Bonn, Germany
| | | | - Şengül Özdek
- Department of Ophthalmology, School of Medicine, Gazi University, Ankara, Turkey
| | | | - James D Reynolds
- Ross Eye Institute, Department of Ophthalmology, University at Buffalo, Buffalo, New York
| | - Parag K Shah
- Department of Pediatric Retina and Ocular Oncology, Aravind Eye Hospital, Coimbatore, Tamil Nadu, India
| | | | - Andreas Stahl
- Department of Ophthalmology, University Medicine Greifswald, Greifswald, Germany
| | - Cynthia Toth
- Department of Ophthalmology, Duke University Medical Center, Durham, North Carolina
| | - Anand Vinekar
- Department of Pediatric Retina, Narayana Nethralaya Eye Institute, Bangalore, Karnataka, India
| | - Linda Visser
- Department of Ophthalmology, University of KwaZulu-Natal, Durban, South Africa
| | - David K Wallace
- Department of Ophthalmology, Indiana University School of Medicine, Indianapolis, Indiana
| | - Wei-Chi Wu
- Department of Ophthalmology, Chang Gung Memorial Hospital, Linkou, Taoyuan, Taiwan, and Chang Gung University, College of Medicine, Taoyuan, Taiwan
| | - Peiquan Zhao
- Department of Ophthalmology, Xinhua Hospital affiliated to Shanghai Jiao Tong University School of Medicine, Shanghai, China
| | - Andrea Zin
- Clinical Research Unit, Fernandes Figueira Institute, FIOCRUZ, Rio de Janeiro, Brazil
| |
Collapse
|
23
|
Singh K, Woodward MA. The Rigorous Work of Evaluating Consistency and Accuracy in Electronic Health Record Data. JAMA Ophthalmol 2021; 139:894-895. [PMID: 34196690 DOI: 10.1001/jamaophthalmol.2021.2042] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/14/2022]
Affiliation(s)
- Karandeep Singh
- Department of Learning Health Sciences, University of Michigan Medical School, Ann Arbor.,Department of Internal Medicine, University of Michigan Medical School, Ann Arbor.,Institute for Healthcare Policy and Innovation, University of Michigan, Ann Arbor
| | - Maria A Woodward
- Institute for Healthcare Policy and Innovation, University of Michigan, Ann Arbor.,Department of Ophthalmology and Visual Sciences, W. K. Kellogg Eye Center, University of Michigan Medical School, Ann Arbor
| |
Collapse
|
24
|
Repka MX. A Revision of the International Classification of Retinopathy of Prematurity. Ophthalmology 2021; 128:1381-1383. [PMID: 34332760 DOI: 10.1016/j.ophtha.2021.07.014] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/07/2021] [Revised: 07/09/2021] [Accepted: 07/09/2021] [Indexed: 11/20/2022] Open
|
25
|
Campbell JP, Kim SJ, Brown JM, Ostmo S, Chan RVP, Kalpathy-Cramer J, Chiang MF. Evaluation of a Deep Learning-Derived Quantitative Retinopathy of Prematurity Severity Scale. Ophthalmology 2021; 128:1070-1076. [PMID: 33121959 PMCID: PMC8076329 DOI: 10.1016/j.ophtha.2020.10.025] [Citation(s) in RCA: 35] [Impact Index Per Article: 11.7] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/22/2020] [Revised: 09/30/2020] [Accepted: 10/20/2020] [Indexed: 11/20/2022] Open
Abstract
PURPOSE To evaluate the clinical usefulness of a quantitative deep learning-derived vascular severity score for retinopathy of prematurity (ROP) by assessing its correlation with clinical ROP diagnosis and by measuring clinician agreement in applying a novel scale. DESIGN Analysis of existing database of posterior pole fundus images and corresponding ophthalmoscopic examinations using 2 methods of assigning a quantitative scale to vascular severity. PARTICIPANTS Images were from clinical examinations of patients in the Imaging and Informatics in ROP Consortium. Four ophthalmologists and 1 study coordinator evaluated vascular severity on a scale from 1 to 9. METHODS A quantitative vascular severity score (1-9) was applied to each image using a deep learning algorithm. A database of 499 images was developed for assessment of interobserver agreement. MAIN OUTCOME MEASURES Distribution of deep learning-derived vascular severity scores with the clinical assessment of zone (I, II, or III), stage (0, 1, 2, or 3), and extent (<3 clock hours, 3-6 clock hours, and >6 clock hours) of stage 3 evaluated using multivariate linear regression and weighted κ values and Pearson correlation coefficients for interobserver agreement on a 1-to-9 vascular severity scale. RESULTS For deep learning analysis, a total of 6344 clinical examinations were analyzed. A higher deep learning-derived vascular severity score was associated with more posterior disease, higher disease stage, and higher extent of stage 3 disease (P < 0.001 for all). For a given ROP stage, the vascular severity score was higher in zone I than zones II or III (P < 0.001). Multivariate regression found zone, stage, and extent all were associated independently with the severity score (P < 0.001 for all). For interobserver agreement, the mean ± standard deviation weighted κ value was 0.67 ± 0.06, and the Pearson correlation coefficient ± standard deviation was 0.88 ± 0.04 on the use of a 1-to-9 vascular severity scale. CONCLUSIONS A vascular severity scale for ROP seems feasible for clinical adoption; corresponds with zone, stage, extent of stage 3, and plus disease; and facilitates the use of objective technology such as deep learning to improve the consistency of ROP diagnosis.
Collapse
Affiliation(s)
- J Peter Campbell
- Department of Ophthalmology, Casey Eye Institute, Oregon Health & Science University, Portland, Oregon
| | - Sang Jin Kim
- Department of Ophthalmology, Casey Eye Institute, Oregon Health & Science University, Portland, Oregon; Department of Ophthalmology, Samsung Medical Center, Sungkyunkwan University School of Medicine, Seoul, Korea
| | - James M Brown
- School of Computer Science, University of Lincoln, Lincoln, United Kingdom
| | - Susan Ostmo
- Department of Ophthalmology, Casey Eye Institute, Oregon Health & Science University, Portland, Oregon
| | - R V Paul Chan
- Department of Ophthalmology and Visual Sciences, Illinois Eye and Ear Infirmary, University of Illinois at Chicago, Chicago, Illinois
| | - Jayashree Kalpathy-Cramer
- Athinoula A. Martinos Center for Biomedical Imaging, Department of Radiology, Massachusetts General Hospital, Charlestown, Massachusetts; Massachusetts General Hospital and Brigham and Women's Hospital Center for Clinical Data Science, Boston, Massachusetts
| | - Michael F Chiang
- Department of Ophthalmology, Casey Eye Institute, Oregon Health & Science University, Portland, Oregon; Department of Medical Informatics & Clinical Epidemiology, Oregon Health & Science University, Portland, Oregon.
| |
Collapse
|
26
|
[Guidelines for ophthalmological screening of premature infants in Germany (S2k level, AWMF guidelines register no. 024/010, March 2020) : Joint recommendation of the German Ophthalmological Society (DOG), Retinological Society (RG), Professional Association of Ophthalmologists in Germany e. V. (BVA), German Society of Paediatrics and Adolescent Medicine (DGKJ), Professional Association of Pediatricians (BVKJ), Federal Association "The Premature Child" , Society for Neonatology and Paediatric Intensive Care Medicine (GNPI)]. Ophthalmologe 2021; 118:117-131. [PMID: 33694101 DOI: 10.1007/s00347-021-01353-0] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Accepted: 02/11/2021] [Indexed: 12/20/2022]
|
27
|
Bao Y, Ming WK, Mou ZW, Kong QH, Li A, Yuan TF, Mi XS. Current Application of Digital Diagnosing Systems for Retinopathy of Prematurity. COMPUTER METHODS AND PROGRAMS IN BIOMEDICINE 2021; 200:105871. [PMID: 33309305 DOI: 10.1016/j.cmpb.2020.105871] [Citation(s) in RCA: 4] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 03/30/2020] [Accepted: 11/18/2020] [Indexed: 06/12/2023]
Abstract
BACKGROUND AND OBJECTIVE Retinopathy of prematurity (ROP), a proliferative vascular eye disease, is one of the leading causes of blindness in childhood and prevails in premature infants with low-birth-weight. The recent progress in digital image analysis offers novel strategies for ROP diagnosis. This paper provides a comprehensive review on the development of digital diagnosing systems for ROP to software researchers. It may also be adopted as a guide to ophthalmologists for selecting the most suitable diagnostic software in the clinical setting, particularly for the remote ophthalmic support. METHODS We review the latest literatures concerning the application of digital diagnosing systems for ROP. The diagnosing systems are analyzed and categorized. Articles published between 1998 and 2020 were screened with the two searching engines Pubmed and Google Scholar. RESULTS Telemedicine is a method of remote image interpretation that can provide medical service to remote regions, and yet requires training to local operators. On the basis of image collection in telemedicine, computer-based image analytical systems for ROP were later developed. So far, the aforementioned systems have been mainly developed by virtue of classic machine learning, deep learning (DL) and multiple machine learning. During the past two decades, various computer-aided systems for ROP based on classic machine learning (e.g. RISA, ROPtool, CAIER) became available and have achieved satisfactory performance. Further, automated systems for ROP diagnosis based on DL are developed for clinical applications and exhibit high accuracy. Moreover, multiple instance learning is another method to establish an automated system for ROP detection besides DL, which, however, warrants further investigation in future. CONCLUSION At present, the incorporation of computer-based image analysis with telemedicine potentially enables the detection, supervision and in-time treatment of ROP for the preterm babies.
Collapse
Affiliation(s)
- Yuekun Bao
- Department of Ophthalmology, the First Affiliated Hospital of Jinan University, Guangzhou, China; State Key Laboratory of Ophthalmology, Zhongshan Ophthalmic Center, Sun Yat-sen University, Guangzhou, China
| | - Wai-Kit Ming
- Clinical Medicine, International School, Jinan University, Guangzhou, China
| | - Zhi-Wei Mou
- Department of Rehabilitation, the First Affiliated Hospital of Jinan University, Guangzhou, China
| | - Qi-Hang Kong
- Department of Ophthalmology, the First Affiliated Hospital of Jinan University, Guangzhou, China
| | - Ang Li
- Guangdong - Hong Kong - Macau Institute of CNS Regeneration, Joint International Research Laboratory of CNS Regeneration Ministry of Education, Jinan University, Guangzhou, China; Bioland Laboratory (Guangzhou Regenerative Medicine and Health Guangdong Laboratory), Guangzhou, China.
| | - Ti-Fei Yuan
- Shanghai Key Laboratory of Psychotic Disorders, Shanghai Mental Health Center, Shanghai Jiaotong University School of Medicine, Shanghai, China.
| | - Xue-Song Mi
- Department of Ophthalmology, the First Affiliated Hospital of Jinan University, Guangzhou, China; Changsha Academician Expert Workstation, Aier Eye Hospital Group, Changsha, China.
| |
Collapse
|
28
|
Management of retinopathy of prematurity (ROP) in a Polish cohort of infants. Sci Rep 2021; 11:4522. [PMID: 33633248 PMCID: PMC7907336 DOI: 10.1038/s41598-021-83985-5] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/06/2020] [Accepted: 02/08/2021] [Indexed: 11/22/2022] Open
Abstract
Retinopathy of prematurity (ROP) is potentially blinding, but screening and timely treatment can stop its progression. The data on treatment outcomes of ROP from Central and Eastern Europe are scarce. Therefore, we aimed to analyze the latest results of ROP management in Poznan medical center to update the data from this world region. In the years 2016–2019, 178 patients (350 eyes) received treatment for ROP (6.1% of the screened population). The mean gestational age was 26 weeks (range 22–31 weeks), the mean birth weight was 868 g (range 410–1890 g). The most frequent ROP stage at treatment was zone II, stage 3 + (34.9%). As the first line of treatment, 115 infants (226 eyes, 64.6%) underwent laser photocoagulation (LP); 61 infants (120 eyes, 34.3%) received intravitreal ranibizumab injections (IVR); and 2 infants (4 eyes, 0.6%) were treated simultaneously with LP and IVR. One hundred twenty-six eyes (36%) of 63 patients required retreatment: 20.4% treated with LP and 66.7% treated with IVR. Retinal detachment occurred in 14 eyes (4%). The incidence of ROP, ROP requiring treatment, and reoccurrence rates are higher in the Polish population than in Western Europe and the USA. The identified treatment patterns find increasing use of anti-VEGF agents.
Collapse
|
29
|
Choi RY, Brown JM, Kalpathy-Cramer J, Chan RVP, Ostmo S, Chiang MF, Campbell JP. Variability in Plus Disease Identified Using a Deep Learning-Based Retinopathy of Prematurity Severity Scale. Ophthalmol Retina 2020; 4:1016-1021. [PMID: 32380115 PMCID: PMC7867469 DOI: 10.1016/j.oret.2020.04.022] [Citation(s) in RCA: 10] [Impact Index Per Article: 2.5] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/29/2020] [Revised: 04/21/2020] [Accepted: 04/24/2020] [Indexed: 11/19/2022]
Abstract
PURPOSE Retinopathy of prematurity is a leading cause of childhood blindness worldwide, but clinical diagnosis is subjective, which leads to treatment differences. Our goal was to determine objective differences in the diagnosis of plus disease between clinicians using an automated retinopathy of prematurity (ROP) vascular severity score. DESIGN This retrospective cohort study used data from the Imaging and Informatics in ROP Consortium, which comprises 8 tertiary care centers in North America. Fundus photographs of all infants undergoing ROP screening examinations between July 1, 2011, and December 31, 2016, were obtained. PARTICIPANTS Infants meeting ROP screening criteria who were diagnosed with plus disease and treatment initiated by an examining physician based on ophthalmoscopic examination results. METHODS An ROP severity score (1-9) was generated for each image using a deep learning (DL) algorithm. MAIN OUTCOME MEASURES The mean, median, and range of ROP vascular severity scores overall and for each examiner when the diagnosis of plus disease was made. RESULTS A total of 5255 clinical examinations in 871 babies were analyzed. Of these, 168 eyes were diagnosed with plus disease by 11 different examiners and were included in the study. The mean ± standard deviation vascular severity score for patients diagnosed with plus disease was 7.4 ± 1.9, median was 8.5 (interquartile range, 5.8-8.9), and range was 1.1 to 9.0. Within some examiners, variability in the level of vascular severity diagnosed as plus disease was present, and 1 examiner routinely diagnosed plus disease in patients with less severe disease than the other examiners (P < 0.01). CONCLUSIONS We observed variability both between and within examiners in the diagnosis of plus disease using DL. Prospective evaluation of clinical trial data using an objective measurement of vascular severity may help to define better the minimum necessary level of vascular severity for the diagnosis of plus disease or how other clinical features such as zone, stage, and extent of peripheral disease ought to be incorporated in treatment decisions.
Collapse
Affiliation(s)
- Rene Y Choi
- Department of Ophthalmology, Casey Eye Institute, Oregon Health & Science University, Portland, Oregon
| | - James M Brown
- Athinoula A. Martinos Center for Biomedical Imaging, Department of Radiology, Massachusetts General Hospital, Charlestown, Massachusetts; Department of Computer Science, University of Lincoln, Lincoln, United Kingdom
| | - Jayashree Kalpathy-Cramer
- Athinoula A. Martinos Center for Biomedical Imaging, Department of Radiology, Massachusetts General Hospital, Charlestown, Massachusetts
| | - R V Paul Chan
- Department of Ophthalmology and Visual Sciences, Illinois Eye and Ear Infirmary, University of Illinois at Chicago, Chicago, Illinois
| | - Susan Ostmo
- Department of Ophthalmology, Casey Eye Institute, Oregon Health & Science University, Portland, Oregon
| | - Michael F Chiang
- Department of Ophthalmology, Casey Eye Institute, Oregon Health & Science University, Portland, Oregon; Department of Medical Informatics and Clinical Epidemiology, Oregon Health & Science University, Portland, Oregon
| | - J Peter Campbell
- Department of Ophthalmology, Casey Eye Institute, Oregon Health & Science University, Portland, Oregon.
| |
Collapse
|
30
|
Seely KR, Wang KL, Tai V, Prakalapakorn SG, Chiu SJ, Viehland C, Grace S, Izatt JA, Freedman SF, Toth CA. Auto-Processed Retinal Vessel Shadow View Images From Bedside Optical Coherence Tomography to Evaluate Plus Disease in Retinopathy of Prematurity. Transl Vis Sci Technol 2020; 9:16. [PMID: 32879772 PMCID: PMC7442872 DOI: 10.1167/tvst.9.9.16] [Citation(s) in RCA: 4] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/27/2020] [Accepted: 06/02/2020] [Indexed: 01/30/2023] Open
Abstract
Purpose To describe the creation of en face retinal vessel shadow view (RVSV) optical coherence tomography (OCT) images and assess the feasibility of using these for evaluating vascular disease in preterm infants at risk for retinopathy of prematurity (ROP). Methods In this exploratory study, we selected images from eyes with a range of ROP vascular disease, prospectively acquired from preterm infants using an investigational, noncontact, handheld, bedside swept-source OCT. We autosegmented OCT volumes using custom infant-specific software, extracted RVSV-OCT images from volumetric data bracketed around the retinal pigment epithelium, and automontaged the resulting RVSV-OCT images. Three masked ophthalmologists graded the RVSV-OCT montages as plus, pre-plus, or neither and ranked them by relative vascular disease severity. Results We selected images from 17 imaging sessions (7 plus, 4 pre-plus, 6 neither on clinical examination). On review, 15/17 (88%) RVSV-OCT montages were gradable for plus, pre-plus, or neither and all 17 montages were rankable for relative severity. Intergrader agreement for plus, pre-plus, or neither grading was good (κ, 0.67; 95% confidence interval, 0.42–0.86) and for relative severity ranking was excellent (intraclass correlation coefficient, 0.98; 95% confidence interval, 0.96–0.99). Conclusions Our novel automatic processing method can create RVSV-OCT montages optimized for retinal vessel visualization for ROP screening. Although our data support the feasibility of using RVSV-OCT montages for ranking relative vascular disease severity, there is room for improved OCT image capture and processing methods in preterm infants screened for ROP. Translational Relevance Creation and grading of RVSV-OCT images could eventually be integrated into an alternative method for ROP screening.
Collapse
Affiliation(s)
- Kai R Seely
- Department of Ophthalmology, Duke University, Durham, NC, USA
| | - Kira L Wang
- Department of Ophthalmology, Duke University, Durham, NC, USA
| | - Vincent Tai
- Department of Ophthalmology, Duke University, Durham, NC, USA
| | | | | | | | - Sara Grace
- Department of Ophthalmology, Duke University, Durham, NC, USA
| | - Joseph A Izatt
- Department of Biomedical Engineering, Duke University, Durham, NC, USA
| | | | - Cynthia A Toth
- Department of Ophthalmology, Duke University, Durham, NC, USA.,Department of Biomedical Engineering, Duke University, Durham, NC, USA
| |
Collapse
|
31
|
Lepore D, Ji MH, Pagliara MM, Lenkowicz J, Capocchiano ND, Tagliaferri L, Boldrini L, Valentini V, Damiani A. Convolutional Neural Network Based on Fluorescein Angiography Images for Retinopathy of Prematurity Management. Transl Vis Sci Technol 2020; 9:37. [PMID: 32855841 PMCID: PMC7424905 DOI: 10.1167/tvst.9.2.37] [Citation(s) in RCA: 13] [Impact Index Per Article: 3.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/29/2019] [Accepted: 05/13/2020] [Indexed: 12/18/2022] Open
Abstract
Purpose The purpose of this study was to explore the use of fluorescein angiography (FA) images in a convolutional neural network (CNN) in the management of retinopathy of prematurity (ROP). Methods The dataset involved a total of 835 FA images of 149 eyes (90 patients), where each eye was associated with a binary outcome (57 “untreated” eyes and 92 “treated”; 308 “untreated” images, 527 “treated”). The resolution of the images was 1600 and 1200 px in 20% of cases, whereas the remaining 80% had a resolution of 640 and 480 px. All the images were resized to 640 and 480 px before training and no other preprocessing was applied. A CNN with four convolutional layers was trained on 90% of the images (n = 752) randomly chosen. The accuracy of the prediction was assessed on the remaining 10% of images (n = 83). Keras version 2.2.0 for R with Tensorflow backend version 1.11.0 was used for the analysis. Results The validation accuracy after 100 epochs was 0.88, whereas training accuracy was 0.97. The receiver operating characteristic (ROC) presented an area under the curve (AUC) of 0.91. Conclusions Our study showed, we believe for the first time, the applicability of artificial intelligence (CNN) technology in the ROP management driven by FA. Further studies are needed to exploit different fields of applications of this technology. Translational Relevance This algorithm is the basis for a system that could be applied to both ROP as well as experimental oxygen induced retinopathy.
Collapse
Affiliation(s)
- Domenico Lepore
- Dipartimento di Oftalmologia, Fondazione Policlinico Universitario "A. Gemelli" IRCCS, Rome, Italy
| | - Marco H Ji
- Byers Eye Institute, Horngren Family Vitreoretinal Center, Department of Ophthalmology, Stanford University School of Medicine, Palo Alto, CA, USA
| | - Monica M Pagliara
- Dipartimento di Oftalmologia, Fondazione Policlinico Universitario "A. Gemelli" IRCCS, Rome, Italy
| | - Jacopo Lenkowicz
- UOC Radioterapia Oncologica, Dipartimento di Diagnostica per Immagini, Radioterapia Oncologica ed Ematologia, Fondazione Policlinico Universitario "A. Gemelli" IRCCS, Rome, Italy
| | - Nikola D Capocchiano
- UOC Radioterapia Oncologica, Dipartimento di Diagnostica per Immagini, Radioterapia Oncologica ed Ematologia, Fondazione Policlinico Universitario "A. Gemelli" IRCCS, Rome, Italy
| | - Luca Tagliaferri
- UOC Radioterapia Oncologica, Dipartimento di Diagnostica per Immagini, Radioterapia Oncologica ed Ematologia, Fondazione Policlinico Universitario "A. Gemelli" IRCCS, Rome, Italy
| | - Luca Boldrini
- UOC Radioterapia Oncologica, Dipartimento di Diagnostica per Immagini, Radioterapia Oncologica ed Ematologia, Fondazione Policlinico Universitario "A. Gemelli" IRCCS, Rome, Italy
| | - Vincenzo Valentini
- UOC Radioterapia Oncologica, Dipartimento di Diagnostica per Immagini, Radioterapia Oncologica ed Ematologia, Fondazione Policlinico Universitario "A. Gemelli" IRCCS, Rome, Italy
| | - Andrea Damiani
- UOC Radioterapia Oncologica, Dipartimento di Diagnostica per Immagini, Radioterapia Oncologica ed Ematologia, Fondazione Policlinico Universitario "A. Gemelli" IRCCS, Rome, Italy
| | | |
Collapse
|
32
|
Raffa LH, Alessa SK, Alamri AS, Malaikah RH. Prediction of retinopathy of prematurity using the screening algorithm WINROP in a Saudi cohort of preterm infants. Saudi Med J 2020; 41:622-627. [PMID: 32518929 PMCID: PMC7502953 DOI: 10.15537/smj.2020.6.25127] [Citation(s) in RCA: 8] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/17/2020] [Accepted: 05/19/2020] [Indexed: 11/16/2022] Open
Abstract
OBJECTIVES To validate the web weight gain-based WINROP (weight, insulin-like growth factor I, neonatal, retinopathy of prematurity [ROP]) algorithm retrospectively to identify type 1 ROP in a Saudi cohort of premature infants. Methods: The records of preterm infants (greater than 23 and less than 32 weeks gestation) born between August 2013 and October 2018, were reviewed. Birth weight, gestational age, and weekly weight measurements of the premature infants were entered online. Based on weekly weight gain, the WINROP algorithm alerted clinicians whether infants were at high-risk for vision‑threatening type 1 ROP. Sensitivity, specificity, positive and negative predictive values were calculated. Results: The median gestational age of the infants at birth was 28 weeks, with median birth weight at 1085 g. Of the 175 infants included in the study, 13 (7.4%) developed type 1 ROP. WINROP positive alarm was triggered in 70.9% (124/175) of all infants and 100% (13/13) of those treated for type 1 ROP. The specificity of the algorithm was 31.5%. Positive predictive values was 10.5% and negative was 100%. Conclusion: The general WINROP sensitivity in identifying type 1 ROP was 100% similar to that reported in developed countries; however, its specificity was low at 31.5%. Tweaking of the algorithm based on the population may increase the specificity and promote the practical utility of this non-invasive screening tool for ophthalmologists and neonatologists in this population.
Collapse
Affiliation(s)
- Lina H Raffa
- Department of Ophthalmology, King Abdulaziz University, Jeddah, Kingdom of Saudi Arabia. E-mail.
| | | | | | | |
Collapse
|
33
|
Horton MB, Brady CJ, Cavallerano J, Abramoff M, Barker G, Chiang MF, Crockett CH, Garg S, Karth P, Liu Y, Newman CD, Rathi S, Sheth V, Silva P, Stebbins K, Zimmer-Galler I. Practice Guidelines for Ocular Telehealth-Diabetic Retinopathy, Third Edition. Telemed J E Health 2020; 26:495-543. [PMID: 32209018 PMCID: PMC7187969 DOI: 10.1089/tmj.2020.0006] [Citation(s) in RCA: 32] [Impact Index Per Article: 8.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/09/2020] [Revised: 01/11/2020] [Accepted: 01/11/2020] [Indexed: 12/24/2022] Open
Abstract
Contributors The following document and appendices represent the third edition of the Practice Guidelines for Ocular Telehealth-Diabetic Retinopathy. These guidelines were developed by the Diabetic Retinopathy Telehealth Practice Guidelines Working Group. This working group consisted of a large number of subject matter experts in clinical applications for telehealth in ophthalmology. The editorial committee consisted of Mark B. Horton, OD, MD, who served as working group chair and Christopher J. Brady, MD, MHS, and Jerry Cavallerano, OD, PhD, who served as cochairs. The writing committees were separated into seven different categories. They are as follows: 1.Clinical/operational: Jerry Cavallerano, OD, PhD (Chair), Gail Barker, PhD, MBA, Christopher J. Brady, MD, MHS, Yao Liu, MD, MS, Siddarth Rathi, MD, MBA, Veeral Sheth, MD, MBA, Paolo Silva, MD, and Ingrid Zimmer-Galler, MD. 2.Equipment: Veeral Sheth, MD (Chair), Mark B. Horton, OD, MD, Siddarth Rathi, MD, MBA, Paolo Silva, MD, and Kristen Stebbins, MSPH. 3.Quality assurance: Mark B. Horton, OD, MD (Chair), Seema Garg, MD, PhD, Yao Liu, MD, MS, and Ingrid Zimmer-Galler, MD. 4.Glaucoma: Yao Liu, MD, MS (Chair) and Siddarth Rathi, MD, MBA. 5.Retinopathy of prematurity: Christopher J. Brady, MD, MHS (Chair) and Ingrid Zimmer-Galler, MD. 6.Age-related macular degeneration: Christopher J. Brady, MD, MHS (Chair) and Ingrid Zimmer-Galler, MD. 7.Autonomous and computer assisted detection, classification and diagnosis of diabetic retinopathy: Michael Abramoff, MD, PhD (Chair), Michael F. Chiang, MD, and Paolo Silva, MD.
Collapse
Affiliation(s)
- Mark B. Horton
- Indian Health Service-Joslin Vision Network (IHS-JVN) Teleophthalmology Program, Phoenix Indian Medical Center, Phoenix, Arizona
| | - Christopher J. Brady
- Division of Ophthalmology, Department of Surgery, Larner College of Medicine, University of Vermont, Burlington, Vermont
| | - Jerry Cavallerano
- Beetham Eye Institute, Joslin Diabetes Center, Massachusetts
- Department of Ophthalmology, Harvard Medical School, Boston, Massachusetts
| | - Michael Abramoff
- Department of Ophthalmology and Visual Sciences, The University of Iowa, Iowa City, Iowa
- Department of Biomedical Engineering, and The University of Iowa, Iowa City, Iowa
- Department of Electrical and Computer Engineering, The University of Iowa, Iowa City, Iowa
- Department of Ophthalmology, Stephen A. Wynn Institute for Vision Research, The University of Iowa, Iowa City, Iowa
- Iowa City VA Health Care System, Iowa City, Iowa
- IDx, Coralville, Iowa
| | - Gail Barker
- Arizona Telemedicine Program, The University of Arizona, Phoenix, Arizona
| | - Michael F. Chiang
- Department of Ophthalmology, Casey Eye Institute, Oregon Health and Science University, Portland, Oregon
- Department of Medical Informatics and Clinical Epidemiology, Oregon Health and Science University, Portland, Oregon
| | | | - Seema Garg
- Department of Ophthalmology, University of North Carolina, Chapel Hill, North Carolina
| | | | - Yao Liu
- Department of Ophthalmology and Visual Sciences, University of Wisconsin-Madison, Madison, Wisconsin
| | | | - Siddarth Rathi
- Department of Ophthalmology, NYU Langone Health, New York, New York
| | - Veeral Sheth
- University Retina and Macula Associates, University of Illinois at Chicago, Chicago, Illinois
| | - Paolo Silva
- Beetham Eye Institute, Joslin Diabetes Center, Massachusetts
- Department of Ophthalmology, Harvard Medical School, Boston, Massachusetts
| | - Kristen Stebbins
- Vision Care Department, Hillrom, Skaneateles Falls, New York, New York
| | | |
Collapse
|
34
|
Li MD, Chang K, Bearce B, Chang CY, Huang AJ, Campbell JP, Brown JM, Singh P, Hoebel KV, Erdoğmuş D, Ioannidis S, Palmer WE, Chiang MF, Kalpathy-Cramer J. Siamese neural networks for continuous disease severity evaluation and change detection in medical imaging. NPJ Digit Med 2020; 3:48. [PMID: 32258430 PMCID: PMC7099081 DOI: 10.1038/s41746-020-0255-1] [Citation(s) in RCA: 40] [Impact Index Per Article: 10.0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/21/2019] [Accepted: 03/06/2020] [Indexed: 01/01/2023] Open
Abstract
Using medical images to evaluate disease severity and change over time is a routine and important task in clinical decision making. Grading systems are often used, but are unreliable as domain experts disagree on disease severity category thresholds. These discrete categories also do not reflect the underlying continuous spectrum of disease severity. To address these issues, we developed a convolutional Siamese neural network approach to evaluate disease severity at single time points and change between longitudinal patient visits on a continuous spectrum. We demonstrate this in two medical imaging domains: retinopathy of prematurity (ROP) in retinal photographs and osteoarthritis in knee radiographs. Our patient cohorts consist of 4861 images from 870 patients in the Imaging and Informatics in Retinopathy of Prematurity (i-ROP) cohort study and 10,012 images from 3021 patients in the Multicenter Osteoarthritis Study (MOST), both of which feature longitudinal imaging data. Multiple expert clinician raters ranked 100 retinal images and 100 knee radiographs from excluded test sets for severity of ROP and osteoarthritis, respectively. The Siamese neural network output for each image in comparison to a pool of normal reference images correlates with disease severity rank (ρ = 0.87 for ROP and ρ = 0.89 for osteoarthritis), both within and between the clinical grading categories. Thus, this output can represent the continuous spectrum of disease severity at any single time point. The difference in these outputs can be used to show change over time. Alternatively, paired images from the same patient at two time points can be directly compared using the Siamese neural network, resulting in an additional continuous measure of change between images. Importantly, our approach does not require manual localization of the pathology of interest and requires only a binary label for training (same versus different). The location of disease and site of change detected by the algorithm can be visualized using an occlusion sensitivity map-based approach. For a longitudinal binary change detection task, our Siamese neural networks achieve test set receiving operator characteristic area under the curves (AUCs) of up to 0.90 in evaluating ROP or knee osteoarthritis change, depending on the change detection strategy. The overall performance on this binary task is similar compared to a conventional convolutional deep-neural network trained for multi-class classification. Our results demonstrate that convolutional Siamese neural networks can be a powerful tool for evaluating the continuous spectrum of disease severity and change in medical imaging.
Collapse
Affiliation(s)
- Matthew D. Li
- Athinoula A. Martinos Center for Biomedical Imaging, Department of Radiology, Massachusetts General Hospital, Boston, MA USA
| | - Ken Chang
- Athinoula A. Martinos Center for Biomedical Imaging, Department of Radiology, Massachusetts General Hospital, Boston, MA USA
| | - Ben Bearce
- Athinoula A. Martinos Center for Biomedical Imaging, Department of Radiology, Massachusetts General Hospital, Boston, MA USA
| | - Connie Y. Chang
- Division of Musculoskeletal Imaging and Intervention, Department of Radiology, Massachusetts General Hospital, Boston, MA USA
| | - Ambrose J. Huang
- Division of Musculoskeletal Imaging and Intervention, Department of Radiology, Massachusetts General Hospital, Boston, MA USA
| | - J. Peter Campbell
- Department of Ophthalmology, Casey Eye Institute, Oregon Health & Science University, Portland, OR USA
| | - James M. Brown
- School of Computer Science, University of Lincoln, Lincoln, UK
| | - Praveer Singh
- Athinoula A. Martinos Center for Biomedical Imaging, Department of Radiology, Massachusetts General Hospital, Boston, MA USA
| | - Katharina V. Hoebel
- Athinoula A. Martinos Center for Biomedical Imaging, Department of Radiology, Massachusetts General Hospital, Boston, MA USA
| | - Deniz Erdoğmuş
- Department of Electrical and Computer Engineering, Northeastern University, Boston, MA USA
| | - Stratis Ioannidis
- Department of Electrical and Computer Engineering, Northeastern University, Boston, MA USA
| | - William E. Palmer
- Division of Musculoskeletal Imaging and Intervention, Department of Radiology, Massachusetts General Hospital, Boston, MA USA
| | - Michael F. Chiang
- Department of Ophthalmology, Casey Eye Institute, Oregon Health & Science University, Portland, OR USA
- Department of Medical Informatics and Clinical Epidemiology, Oregon Health & Science University, Portland, OR USA
| | - Jayashree Kalpathy-Cramer
- Athinoula A. Martinos Center for Biomedical Imaging, Department of Radiology, Massachusetts General Hospital, Boston, MA USA
- MGH and BWH Center for Clinical Data Science, Massachusetts General Hospital, Boston, MA USA
| |
Collapse
|
35
|
Brady CJ, D'Amico S, Campbell JP. Telemedicine for Retinopathy of Prematurity. Telemed J E Health 2020; 26:556-564. [PMID: 32209016 DOI: 10.1089/tmj.2020.0010] [Citation(s) in RCA: 16] [Impact Index Per Article: 4.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/24/2022] Open
Abstract
Background: Retinopathy of prematurity (ROP) is a disease of the retinal vasculature that remains a leading cause of childhood blindness worldwide despite improvements in the systemic care of premature newborns. Screening for ROP is effective and cost-effective, but in many areas, access to skilled examiners to conduct dilated examinations is poor. Remote screening with retinal photography is an alternative strategy that may allow for improved ROP care. Methods: The current literature was reviewed to find clinical trials and expert consensus documents on the state-of-the-art of telemedicine for ROP. Results: Several studies have confirmed the utility of telemedicine for ROP. In addition, several clinical studies have reported favorable long-term results. Many investigators have reinforced the need for detailed protocols on image acquisition and image interpretation. Conclusions: Telemedicine for ROP appears to be a viable alternative to live ophthalmoscopic examinations in many circumstances. Standardization and documentation afforded by telemedicine may provide additional benefits to providers and their patients. With continued improvements in image quality and affordability of imaging systems as well as improved automated image interpretation tools anticipated in the near future, telemedicine for ROP is expected to play an expanding role for a uniquely vulnerable patient population.
Collapse
Affiliation(s)
- Christopher J Brady
- Division of Ophthalmology, Department of Surgery, Larner College of Medicine, University of Vermont, Burlington, Vermont
| | - Samantha D'Amico
- Division of Ophthalmology, Department of Surgery, University of Vermont Medical Center, Burlington, Vermont
| | - J Peter Campbell
- Casey Eye Institute, Oregon Health and Science University, Portland, Oregon
| |
Collapse
|
36
|
Yildiz VM, Tian P, Yildiz I, Brown JM, Kalpathy-Cramer J, Dy J, Ioannidis S, Erdogmus D, Ostmo S, Kim SJ, Chan RVP, Campbell JP, Chiang MF. Plus Disease in Retinopathy of Prematurity: Convolutional Neural Network Performance Using a Combined Neural Network and Feature Extraction Approach. Transl Vis Sci Technol 2020; 9:10. [PMID: 32704416 PMCID: PMC7346878 DOI: 10.1167/tvst.9.2.10] [Citation(s) in RCA: 23] [Impact Index Per Article: 5.8] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/30/2023] Open
Abstract
Purpose Retinopathy of prematurity (ROP), a leading cause of childhood blindness, is diagnosed by clinical ophthalmoscopic examinations or reading retinal images. Plus disease, defined as abnormal tortuosity and dilation of the posterior retinal blood vessels, is the most important feature to determine treatment-requiring ROP. We aimed to create a complete, publicly available and feature-extraction-based pipeline, I-ROP ASSIST, that achieves convolutional neural network (CNN)-like performance when diagnosing plus disease from retinal images. Methods We developed two datasets containing 100 and 5512 posterior retinal images, respectively. After segmenting retinal vessels, we detected the vessel centerlines. Then, we extracted features relevant to ROP, including tortuosity and dilation measures, and used these features in the classifiers including logistic regression, support vector machine and neural networks to assess a severity score for the input. We tested our system with fivefold cross-validation and calculated the area under the curve (AUC) metric for each classifier and dataset. Results For predicting plus versus not-plus categories, we achieved 99% and 94% AUC on the first and second datasets, respectively. For predicting pre-plus or worse versus normal categories, we achieved 99% and 88% AUC on the first and second datasets, respectively. The CNN method achieved 98% and 94% for predicting two categories on the second dataset. Conclusions Our system combining automatic retinal vessel segmentation, tracing, feature extraction and classification is able to diagnose plus disease in ROP with CNN-like performance. Translational Relevance The high performance of I-ROP ASSIST suggests potential applications in automated and objective diagnosis of plus disease.
Collapse
Affiliation(s)
- Veysi M Yildiz
- Cognitive Systems Laboratory, Northeastern University, Boston, MA, USA
| | - Peng Tian
- Cognitive Systems Laboratory, Northeastern University, Boston, MA, USA
| | - Ilkay Yildiz
- Cognitive Systems Laboratory, Northeastern University, Boston, MA, USA
| | - James M Brown
- Department of Computer Science, University of Lincoln, Lincoln, UK
| | - Jayashree Kalpathy-Cramer
- Department of Radiology, Athinoula A. Martinos Center for Biomedical Imaging, Massachusetts General Hospital, Charlestown, MA, USA
| | - Jennifer Dy
- Cognitive Systems Laboratory, Northeastern University, Boston, MA, USA
| | - Stratis Ioannidis
- Cognitive Systems Laboratory, Northeastern University, Boston, MA, USA
| | - Deniz Erdogmus
- Cognitive Systems Laboratory, Northeastern University, Boston, MA, USA
| | - Susan Ostmo
- Department of Ophthalmology, Casey Eye Institute, Oregon Health & Science University, Portland, OR, USA
| | - Sang Jin Kim
- Sungkyunkwan University School of Medicine, Seoul, South Korea
| | - R V Paul Chan
- Eye and Ear Infirmary, University of Illinois at Chicago, Chicago, IL, USA
| | - J Peter Campbell
- Department of Ophthalmology, Casey Eye Institute, Oregon Health & Science University, Portland, OR, USA
| | - Michael F Chiang
- Department of Ophthalmology, Casey Eye Institute, Oregon Health & Science University, Portland, OR, USA
| | | |
Collapse
|
37
|
Aggressive Posterior Retinopathy of Prematurity: Clinical and Quantitative Imaging Features in a Large North American Cohort. Ophthalmology 2020; 127:1105-1112. [PMID: 32197913 DOI: 10.1016/j.ophtha.2020.01.052] [Citation(s) in RCA: 16] [Impact Index Per Article: 4.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/17/2019] [Revised: 12/06/2019] [Accepted: 01/29/2020] [Indexed: 01/24/2023] Open
Abstract
PURPOSE Aggressive posterior retinopathy of prematurity (AP-ROP) is a vision-threatening disease with a significant rate of progression to retinal detachment. The purpose of this study was to characterize AP-ROP quantitatively by demographics, rate of disease progression, and a deep learning-based vascular severity score. DESIGN Retrospective analysis. PARTICIPANTS The Imaging and Informatics in ROP cohort from 8 North American centers, consisting of 947 patients and 5945 clinical eye examinations with fundus images, was used. Pretreatment eyes were categorized by disease severity: none, mild, type 2 or pre-plus, treatment-requiring (TR) without AP-ROP, TR with AP-ROP. Analyses compared TR with AP-ROP and TR without AP-ROP to investigate differences between AP-ROP and other TR disease. METHODS A reference standard diagnosis was generated for each eye examination using previously published methods combining 3 independent image-based gradings and 1 ophthalmoscopic grading. All fundus images were analyzed using a previously published deep learning system and were assigned a score from 1 through 9. MAIN OUTCOME MEASURES Birth weight, gestational age, postmenstrual age, and vascular severity score. RESULTS Infants who demonstrated AP-ROP were more premature by birth weight (617 g vs. 679 g; P = 0.01) and gestational age (24.3 weeks vs. 25.0 weeks; P < 0.01) and reached peak severity at an earlier postmenstrual age (34.7 weeks vs. 36.9 weeks; P < 0.001) compared with infants with TR without AP-ROP. The mean vascular severity score was greatest in TR with AP-ROP infants compared with TR without AP-ROP infants (8.79 vs. 7.19; P < 0.001). Analyzing the severity score over time, the rate of progression was fastest in infants with AP-ROP (P < 0.002 at 30-32 weeks). CONCLUSIONS Premature infants in North America with AP-ROP are born younger and demonstrate disease earlier than infants with less severe ROP. Disease severity is quantifiable with a deep learning-based score, which correlates with clinically identified categories of disease, including AP-ROP. The rate of progression to peak disease is greatest in eyes that demonstrate AP-ROP compared with other treatment-requiring eyes. Analysis of quantitative characteristics of AP-ROP may help improve diagnosis and treatment of an aggressive, vision-threatening form of ROP.
Collapse
|
38
|
Smith LEH, Hellström A, Stahl A, Fielder A, Chambers W, Moseley J, Toth C, Wallace D, Darlow BA, Aranda JV, Hallberg B, Davis JM. Development of a Retinopathy of Prematurity Activity Scale and Clinical Outcome Measures for Use in Clinical Trials. JAMA Ophthalmol 2019; 137:305-311. [PMID: 30543348 DOI: 10.1001/jamaophthalmol.2018.5984] [Citation(s) in RCA: 12] [Impact Index Per Article: 2.4] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/14/2022]
Abstract
Importance To facilitate drug and device development for neonates, the International Neonatal Consortium brings together key stakeholders, including pharmaceutical companies, practitioners, regulators, funding agencies, scientists, and families, to address the need for objective, standardized clinical trial outcome measurements to fulfill regulatory requirements. Retinopathy of prematurity (ROP) is a disease that affects preterm neonates. The current International Classification of Retinopathy of Prematurity does not take into account all of the characteristics of ROP and does not adequately discriminate small changes in disease after treatment. These factors are critical for evaluating outcomes in clinical trials. Observations There is need for an updated ROP acute disease activity and structure scale as well as end-stage structure and ophthalmologic outcome measures designed for use at different ages. The scale and measures, based on current diagnostic methods and treatments, could be used as a guideline for clinical intervention trials. The scale is intended to be validated against retrospective data and revised for use in future trials. An iterative revision process can be accomplished if new measures are added to clinical trials and evaluated at the end of each trial for prognostic value. The new measures would then be incorporated into a new version of the activity scale and the outcome measures revised. Conclusions and Relevance An ROP activity scale and outcome measures to obtain the most robust and discriminatory data for clinical trials are needed. The scales should be dynamic and modified as knowledge and imaging modalities improve and then validated using data from well-documented clinical trials. This approach is relevant to improving clinical trial data quality.
Collapse
Affiliation(s)
- Lois E H Smith
- Department of Ophthalmology, Harvard Medical School, Boston Children's Hospital, Boston, Massachusetts
| | - Ann Hellström
- Department of Clinical Neuroscience at Institute of Neuroscience and Physiology, University of Göteborg, Göteborg, Sweden
| | - Andreas Stahl
- Department of Ophthalmology, University of Freiburg, Freiburg, Germany
| | - Alistair Fielder
- Department of Optometry & Visual Science, City University of London, London, United Kingdom
| | - Wiley Chambers
- Division of Transplant and Ophthalmology Products, US Food and Drug Administration, Bethesda, Maryland
| | - Jane Moseley
- Human Medicines Research and Development Support Division, European Medicines Agency, London, United Kingdom
| | - Cynthia Toth
- Department of Ophthalmology, Duke University, Durham, North Carolina
| | - David Wallace
- Department of Ophthalmology, Indiana University, Indianapolis
| | - Brian A Darlow
- Department of Pathology and Biomedical Science, University of Otago, Christchurch, New Zealand
| | - Jacob V Aranda
- Department of Ophthalmology, State University of New York Downstate Medical Center, Brooklyn
| | - Boubou Hallberg
- Division of Paediatrics, Karolinska University Hospital, Stockholm, Sweden
| | - Jonathan M Davis
- The Floating Hospital for Children, Tufts Medical Center, Boston, Massachusetts.,Department of Pediatrics, Tufts University School of Medicine, Boston, Massachusetts
| | | |
Collapse
|
39
|
Tan Z, Simkin S, Lai C, Dai S. Deep Learning Algorithm for Automated Diagnosis of Retinopathy of Prematurity Plus Disease. Transl Vis Sci Technol 2019; 8:23. [PMID: 31819832 PMCID: PMC6892443 DOI: 10.1167/tvst.8.6.23] [Citation(s) in RCA: 43] [Impact Index Per Article: 8.6] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/16/2019] [Accepted: 09/30/2019] [Indexed: 12/20/2022] Open
Abstract
PURPOSE This study describes the initial development of a deep learning algorithm, ROP.AI, to automatically diagnose retinopathy of prematurity (ROP) plus disease in fundal images. METHODS ROP.AI was trained using 6974 fundal images from Australasian image databases. Each image was given a diagnosis as part of real-world routine ROP screening and classified as normal or plus disease. The algorithm was trained using 80% of the images and validated against the remaining 20% within a hold-out test set. Performance in diagnosing plus disease was evaluated against an external set of 90 images. Performance in detecting pre-plus disease was also tested. As a screening tool, the algorithm's operating point was optimized for sensitivity and negative predictive value, and its performance reevaluated. RESULTS For plus disease diagnosis within the 20% hold-out test set, the algorithm achieved a 96.6% sensitivity, 98.0% specificity, and 97.3% ± 0.7% accuracy. Area under the receiver operating characteristic curve was 0.993. Within the independent test set, the algorithm achieved a 93.9% sensitivity, 80.7% specificity, and 95.8% negative predictive value. For detection of pre-plus and plus disease, the algorithm achieved 81.4% sensitivity, 80.7% specificity, and 80.7% negative predictive value. Following the identification of an optimized operating point, the algorithm diagnosed plus disease with a 97.0% sensitivity and 97.8% negative predictive value. CONCLUSIONS ROP.AI is a deep learning algorithm able to automatically diagnose ROP plus disease with high sensitivity and negative predictive value. TRANSLATIONAL RELEVANCE In the context of increasing global disease burden, future development may improve access to ROP diagnosis and care.
Collapse
Affiliation(s)
- Zachary Tan
- Save Sight Institute, The University of Sydney, Sydney, New South Wales, Australia
- St Vincent's Hospital Sydney, Sydney, New South Wales, Australia
- Faculty of Medicine, The University of Queensland, Brisbane, Queensland, Australia
| | - Samantha Simkin
- Department of Ophthalmology, New Zealand National Eye Centre, Faculty of Medical and Health Sciences, The University of Auckland, Auckland, New Zealand
| | - Connie Lai
- Queen Mary Hospital, Hong Kong, China
- Department of Ophthalmology, The University of Hong Kong, Hong Kong, China
| | - Shuan Dai
- Department of Ophthalmology, New Zealand National Eye Centre, Faculty of Medical and Health Sciences, The University of Auckland, Auckland, New Zealand
- Department of Ophthalmology, Queensland Children's Hospital, Brisbane, Queensland, Australia
| |
Collapse
|
40
|
Kim SJ, Campbell JP, Kalpathy-Cramer J, Ostmo S, Jonas KE, Choi D, Chan RVP, Chiang MF. Accuracy and Reliability of Eye-Based vs Quadrant-Based Diagnosis of Plus Disease in Retinopathy of Prematurity. JAMA Ophthalmol 2019; 136:648-655. [PMID: 29710185 DOI: 10.1001/jamaophthalmol.2018.1195] [Citation(s) in RCA: 15] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/10/2023]
Abstract
Importance Presence of plus disease in retinopathy of prematurity is the most critical element in identifying treatment-requiring disease. However, there is significant variability in plus disease diagnosis. In particular, plus disease has been defined as 2 or more quadrants of vascular abnormality, and it is not clear whether it is more reliably and accurately diagnosed by eye-based assessment of overall retinal appearance or by quadrant-based assessment combining grades of 4 individual quadrants. Objective To compare eye-based vs quadrant-based diagnosis of plus disease and to provide insight for ophthalmologists about the diagnostic process. Design, Setting, and Participants In this multicenter cohort study, we developed a database of 197 wide-angle retinal images from 141 preterm infants from neonatal intensive care units at 9 academic institutions (enrolled from July 2011 to December 2016). Each image was assigned a reference standard diagnosis based on consensus image-based and clinical diagnosis. Data analysis was performed from February 2017 to September 2017. Interventions Six graders independently diagnosed each of the 4 quadrants (cropped images) of the 197 eyes (quadrant-based diagnosis) as well as the entire image (eye-based diagnosis). Images were displayed individually, in random order. Quadrant-based diagnosis of plus disease was made when 2 or more quadrants were diagnosed as indicating plus disease by combining grades of individual quadrants post hoc. Main Outcomes and Measures Intragrader and intergrader reliability (absolute agreement and κ statistic) and accuracy compared with the reference standard diagnosis. Results Of the 141 included preterm infants, 65 (46.1%) were female and 116 (82.3%) white, and the mean (SD) gestational age was 27.0 (2.6) weeks. There was variable agreement between eye-based and quadrant-based diagnosis among the 6 graders (Cohen κ range, 0.32-0.75). Four graders showed underdiagnosis of plus disease with quadrant-based diagnosis compared with eye-based diagnosis (by McNemar test). Intergrader agreement of quadrant-based diagnosis was lower than that of eye-based diagnosis (Fleiss κ, 0.75 [95% CI, 0.71-0.78] vs 0.55 [95% CI, 0.51-0.59]). The accuracy of eye-based diagnosis compared with the reference standard diagnosis was substantial to near-perfect, whereas that of quadrant-based plus disease diagnosis was only moderate to substantial for each grader. Conclusions and Relevance Graders had lower reliability and accuracy using quadrant-based diagnosis combining grades of individual quadrants than with eye-based diagnosis, suggesting that eye-based diagnosis has advantages over quadrant-based diagnosis. This has implications for more precise definitions of plus disease regarding the criterion of 2 or more quadrants, clinical care, computer-based image analysis, and education for all ophthalmologists who manage retinopathy of prematurity.
Collapse
Affiliation(s)
- Sang Jin Kim
- Department of Ophthalmology, Casey Eye Institute, Oregon Health and Science University, Portland.,Department of Ophthalmology, Samsung Medical Center, Sungkyunkwan University School of Medicine, Seoul, Republic of Korea
| | - J Peter Campbell
- Department of Ophthalmology, Casey Eye Institute, Oregon Health and Science University, Portland
| | - Jayashree Kalpathy-Cramer
- Athinoula A. Martinos Center for Biomedical Imaging, Department of Radiology, Massachusetts General Hospital, Charlestown.,Massachusetts General Hospital and Brigham and Women's Hospital Center for Clinical Data Science, Boston
| | - Susan Ostmo
- Department of Ophthalmology, Casey Eye Institute, Oregon Health and Science University, Portland
| | - Karyn E Jonas
- Department of Ophthalmology and Visual Sciences, Illinois Eye and Ear Infirmary, University of Illinois at Chicago
| | - Dongseok Choi
- Department of Ophthalmology, Casey Eye Institute, Oregon Health and Science University, Portland.,Graduate School of Dentistry, Kyung Hee University, Seoul, Republic of Korea
| | - R V Paul Chan
- Department of Ophthalmology and Visual Sciences, Illinois Eye and Ear Infirmary, University of Illinois at Chicago.,Center for Global Health, College of Medicine, University of Illinois at Chicago
| | - Michael F Chiang
- Department of Ophthalmology, Casey Eye Institute, Oregon Health and Science University, Portland.,Department of Medical Informatics and Clinical Epidemiology, Oregon Health and Science University, Portland
| | | |
Collapse
|
41
|
Biten H, Redd TK, Moleta C, Campbell JP, Ostmo S, Jonas K, Chan RVP, Chiang MF. Diagnostic Accuracy of Ophthalmoscopy vs Telemedicine in Examinations for Retinopathy of Prematurity. JAMA Ophthalmol 2019; 136:498-504. [PMID: 29621387 DOI: 10.1001/jamaophthalmol.2018.0649] [Citation(s) in RCA: 37] [Impact Index Per Article: 7.4] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/10/2023]
Abstract
Importance Examinations for retinopathy of prematurity (ROP) are typically performed using binocular indirect ophthalmoscopy. Telemedicine studies have traditionally assessed the accuracy of telemedicine compared with ophthalmoscopy as a criterion standard. However, it is not known whether ophthalmoscopy is truly more accurate than telemedicine. Objective To directly compare the accuracy and sensitivity of ophthalmoscopy vs telemedicine in diagnosing ROP using a consensus reference standard. Design, Setting, and Participants This multicenter prospective study conducted between July 1, 2011, and November 30, 2014, at 7 neonatal intensive care units and academic ophthalmology departments in the United States and Mexico included 281 premature infants who met the screening criteria for ROP. Exposures Each examination consisted of 1 eye undergoing binocular indirect ophthalmoscopy by an experienced clinician followed by remote image review of wide-angle fundus photographs by 3 independent telemedicine graders. Main Outcomes and Measures Results of both examination methods were combined into a consensus reference standard diagnosis. The agreement of both ophthalmoscopy and telemedicine was compared with this standard, using percentage agreement and weighted κ statistics. Results Among the 281 infants in the study (127 girls and 154 boys; mean [SD] gestational age, 27.1 [2.4] weeks), a total of 1553 eye examinations were classified using both ophthalmoscopy and telemedicine. Ophthalmoscopy and telemedicine each had similar sensitivity for zone I disease (78% [95% CI, 71%-84%] vs 78% [95% CI, 73%-83%]; P > .99 [n = 165]), plus disease (74% [95% CI, 61%-87%] vs 79% [95% CI, 72%-86%]; P = .41 [n = 50]), and type 2 ROP (stage 3, zone I, or plus disease: 86% [95% CI, 80%-92%] vs 79% [95% CI, 75%-83%]; P = .10 [n = 251]), but ophthalmoscopy was slightly more sensitive in identifying stage 3 disease (85% [95% CI, 79%-91%] vs 73% [95% CI, 67%-78%]; P = .004 [n = 136]). Conclusions and Relevance No difference was found in overall accuracy between ophthalmoscopy and telemedicine for the detection of clinically significant ROP, although, on average, ophthalmoscopy had slightly higher accuracy for the diagnosis of zone III and stage 3 ROP. With the caveat that there was variable accuracy between examiners using both modalities, these results support the use of telemedicine for the diagnosis of clinically significant ROP.
Collapse
Affiliation(s)
- Hilal Biten
- Department of Ophthalmology, Casey Eye Institute, Oregon Health & Science University, Portland
| | - Travis K Redd
- Department of Ophthalmology, Casey Eye Institute, Oregon Health & Science University, Portland
| | - Chace Moleta
- Department of Ophthalmology, Casey Eye Institute, Oregon Health & Science University, Portland
| | - J Peter Campbell
- Department of Ophthalmology, Casey Eye Institute, Oregon Health & Science University, Portland
| | - Susan Ostmo
- Department of Ophthalmology, Casey Eye Institute, Oregon Health & Science University, Portland
| | - Karyn Jonas
- Department of Ophthalmology and Visual Sciences, Illinois Eye and Ear Infirmary, University of Illinois at Chicago
| | - R V Paul Chan
- Department of Ophthalmology and Visual Sciences, Illinois Eye and Ear Infirmary, University of Illinois at Chicago.,Center for Global Health, College of Medicine, University of Illinois at Chicago
| | - Michael F Chiang
- Department of Ophthalmology, Casey Eye Institute, Oregon Health & Science University, Portland.,Department of Medical Informatics and Clinical Epidemiology, Oregon Health & Science University, Portland
| | | |
Collapse
|
42
|
Gupta K, Campbell JP, Taylor S, Brown JM, Ostmo S, Chan RVP, Dy J, Erdogmus D, Ioannidis S, Kalpathy-Cramer J, Kim SJ, Chiang MF. A Quantitative Severity Scale for Retinopathy of Prematurity Using Deep Learning to Monitor Disease Regression After Treatment. JAMA Ophthalmol 2019; 137:1029-1036. [PMID: 31268499 PMCID: PMC6613298 DOI: 10.1001/jamaophthalmol.2019.2442] [Citation(s) in RCA: 50] [Impact Index Per Article: 10.0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/20/2018] [Accepted: 04/14/2019] [Indexed: 01/10/2023]
Abstract
Importance Retinopathy of prematurity (ROP) is a leading cause of childhood blindness worldwide, but treatment failure and disease recurrence are important causes of adverse outcomes in patients with treatment-requiring ROP (TR-ROP). Objectives To apply an automated ROP vascular severity score obtained using a deep learning algorithm and to assess its utility for objectively monitoring ROP regression after treatment. Design, Setting, and Participants This retrospective cohort study used data from the Imaging and Informatics in ROP consortium, which comprises 9 tertiary referral centers in North America that screen high volumes of at-risk infants for ROP. Images of 5255 clinical eye examinations from 871 infants performed between July 2011 and December 2016 were assessed for eligibility in the present study. The disease course was assessed with time across the numerous examinations for patients with TR-ROP. Infants born prematurely meeting screening criteria for ROP who developed TR-ROP and who had images captured within 4 weeks before and after treatment as well as at the time of treatment were included. Main Outcomes and Measures The primary outcome was mean (SD) ROP vascular severity score before, at time of, and after treatment. A deep learning classifier was used to assign a continuous ROP vascular severity score, which ranged from 1 (normal) to 9 (most severe), at each examination. A secondary outcome was the difference in ROP vascular severity score among eyes treated with laser or the vascular endothelial growth factor antagonist bevacizumab. Differences between groups for both outcomes were assessed using unpaired 2-tailed t tests with Bonferroni correction. Results Of 5255 examined eyes, 91 developed TR-ROP, of which 46 eyes met the inclusion criteria based on the available images. The mean (SD) birth weight of those patients was 653 (185) g, with a mean (SD) gestational age of 24.9 (1.3) weeks. The mean (SD) ROP vascular severity scores significantly increased 2 weeks prior to treatment (4.19 [1.75]), peaked at treatment (7.43 [1.89]), and decreased for at least 2 weeks after treatment (4.00 [1.88]) (all P < .001). Eyes requiring retreatment with laser had higher ROP vascular severity scores at the time of initial treatment compared with eyes receiving a single treatment (P < .001). Conclusions and Relevance This quantitative ROP vascular severity score appears to consistently reflect clinical disease progression and posttreatment regression in eyes with TR-ROP. These study results may have implications for the monitoring of patients with ROP for treatment failure and disease recurrence and for determining the appropriate level of disease severity for primary treatment in eyes with aggressive disease.
Collapse
Affiliation(s)
- Kishan Gupta
- Department of Ophthalmology, Casey Eye Institute, Oregon Health & Science University, Portland
| | - J. Peter Campbell
- Department of Ophthalmology, Casey Eye Institute, Oregon Health & Science University, Portland
| | - Stanford Taylor
- Department of Ophthalmology, Casey Eye Institute, Oregon Health & Science University, Portland
| | - James M. Brown
- Athinoula A. Martinos Center for Biomedical Imaging, Department of Radiology, Massachusetts General Hospital, Charlestown
| | - Susan Ostmo
- Department of Ophthalmology, Casey Eye Institute, Oregon Health & Science University, Portland
| | - R. V. Paul Chan
- Department of Ophthalmology and Visual Sciences, Illinois Eye and Ear Infirmary, University of Illinois at Chicago
| | - Jennifer Dy
- Department of Electrical and Computer Engineering, Northeastern University, Boston, Massachusetts
| | - Deniz Erdogmus
- Department of Electrical and Computer Engineering, Northeastern University, Boston, Massachusetts
| | - Stratis Ioannidis
- Department of Electrical and Computer Engineering, Northeastern University, Boston, Massachusetts
| | - Jayashree Kalpathy-Cramer
- Athinoula A. Martinos Center for Biomedical Imaging, Department of Radiology, Massachusetts General Hospital, Charlestown
- Massachusetts General Hospital & Brigham and Women’s Hospital Center for Clinical Data Science, Boston
| | - Sang J. Kim
- Department of Ophthalmology, Casey Eye Institute, Oregon Health & Science University, Portland
- Department of Ophthalmology, Samsung Medical Center, Sungkyunkwan University School of Medicine, Seoul, Republic of Korea
| | - Michael F. Chiang
- Department of Ophthalmology, Casey Eye Institute, Oregon Health & Science University, Portland
- Department of Medical Informatics and Clinical Epidemiology, Oregon Health & Science University, Portland
| |
Collapse
|
43
|
Taylor S, Brown JM, Gupta K, Campbell JP, Ostmo S, Chan RVP, Dy J, Erdogmus D, Ioannidis S, Kim SJ, Kalpathy-Cramer J, Chiang MF. Monitoring Disease Progression With a Quantitative Severity Scale for Retinopathy of Prematurity Using Deep Learning. JAMA Ophthalmol 2019; 137:1022-1028. [PMID: 31268518 PMCID: PMC6613341 DOI: 10.1001/jamaophthalmol.2019.2433] [Citation(s) in RCA: 60] [Impact Index Per Article: 12.0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/20/2018] [Accepted: 04/14/2019] [Indexed: 01/08/2023]
Abstract
Importance Retinopathy of prematurity (ROP) is a leading cause of childhood blindness worldwide, but clinical diagnosis is subjective and qualitative. Objective To describe a quantitative ROP severity score derived using a deep learning algorithm designed to evaluate plus disease and to assess its utility for objectively monitoring ROP progression. Design, Setting, and Participants This retrospective cohort study included images from 5255 clinical examinations of 871 premature infants who met the ROP screening criteria of the Imaging and Informatics in ROP (i-ROP) Consortium, which comprises 9 tertiary care centers in North America, from July 1, 2011, to December 31, 2016. Data analysis was performed from July 2017 to May 2018. Exposure A deep learning algorithm was used to assign a continuous ROP vascular severity score from 1 (most normal) to 9 (most severe) at each examination based on a single posterior photograph compared with a reference standard diagnosis (RSD) simplified into 4 categories: no ROP, mild ROP, type 2 ROP or pre-plus disease, or type 1 ROP. Disease course was assessed longitudinally across multiple examinations for all patients. Main Outcomes and Measures Mean ROP vascular severity score progression over time compared with the RSD. Results A total of 5255 clinical examinations from 871 infants (mean [SD] gestational age, 27.0 [2.0] weeks; 493 [56.6%] male; mean [SD] birth weight, 949 [271] g) were analyzed. The median severity scores for each category were as follows: 1.1 (interquartile range [IQR], 1.0-1.5) (no ROP), 1.5 (IQR, 1.1-3.4) (mild ROP), 4.6 (IQR, 2.4-5.3) (type 2 and pre-plus), and 7.5 (IQR, 5.0-8.7) (treatment-requiring ROP) (P < .001). When the long-term differences in the median severity scores across time between the eyes progressing to treatment and those who did not eventually require treatment were compared, the median score was higher in the treatment group by 0.06 at 30 to 32 weeks, 0.75 at 32 to 34 weeks, 3.56 at 34 to 36 weeks, 3.71 at 36 to 38 weeks, and 3.24 at 38 to 40 weeks postmenstrual age (P < .001 for all comparisons). Conclusions and Relevance The findings suggest that the proposed ROP vascular severity score is associated with category of disease at a given point in time and clinical progression of ROP in premature infants. Automated image analysis may be used to quantify clinical disease progression and identify infants at high risk for eventually developing treatment-requiring ROP. This finding has implications for quality and delivery of ROP care and for future approaches to disease classification.
Collapse
Affiliation(s)
- Stanford Taylor
- Department of Ophthalmology, Casey Eye institute, Oregon Health & Science University, Portland
| | - James M. Brown
- Athinoula A. Martinos Center for Biomedical Imaging, Department of Radiology, Massachusetts General Hospital, Charlestown
| | - Kishan Gupta
- Department of Ophthalmology, Casey Eye institute, Oregon Health & Science University, Portland
| | - J. Peter Campbell
- Department of Ophthalmology, Casey Eye institute, Oregon Health & Science University, Portland
| | - Susan Ostmo
- Department of Ophthalmology, Casey Eye institute, Oregon Health & Science University, Portland
| | - R. V. Paul Chan
- Department of Ophthalmology and Visual Sciences, Illinois Eye and Ear Infirmary, University of Illinois at Chicago, Chicago
| | - Jennifer Dy
- Department of Electrical and Computer Engineering, Northeastern University, Boston, Massachusetts
| | - Deniz Erdogmus
- Department of Electrical and Computer Engineering, Northeastern University, Boston, Massachusetts
| | - Stratis Ioannidis
- Department of Electrical and Computer Engineering, Northeastern University, Boston, Massachusetts
| | - Sang J. Kim
- Department of Ophthalmology, Casey Eye institute, Oregon Health & Science University, Portland
- Department of Ophthalmology, Samsung Medical Center, Sungkyunkwan University School of Medicine, Seoul, Republic of Korea
| | - Jayashree Kalpathy-Cramer
- Athinoula A. Martinos Center for Biomedical Imaging, Department of Radiology, Massachusetts General Hospital, Charlestown
- Massachusetts General Hospital and Brigham and Women’s Hospital Center for Clinical Data Science, Boston
| | - Michael F. Chiang
- Department of Ophthalmology, Casey Eye institute, Oregon Health & Science University, Portland
- Department of Medical Informatics and Clinical Epidemiology, Oregon Health & Science University, Portland
| |
Collapse
|
44
|
Ting DS, Peng L, Varadarajan AV, Keane PA, Burlina PM, Chiang MF, Schmetterer L, Pasquale LR, Bressler NM, Webster DR, Abramoff M, Wong TY. Deep learning in ophthalmology: The technical and clinical considerations. Prog Retin Eye Res 2019; 72:100759. [DOI: 10.1016/j.preteyeres.2019.04.003] [Citation(s) in RCA: 137] [Impact Index Per Article: 27.4] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/23/2018] [Revised: 04/21/2019] [Accepted: 04/23/2019] [Indexed: 12/22/2022]
|
45
|
Plus Disease in Telemedicine Approaches to Evaluating Acute-Phase ROP (e-ROP) Study: Characteristics, Predictors, and Accuracy of Image Grading. Ophthalmology 2019; 126:868-875. [DOI: 10.1016/j.ophtha.2019.01.021] [Citation(s) in RCA: 7] [Impact Index Per Article: 1.4] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/19/2018] [Revised: 01/09/2019] [Accepted: 01/16/2019] [Indexed: 11/24/2022] Open
|
46
|
Fielder AR, Wallace DK, Stahl A, Reynolds JD, Chiang MF, Quinn GE. Describing Retinopathy of Prematurity: Current Limitations and New Challenges. Ophthalmology 2019; 126:652-654. [PMID: 31005186 DOI: 10.1016/j.ophtha.2018.12.034] [Citation(s) in RCA: 9] [Impact Index Per Article: 1.8] [Reference Citation Analysis] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/22/2018] [Revised: 12/12/2018] [Accepted: 12/17/2018] [Indexed: 11/19/2022] Open
|
47
|
Lertjirachai I, Stem MS, Moysidis SN, Koulisis N, Capone A, Drenser KA, Trese MT. Vessel Tortuosity Cutoff Values Using the Modified ROPtool May Predict Need for Treatment in Retinopathy of Prematurity. Ophthalmic Surg Lasers Imaging Retina 2019; 50:215-220. [PMID: 30998242 DOI: 10.3928/23258160-20190401-03] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/15/2018] [Accepted: 11/02/2018] [Indexed: 11/20/2022]
Abstract
BACKGROUND AND OBJECTIVE To quantify vessel tortuosity among infants with retinopathy of prematurity (ROP). PATIENTS AND METHODS This was a retrospective study including 61 RetCam images from 33 infants. The laser treatment (LT) group included 17 infants who underwent laser for ROP. The no-treatment (NT) group included 16 infants. The modified ROPtool was used to calculate mean vessel tortuosity (MVT) and highest vessel tortuosity (HVT) for the participants and for the standard plus disease photograph from the Early Treatment for Retinopathy of Prematurity (ETROP) study. RESULTS The median MVT was 1.226 versus 1.056 for the LT and NT groups, respectively (P < .001). The median HVT was 1.346 versus 1.088 (P < .001). An MVT of 1.124 was 96.7% sensitive and 100% specific for identifying infants with treatment-requiring ROP. Both MVT and HVT cutoff values correctly captured plus disease in the standard ETROP trial photograph. CONCLUSION The modified ROPtool can be used to identify infants who have treatment-requiring ROP with a high level of sensitivity and specificity. [Ophthalmic Surg Lasers Imaging Retina. 2019;50:215-220.].
Collapse
|
48
|
Hu J, Chen Y, Zhong J, Ju R, Yi Z. Automated Analysis for Retinopathy of Prematurity by Deep Neural Networks. IEEE TRANSACTIONS ON MEDICAL IMAGING 2019; 38:269-279. [PMID: 30080144 DOI: 10.1109/tmi.2018.2863562] [Citation(s) in RCA: 53] [Impact Index Per Article: 10.6] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/08/2023]
Abstract
Retinopathy of Prematurity (ROP) is a retinal vasproliferative disorder disease principally observed in infants born prematurely with low birth weight. ROP is an important cause of childhood blindness. Although automatic or semi-automatic diagnosis of ROP has been conducted, most previous studies have focused on "plus" disease, which is indicated by abnormalities of retinal vasculature. Few studies have reported methods for identifying the "stage" of the ROP disease. Deep neural networks have achieved impressive results in many computer vision and medical image analysis problems, raising expectations that it might be a promising tool in the automatic diagnosis of ROP. In this paper, convolutional neural networks with a novel architecture are proposed to recognize the existence and severity of ROP disease per-examination. The severity of ROP is divided into mild and severe cases according to the disease progression. The proposed architecture consists of two sub-networks connected by a feature aggregate operator. The first sub-network is designed to extract high-level features from images of the fundus. These features from different images in an examination are fused by the aggregate operator, then used as the input for the second sub-network to predict its class. A large data set imaged by RetCam 3 is used to train and evaluate the model. The high classification accuracy in the experiment demonstrates the effectiveness of the proposed architecture for recognizing the ROP disease.
Collapse
|
49
|
Redd TK, Campbell JP, Brown JM, Kim SJ, Ostmo S, Chan RVP, Dy J, Erdogmus D, Ioannidis S, Kalpathy-Cramer J, Chiang MF. Evaluation of a deep learning image assessment system for detecting severe retinopathy of prematurity. Br J Ophthalmol 2018; 103:bjophthalmol-2018-313156. [PMID: 30470715 PMCID: PMC7880608 DOI: 10.1136/bjophthalmol-2018-313156] [Citation(s) in RCA: 83] [Impact Index Per Article: 13.8] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/31/2018] [Revised: 10/03/2018] [Accepted: 10/17/2018] [Indexed: 02/04/2023]
Abstract
BACKGROUND Prior work has demonstrated the near-perfect accuracy of a deep learning retinal image analysis system for diagnosing plus disease in retinopathy of prematurity (ROP). Here we assess the screening potential of this scoring system by determining its ability to detect all components of ROP diagnosis. METHODS Clinical examination and fundus photography were performed at seven participating centres. A deep learning system was trained to detect plus disease, generating a quantitative assessment of retinal vascular abnormality (the i-ROP plus score) on a 1-9 scale. Overall ROP disease category was established using a consensus reference standard diagnosis combining clinical and image-based diagnosis. Experts then ranked ordered a second data set of 100 posterior images according to overall ROP severity. RESULTS 4861 examinations from 870 infants were analysed. 155 examinations (3%) had a reference standard diagnosis of type 1 ROP. The i-ROP deep learning (DL) vascular severity score had an area under the receiver operating curve of 0.960 for detecting type 1 ROP. Establishing a threshold i-ROP DL score of 3 conferred 94% sensitivity, 79% specificity, 13% positive predictive value and 99.7% negative predictive value for type 1 ROP. There was strong correlation between expert rank ordering of overall ROP severity and the i-ROP DL vascular severity score (Spearman correlation coefficient=0.93; p<0.0001). CONCLUSION The i-ROP DL system accurately identifies diagnostic categories and overall disease severity in an automated fashion, after being trained only on posterior pole vascular morphology. These data provide proof of concept that a deep learning screening platform could improve objectivity of ROP diagnosis and accessibility of screening.
Collapse
Affiliation(s)
- Travis K Redd
- Department of Ophthalmology, Casey Eye Institute, Oregon Health & Science University, Portland, Oregon, USA
| | - John Peter Campbell
- Department of Ophthalmology, Casey Eye Institute, Oregon Health & Science University, Portland, Oregon, USA
| | - James M Brown
- Athinoula A. Martinos Center for Biomedical Imaging, Department of Radiology, Massachusetts General Hospital, Charlestown, Maryland, USA
| | - Sang Jin Kim
- Department of Ophthalmology, Casey Eye Institute, Oregon Health & Science University, Portland, Oregon, USA
- Department of Ophthalmology, Samsung Medical Center, Sungkyunkwan University School of Medicine, Seoul, Republic of Korea
| | - Susan Ostmo
- Department of Ophthalmology, Casey Eye Institute, Oregon Health & Science University, Portland, Oregon, USA
| | - Robison Vernon Paul Chan
- Department of Ophthalmology and Visual Sciences, Illinois Eye and Ear Infirmary, University of Illinois at Chicago, Chicago, Illinois, USA
| | - Jennifer Dy
- Department of Electrical and Computer Engineering, Northeastern University, Boston, Massachusetts, USA
| | - Deniz Erdogmus
- Department of Electrical and Computer Engineering, Northeastern University, Boston, Massachusetts, USA
| | - Stratis Ioannidis
- Department of Electrical and Computer Engineering, Northeastern University, Boston, Massachusetts, USA
| | - Jayashree Kalpathy-Cramer
- Athinoula A. Martinos Center for Biomedical Imaging, Department of Radiology, Massachusetts General Hospital, Charlestown, Maryland, USA
| | - Michael F Chiang
- Department of Ophthalmology, Casey Eye Institute, Oregon Health & Science University, Portland, Oregon, USA
- Department of Medical Informatics and Clinical Epidemiology, Oregon Health & Science University, Portland, Oregon, USA
| |
Collapse
|
50
|
Ting DSW, Wu WC, Toth C. Deep learning for retinopathy of prematurity screening. Br J Ophthalmol 2018; 103:bjophthalmol-2018-313290. [PMID: 30470712 DOI: 10.1136/bjophthalmol-2018-313290] [Citation(s) in RCA: 16] [Impact Index Per Article: 2.7] [Reference Citation Analysis] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/22/2022]
Affiliation(s)
- Daniel S W Ting
- Duke-NUS Medical School, Singapore National Eye Centre, Singapore, Singapore
| | - Wei-Chi Wu
- Department of Ophthalmology, Chang Gung Memorial Hospital, Linkou, Taoyuan, Taiwan
- College of Medicine, Chang Gung University, Taoyuan, Taiwan
| | - Cynthia Toth
- Duke Eye Center, Duke University, Durham, North Carolina, USA
| |
Collapse
|