Evaluation of a Qualitative Approach for Detecting Glaucomatous Progression Using Wide-Field Optical Coherence Tomography Scans

Exploring the relationship between 24-2 visual field and widefield optical coherence tomography data across healthy, glaucoma suspect and glaucoma eyes

PURPOSE

To utilise ganglion cell-inner plexiform layer (GCIPL) measurements acquired using widefield optical coherence tomography (OCT) scans spanning 55° × 45° to explore the link between co-localised structural parameters and clinical visual field (VF) data.

METHODS

Widefield OCT scans acquired from 311 healthy, 268 glaucoma suspect and 269 glaucoma eyes were segmented to generate GCIPL thickness measurements. Estimated ganglion cell (GC) counts, calculated from GCIPL measurements, were plotted against 24-2 SITA Faster visual field (VF) thresholds, and regression models were computed with data categorised by diagnosis and VF status. Classification of locations as VF defective or non-defective using GCIPL parameters computed across eccentricity- and hemifield-dependent clusters was assessed by analysing areas under receiver operating characteristic curves (AUROCCs). Sensitivities and specificities were calculated per diagnostic category.

RESULTS

Segmented linear regression models between GC counts and VF thresholds demonstrated higher variability in VF defective locations relative to non-defective locations (mean absolute error 6.10-9.93 dB and 1.43-1.91 dB, respectively). AUROCCs from cluster-wide GCIPL parameters were similar across methods centrally (p = 0.06-0.84) but significantly greater peripherally, especially when considering classification of more central locations (p < 0.0001). Across diagnoses, cluster-wide GCIPL parameters demonstrated variable sensitivities and specificities (0.36-0.93 and 0.65-0.98, respectively), with the highest specificities observed across healthy eyes (0.73-0.98).

CONCLUSIONS

Quantitative prediction of VF thresholds from widefield OCT is affected by high variability at VF defective locations. Prediction of VF status based on cluster-wide GCIPL parameters from widefield OCT could become useful to aid clinical decision-making in appropriately targeting VF assessments.

OCT Segmentation Errors with Bruch's Membrane Opening-Minimum Rim Width as Compared with Retinal Nerve Fiber Layer Thickness

OBJECTIVE

To compare the magnitude and location of automated segmentation errors of the Bruch's membrane opening-minimum rim width (BMO-MRW) and retinal nerve fiber layer thickness (RNFLT).

DESIGN

Cross-sectional study.

PARTICIPANTS

We included 162 glaucoma suspect or open-angle glaucoma eyes from 162 participants.

METHODS

We used spectral-domain optic coherence tomography (Spectralis 870 nm, Heidelberg Engineering) to image the optic nerve with 24 radial optic nerve head B-scans and a 12-degree peripapillary circle scan, and exported the native "automated segmentation only" results for BMO-MRW and RNFLT. We also exported the results after "manual refinement" of the measurements.

MAIN OUTCOME MEASURES

We calculated the absolute and proportional error globally and within the 12 30-degree sectors of the optic disc. We determined whether the glaucoma classifications were different between BMO-MRW and RNFLT as a result of manual and automatic segmentation.

RESULTS

The absolute error mean was larger for BMO-MRW than for RNFLT (10.8 μm vs. 3.58 μm, P < 0.001). However, the proportional errors were similar (4.3% vs. 4.4%, P = 0.47). In a multivariable regression model, errors in BMO-MRW were not significantly associated with age, location, magnitude, or severity of glaucoma loss (all P ≥ 0.05). However, larger RNFLT errors were associated with the superior and inferior sector location, thicker nerve fiber layer, and worse visual field (all P < 0.05). Errors in BMO-MRW and RNFLT were not likely to occur in the same sector location (R2 = 0.001; P = 0.15). With manual refinement, the glaucoma classification changed in 7.8% and 6.2% of eyes with BMO-MRW and RNFLT, respectively.

CONCLUSIONS

Both BMO-MRW and RNFLT measurements included segmentation errors, which did not seem to have a common location, and may result in differences in glaucoma classification.

FINANCIAL DISCLOSURE(S)

Proprietary or commercial disclosure may be found in the Footnotes and Disclosures at the end of this article.

Characterisation of the normal human ganglion cell-inner plexiform layer using widefield optical coherence tomography

PURPOSE

To describe variations in ganglion cell-inner plexiform layer (GCIPL) thickness in a healthy cohort from widefield optical coherence tomography (OCT) scans.

METHODS

Widefield OCT scans spanning 55° × 45° were acquired from 470 healthy eyes. The GCIPL was automatically segmented using deep learning methods. Thickness measurements were extracted after correction for warpage and retinal tilt. Multiple linear regression analysis was applied to discern trends between global GCIPL thickness and age, axial length and sex. To further characterise age-related change, hierarchical and two-step cluster algorithms were applied to identify locations sharing similar ageing properties, and rates of change were quantified using regression analyses with data pooled by cluster analysis outcomes.

RESULTS

Declines in widefield GCIPL thickness with age, increasing axial length and female sex were observed (parameter estimates -0.053, -0.436 and -0.464, p-values <0.001, <0.001 and 0.02, respectively). Cluster analyses revealed concentric, slightly nasally displaced, horseshoe patterns of age-related change in the GCIPL, with up to four statistically distinct clusters outside the macula. Linear regression analyses revealed significant ageing decline in GCIPL thickness across all clusters, with faster rates of change observed at central locations when expressed as absolute (slope = -0.19 centrally vs. -0.04 to -0.12 peripherally) and percentage rates of change (slope = -0.001 centrally vs. -0.0005 peripherally).

CONCLUSIONS

Normative variations in GCIPL thickness from widefield OCT with age, axial length and sex were noted, highlighting factors worth considering in further developments. Widefield OCT has promising potential to facilitate quantitative detection of abnormal GCIPL outside standard fields of view.

Agreement Between Trend-Based and Qualitative Analysis of the Retinal Nerve Fiber Layer Thickness for Glaucoma Progression on Spectral-Domain Optical Coherence Tomography

INTRODUCTION

To evaluate the agreement between trend-based analysis and qualitative assessment of the retinal nerve fiber layer (RNFL) thickness for glaucomatous progression on spectral-domain optical coherence tomography (SDOCT).

METHODS

Retrospective review of 190 eyes from 103 patients with glaucoma or suspected glaucoma that underwent SDOCT imaging during four consecutive clinic visits. Trend-based progression was characterized by a significantly negative slope. Progression by qualitative analysis was determined by review of raw SDOCT B-scans.

RESULTS

The slope was significantly greater in those with progression than without progression for both trend-based and qualitative analysis (p < 0.001). However, the qualitative grading classified a significantly greater proportion of eyes as progressing compared to trend-based analysis in both the superotemporal (ST) (23.2% vs. 10.5%, p = 0.001) and inferotemporal (IT) RNFL (27.4% vs 8.4%, p < 0.001). The trend-based and qualitative classifications of progression showed poor agreement in both the ST (kappa = 0.0135) and IT RNFL (kappa = 0.1222). The agreement between trend-based and qualitative analysis was lower for eyes with artifacts (ST = 58.11%; IT = 68.7%) than those without artifacts (ST = 80.2%; IT = 74.8%). Moreover, among eyes with artifacts, there was no significant difference in slope between those qualitatively categorized as progressing versus not progressing (p > 0.05).

CONCLUSIONS

Poor agreement was found between a trend-based and qualitative analysis of change in RNFL on SDOCT. Careful qualitative review of SDOCT imaging may identify specific areas of glaucoma progression not captured by trend-based methods, especially in the presence of artifacts. Such an approach may also prove useful for detecting glaucoma progression in a clinical setting when there are few data points available.

Did the OCT Show Progression Since the Last Visit?

Identifying progression is of fundamental importance to the management of glaucoma. It is also a challenge. The most sophisticated, and probably the most useful, commercially available clinical tool for identifying progression is the Guided Progression Analysis (GPA), which was initially developed to identify progression using 24-2 visual field tests. More recently, it has been extended to retinal nerve fiber layer (RNFL) and ganglion cell+inner plexiform layer thicknesses measured with optical coherence tomography (OCT). However, the OCT GPA requires a minimum of 3 tests to determine "possible loss (progression)" and a minimum of 4 tests to determine if the patient shows "likely loss (progression)." Thus, it is not designed to answer a fundamental question asked by both the clinician and the patient, namely: Did damage progress since the last visit? Some clinicians use changes in summary statistics, such as global/average circumpapillary RNFL thickness. However, these statistics have poor sensitivity and specificity due to segmentation and alignment errors. Instead of relying on the GPA analysis or summary statistics, one needs to evaluate RNFL and ganglion cell+inner plexiform layer probability maps and circumpapillary OCT B-scan images. In addition, we argue that the clinician can make a better decision about suspected progression between 2 test days by topographically comparing the changes in the different OCT maps and images, in addition to topographically comparing the changes in the visual field with the changes in OCT probability maps.

Strategies to Improve Convolutional Neural Network Generalizability and Reference Standards for Glaucoma Detection From OCT Scans

Purpose

To develop and evaluate methods to improve the generalizability of convolutional neural networks (CNNs) trained to detect glaucoma from optical coherence tomography retinal nerve fiber layer probability maps, as well as optical coherence tomography circumpapillary disc (circle) b-scans, and to explore impact of reference standard (RS) on CNN accuracy.

Methods

CNNs previously optimized for glaucoma detection from retinal nerve fiber layer probability maps, and newly developed CNNs adapted for glaucoma detection from optical coherence tomography b-scans, were evaluated on an unseen dataset (i.e., data collected at a different site). Multiple techniques were used to enhance CNN generalizability, including augmenting the training dataset, using multimodal input, and training with confidently rated images. Model performance was evaluated with different RS.

Results

Training with data augmentation and training on confident images enhanced the accuracy of the CNNs for glaucoma detection on a new dataset by 5% to 9%. CNN performance was optimal when a similar RS was used to establish labels both for the training and the testing sets. However, interestingly, the CNNs described here were robust to variation in the RS.

Conclusions

CNN generalizability can be improved with data augmentation, multiple input image modalities, and training on images with confident ratings. CNNs trained and tested with the same RS achieved best accuracy, suggesting that choosing a thorough and consistent RS for training and testing improves generalization to new datasets.

Translational Relevance

Strategies for enhancing CNN generalizability and for choosing optimal RS should be standard practice for CNNs before their deployment for glaucoma detection.

Comparing the Rule of 5 to Trend-based Analysis for Detecting Glaucoma Progression on OCT

PURPOSE

The rule of 5 is a simple rule for detecting retinal nerve fiber layer (RNFL) change on spectral-domain OCT (SD-OCT), in which a loss of 5 μm of global RNFL on a follow-up test is considered evidence of significant change when compared with the baseline. The rule is based on short-term test-retest variability of SD-OCT and is often used in clinical practice. The purpose of this study was to compare the rule of 5 with trend-based analysis of global RNFL thickness over time for detecting glaucomatous progression.

DESIGN

Prospective cohort.

PARTICIPANTS

A total of 300 eyes of 210 glaucoma subjects followed for an average of 5.4±1.5 years with a median of 11 (interquartile range, 7-14) visits.

METHODS

Trend-based analysis was performed by ordinary least-squares (OLS) linear regression of global RNFL thickness over time. For estimation of specificity, false-positives were obtained by assessing for progression on series of randomly permutated follow-up visits for each eye, which removes any systematic trend over time. The specificity of trend-based analysis was matched to that of the rule of 5 to allow meaningful comparison of the "hit rate," or the proportion of glaucoma eyes categorized as progressing at each time point, using the original sequence of visits.

MAIN OUTCOME MEASURES

Comparison between hit rates of trend-analysis versus rule of 5 at matched specificity.

RESULTS

After 5 years, the simple rule of 5 identified 37.5% of eyes as progressing at a specificity of 81.1%. At the same specificity, the hit rate for trend-based analysis was significantly greater than that of the rule of 5 (62.9% vs. 37.5%; P < 0.001). If the rule of 5 was required to be repeatable on a consecutive test, specificity improved to 93.4%, but hit rate decreased to 21.0%. At this higher specificity, trend-based analysis still had a significantly greater hit rate than the rule of 5 (47.4% vs. 21.0%, respectively; P < 0.001).

CONCLUSIONS

Trend-based analysis was superior to the simple rule of 5 for identifying progression in glaucoma eyes and should be preferred as a method for longitudinal assessment of global SD-OCT RNFL change over time.

Enhancing the Accuracy of Glaucoma Detection from OCT Probability Maps using Convolutional Neural Networks

We describe and assess convolutional neural network (CNN) models for detection of glaucoma based upon optical coherence tomography (OCT) retinal nerve fiber layer (RNFL) probability maps. CNNs pretrained on natural images performed comparably to CNNs trained solely on OCT data, and all models showed high accuracy in detecting glaucoma, with receiver operating characteristic area under the curve (AUC) scores ranging from 0.930 to 0.989. Attention-based heat maps of CNN regions of interest suggest that these models could be improved by incorporation of blood vessel location information. Such CNN models have the potential to work in tandem with human experts to maintain overall eye health and expedite detection of blindness-causing eye disease.

An Automated Method for Assessing Topographical Structure-Function Agreement in Abnormal Glaucomatous Regions

Purpose

To develop an automated/objective method for topographically comparing abnormal regions on optical coherence tomography (OCT) and visual field (VF) tests of eyes with early glaucoma.

Methods

A custom R program was developed that allows for both visualization and automatic assessment of the topographical agreement between functional (24-2 and/or 10-2 VF) and structural (widefield OCT retinal nerve fiber layer and/or retinal ganglion cell layer) deviation/probability maps. It was optimized using information from 98 eyes: 53 diagnosed as "definitely glaucoma" (DG) and 45 recruited as healthy (H) controls. Different pairs of abnormal VF (P <1%, <2%, <5%) and abnormal OCT (P <5%, <10%, <15%) criteria were evaluated. The percentages of abnormal structure-abnormal function (aS-aF) agreement found in DG eyes and nonagreement found in H eyes were used to define the optimal criteria and number of aS-aF locations for the detection of aS-aF agreement.

Results

A criterion of two aS-aF locations with "OCT <10% and VF <5%" on VF pattern deviation (PD) probability and OCT deviation/probability maps yielded high overall agreement (92%) with high aS-aF agreement for the DG eyes (89%) and high aS-aF nonagreement for the H eyes (95%). Total deviation probability maps achieved slightly lower performance than PD maps.

Conclusions

The method described here can automatically and objectively evaluate aS-aF agreement with a direct comparison of abnormal regions of function and structure.

Translational Relevance

As glaucoma diagnosis often involves assessing structure-function agreement, this technique can overcome subjectivity in this assessment.

Structure-Function Agreement Is Better Than Commonly Thought in Eyes With Early Glaucoma

Purpose

To assess the agreement between structural (optical coherence tomography [OCT]) and functional (visual field [VF]) glaucomatous damage with an automated method and deviation/probability maps, and to compare this method to a metric method.

Methods

Wide-field spectral-domain OCT scans, including the disc and macula, and 24-2 and 10-2 VFs were obtained from 45 healthy control (H) eyes/individuals, and 53 eyes/patients with 24-2 mean deviation (MD) better than -6 dB diagnosed as "definite glaucoma" (DG) by experts. Abnormal structure-abnormal function (aS-aF) agreement was assessed with an automated topographic (T) method based upon VF pattern deviation and OCT probability maps. Results were compared to a metric (M) method optimized for accuracy, (abnormal 24-2 glaucoma hemifield test [GHT] or pattern standard deviation [PSD], or 10-2 PSD AND abnormal OCT [quadrant]).

Results

For the T-method, 47 (88.7%) of the 53 DG eyes showed aS-aF agreement, compared to 2 (4.5%) of the 45 H eyes. The aS-aF agreement for these two H eyes was easily identified as mistaken, and did not replicate on a subsequent test. Without the 10-2, the aS-aF agreement decreased from 47 to 34 (64.2%) of 53 DG eyes. For the M-method, 37 (69.8%) of the 53 DG eyes showed aS-aF agreement, while omitting the 10-2 VF resulted in agreement in only 33 (62.3%) eyes.

Conclusions

There is good agreement between structural and functional damage, even in eyes with confirmed early glaucomatous damage, if both 24-2 and 10-2 VFs are obtained, and abnormal locations on the VFs are compared to abnormal regions seen on OCT macular and disc scans. This can be done in an objective, automated fashion. (ClinicalTrials.gov number, NCT02547740.).

Sample Size Requirements of Glaucoma Clinical Trials When Using Combined Optical Coherence Tomography and Visual Field Endpoints

Glaucoma clinical trials using visual field (VF) endpoints currently require large sample sizes because of the slowly-progressive nature of this disease. We sought to examine whether the combined use of VF testing and non-invasive optical coherence tomography (OCT) imaging of the neuroretinal tissue could improve the feasibility of such trials. To examine this, we included 192 eyes of 121 glaucoma participants seen at ≥5 visits over a 2-year period to extract real-world estimates of the rates of change and variability of VF and OCT imaging measurements for computer simulations to obtain sample size estimates. We observed that the combined use of VF and OCT endpoints led to a 31-33% reduction in sample size requirements compared to using VF endpoints alone for various treatment effect sizes. For example, 189 participants would be required per group to detect a 30% treatment effect with 90% power with combined VF and OCT endpoints, whilst 276 and 285 participants would be required when using VF and OCT endpoints alone respectively. The combined use of OCT and VF endpoints thus has the potential to effectively improve the feasibility of future glaucoma clinical trials.

Performance of the Rule of 5 for Detecting Glaucoma Progression between Visits with OCT

PURPOSE

To evaluate whether loss of 5 μm or more in global retinal nerve fiber layer (RNFL) thickness on spectral-domain (SD) between 2 consecutive visits is specific for glaucoma progression.

DESIGN

Prospective cohort.

PARTICIPANTS

Ninety-two eyes of 49 control participants and 300 eyes of 210 glaucoma patients.

METHODS

Patients completed at least 5 standard automated perimetry and SD OCT examinations at 6-month intervals over at least 2 years. Eyes were categorized as progressing from glaucoma if the average RNFL declined by 5 μm or more between 2 consecutive visits. The false-positive proportion was estimated by 2 methods: (1) 5-μm or more loss in control participants and (2) 5-μm or more gain in glaucoma. The false-positive proportion was subtracted from the cumulative proportion of eyes categorized with glaucoma progression to estimate the true progression prevalence.

MAIN OUTCOME MEASURES

False-positive and true progression prevalence of patients with glaucoma detected as progressing on SD OCT.

RESULTS

After 5 years of semiannual testing, the cumulative proportion of false-positive results based on 5-μm or more RNFL losses between visits was 24.8% in the control participants. Although 40.6% of glaucoma eyes were diagnosed with progression at 5 years, only 15.8% would have been considered to show true progression based on the expected false-positive ratio from the control participants (i.e., 40.6%-24.8%). The cumulative proportion of an intervisit gain of 5 μm or more at 5 years was 27.4% in glaucoma eyes, suggesting that only 13.2% of eyes with glaucoma truly had progressed (i.e., 40.6%-27.4%).

CONCLUSIONS

Loss of 5 μm or more in average RNFL thickness between consecutive SD OCT tests is not specific for glaucoma progression. Application of this intervisit rule of 5 can result in a high cumulative proportion of false-positive results over time, which could lead to unnecessary interventions in patients whose disease is stable. More specific diagnostic criteria are needed to help clinicians determine whether patients with glaucoma are progressing so that therapy escalation is both timely and appropriate.

Repeatability of quantitative measurements of retinal layers with SD-OCT and agreement between vertical and horizontal scan protocols in healthy eyes

Purpose

To evaluate the repeatability of the new spectral domain optical coherence tomography (HOCT-1F), and also to evaluate the agreement between vertical and horizontal scan protocols. In addition, we also evaluated the relation between the repeatability and age.

Methods

Three consecutive measurements of the inner limiting membrane–retinal pigment epithelium (ILM-RPE), inner limiting membrane–inner plexiform layer (ILM-IPL) from macular horizontal and vertical scans, and inner limiting membrane–retinal nerve fiber layer (ILM-RNFL) from optic disc horizontal scan. 159 subjects were included in the analysis. The within subject standard deviation (Sw) and the repeatability limits (R_limit) are used to represent the repeatability of the parameters for the different sectors.

Results

The Sw for the ILM-RPE thickness was less than 3.5 μm for each sector and scan direction. The Sw values varied within the sectors and scan modes, with horizontal scan modes resulting in better values for the horizontal sectors, and vice versa. The Sw for the GCL-IPL thickness was less than 2 μm, and was similar between the vertical and horizontal scan modes for each sector map. For the optic disc scan, the Sw was not symmetric along the clock-hour map sectors, the largest Sw values were seen in the vertical sectors (8.6 μm). The mean difference between the vertical and horizontal scans was less than 2 μm for each retinal thickness sector map. Significant but weak correlation between the Sw and the subject’s age was seen in both macular and optic disc scans.

Conclusions

The repeatability of the HOCT-1F to measure the ILM-RPE-, ILM-IPL- and ILM-RNFL-thickness is good. The repeatability of the ILM-RPE thickness is dependent on the scan direction, which should be taken into account when calculating retinal thickness. There is a weak correlation between the repeatability and the subject’s age.

21st century glaucoma care

Glaucoma care has evolved dramatically over the past generation, with changes that have incorporated new technology and improved understanding of the disease process. A major need is to construct a useful definition of glaucomatous optic neuropathy that can be used to compare data across clinical research studies. The treatment of glaucoma should now be based on achievement of a goal target for intraocular pressure, unique to each patient. Adherence with eye drop treatment is far from ideal and can be improved using reminder systems. Sustained delivery of glaucoma medication is on the horizon. New surgical approaches to glaucoma are being actively studied but have not as yet found their place in its care, with rigorous testing against present treatments needed.