Multi-surface and multi-field co-segmentation of 3-D retinal optical coherence tomography

Hybrid deep learning and optimal graph search method for optical coherence tomography layer segmentation in diseases affecting the optic nerve

Accurate segmentation of retinal layers in optical coherence tomography (OCT) images is critical for assessing diseases that affect the optic nerve, but existing automated algorithms often fail when pathology causes irregular layer topology, such as extreme thinning of the ganglion cell-inner plexiform layer (GCIPL). Deep LOGISMOS, a hybrid approach that combines the strengths of deep learning and 3D graph search to overcome their limitations, was developed to improve the accuracy, robustness and generalizability of retinal layer segmentation. The method was trained on 124 OCT volumes from both eyes of 31 non-arteritic anterior ischemic optic neuropathy (NAION) patients and tested on three cross-sectional datasets with available reference tracings: Test-NAION (40 volumes from both eyes of 20 NAION subjects), Test-G (29 volumes from 29 glaucoma subjects/eyes), and Test-JHU (35 volumes from 21 multiple sclerosis and 14 control subjects/eyes) and one longitudinal dataset without reference tracings: Test-G-L (155 volumes from 15 glaucoma patients/eyes). In the three test datasets with reference tracings (Test-NAION, Test-G, and Test-JHU), Deep LOGISMOS achieved very high Dice similarity coefficients (%) on GCIPL: 89.97±3.59, 90.63±2.56, and 94.06±1.76, respectively. In the same context, Deep LOGISMOS outperformed the Iowa reference algorithms by improving the Dice score by 17.5, 5.4, and 7.5, and also surpassed the deep learning framework nnU-Net with improvements of 4.4, 3.7, and 1.0. For the 15 severe glaucoma eyes with marked GCIPL thinning (Test-G-L), it demonstrated reliable regional GCIPL thickness measurement over five years. The proposed Deep LOGISMOS approach has potential to enhance precise quantification of retinal structures, aiding diagnosis and treatment management of optic nerve diseases.

Automated inter-device 3D OCT image registration using deep learning and retinal layer segmentation

Optical coherence tomography (OCT) is the most widely used imaging modality in ophthalmology. There are multiple variations of OCT imaging capable of producing complementary information. Thus, registering these complementary volumes is desirable in order to combine their information. In this work, we propose a novel automated pipeline to register OCT images produced by different devices. This pipeline is based on two steps: a multi-modal 2D en-face registration based on deep learning, and a Z-axis (axial axis) registration based on the retinal layer segmentation. We evaluate our method using data from a Heidelberg Spectralis and an experimental PS-OCT device. The empirical results demonstrated high-quality registrations, with mean errors of approximately 46 µm for the 2D registration and 9.59 µm for the Z-axis registration. These registrations may help in multiple clinical applications such as the validation of layer segmentations among others.

Deep ensemble learning for automated non-advanced AMD classification using optimized retinal layer segmentation and SD-OCT scans

BACKGROUND

Accurate retinal layer segmentation in optical coherence tomography (OCT) images is crucial for quantitatively analyzing age-related macular degeneration (AMD) and monitoring its progression. However, previous retinal segmentation models depend on experienced experts and manually annotating retinal layers is time-consuming. On the other hand, accuracy of AMD diagnosis is directly related to the segmentation model's performance. To address these issues, we aimed to improve AMD detection using optimized retinal layer segmentation and deep ensemble learning.

METHOD

We integrated a graph-cut algorithm with a cubic spline to automatically annotate 11 retinal boundaries. The refined images were fed into a deep ensemble mechanism that combined a Bagged Tree and end-to-end deep learning classifiers. We tested the developed deep ensemble model on internal and external datasets.

RESULTS

The total error rates for our segmentation model using the boundary refinement approach was significantly lower than OCT Explorer segmentations (1.7% vs. 7.8%, p-value = 0.03). We utilized the refinement approach to quantify 169 imaging features using Zeiss SD-OCT volume scans. The presence of drusen and thickness of total retina, neurosensory retina, and ellipsoid zone to inner-outer segment (EZ-ISOS) thickness had higher contributions to AMD classification compared to other features. The developed ensemble learning model obtained a higher diagnostic accuracy in a shorter time compared with two human graders. The area under the curve (AUC) for normal vs. early AMD was 99.4%.

CONCLUSION

Testing results showed that the developed framework is repeatable and effective as a potentially valuable tool in retinal imaging research.

Visual Field Sensitivity Prediction Using Optical Coherence Tomography Analysis in Hydroxychloroquine Toxicity

Purpose

This study investigates the association between local retina structure and visual function in a cohort with long-term hydroxychloroquine (HCQ) use.

Methods

The study included 84 participants (54 participants without toxicity and 30 participants with toxicity) with history of chronic HCQ use (14.5 ± 7.4 years) who had testing with spectral-domain optical coherence tomography (SD-OCT) imaging and Humphrey 10-2 visual fields. Optical coherence tomography (OCT) metrics (total and outer retina thickness [TRT and ORT], minimum intensity [MinI], and ellipsoid zone [EZ] loss) were sampled in regions corresponding to visual field test locations. Univariate linear correlations were investigated and a multivariate random forest regression using a combination of OCT metrics was used to predict visual field sensitivity by locus using a leave-one-out cross-validation strategy.

Results

In univariate linear regression, EZ loss demonstrated the strongest relationship with visual field sensitivities in the parafoveal ring with R2 = 0.58. TRT and ORT revealed positive correlations with visual field sensitivity (R2 = 0.57 and 0.40, respectively), whereas total and outer retinal MinI yielded negative correlations (R2 = 0.10 and 0.22). The multivariate model improved correlations (R2 = 0.66) yielding a root mean squared error of 3.8 decibel (dB). Feature importance analysis identified EZ loss as the most relevant predictor of function.

Conclusions

Multiple OCT-derived quantitative metrics used in combination can provide information to predict local sensitivities. The results indicate a strong relationship between retinal function and OCT measures, which contribute to the understanding of the retinal toxicity caused by HCQ as well as being applicable to outcome development for other degenerative diseases of the outer retina.

Noise Reduction for SD-OCT Using a Structure-Preserving Domain Transfer Approach

Spectral-domain optical coherence tomography (SD-OCT) images inevitably suffer from multiplicative speckle noise caused by random interference. This study proposes an unsupervised domain adaptation approach for noise reduction by translating the SD-OCT to the corresponding high-quality enhanced depth imaging (EDI)-OCT. We propose a structure-persevered cycle-consistent generative adversarial network for unpaired image-to-image translation, which can be applied to imbalanced unpaired data, and can effectively preserve retinal details based on a structure-specific cross-domain description. It also imposes smoothness by penalizing the intensity variation of the low reflective region between consecutive slices. Our approach was tested on a local data set that consisted of 268 SD-OCT volumes and two public independent validation datasets including 20 SD-OCT volumes and 17 B-scans, respectively. Experimental results show that our method can effectively suppress noise and maintain the retinal structure, compared with other traditional approaches and deep learning methods in terms of qualitative and quantitative assessments. Our proposed method shows good performance for speckle noise reduction and can assist downstream tasks of OCT analysis.

A novel 3D segmentation approach for extracting retinal layers from optical coherence tomography images

PURPOSE

Accurate segmentation of retinal layers of the eye in 3D Optical Coherence Tomography (OCT) data provides relevant information for clinical diagnosis. This manuscript describes a 3D segmentation approach that uses an adaptive patient-specific retinal atlas, as well as an appearance model for 3D OCT data.

METHODS

To reconstruct the atlas of 3D retinal scan, the central area of the macula (macula mid-area) where the fovea could be clearly identified, was segmented initially. Markov Gibbs Random Field (MGRF) including intensity, spatial information, and shape of 12 retinal layers were used to segment the selected area of retinal fovea. A set of coregistered OCT scans that were gathered from 200 different individuals were used to build a 2D shape prior. This shape prior was adapted subsequently to the first order appearance and second order spatial interaction MGRF model. After segmenting the center of the macula "foveal area", the labels and appearances of the layers that were segmented were utilized to segment the adjacent slices. The final step was repeated recursively until a 3D OCT scan of the patient was segmented.

RESULTS

This approach was tested in 50 patients with normal and with ocular pathological conditions. The segmentation was compared to a manually segmented ground truth. The results were verified by clinical retinal experts. Dice Similarity Coefficient (DSC), 95% bidirectional modified Hausdorff Distance (HD), Unsigned Mean Surface Position Error (MSPE), and Average Volume Difference (AVD) metrics were used to quantify the performance of the proposed approach. The proposed approach was proved to be more accurate than the current state-of-the-art 3D OCT approaches.

CONCLUSIONS

The proposed approach has the advantage of segmenting all the 12 retinal layers rapidly and more accurately than current state-of-the-art 3D OCT approaches.

3D Retinal Vessel Density Mapping With OCT-Angiography

Optical Coherence Tomography Angiography (OCTA) is a novel, non-invasive imaging modality of retinal capillaries at micron resolution. Recent studies have correlated macular OCTA vascular measures with retinal disease severity and supported their use as a diagnostic tool. However, these measurements mostly rely on a few summary statistics in retinal layers or regions of interest in the two-dimensional (2D) en face projection images. To enable 3D and localized comparisons of retinal vasculature between longitudinal scans and across populations, we develop a novel approach for mapping retinal vessel density from OCTA images. We first obtain a high-quality 3D representation of OCTA-based vessel networks via curvelet-based denoising and optimally oriented flux (OOF). Then, an effective 3D retinal vessel density mapping method is proposed. In this framework, a vessel density image (VDI) is constructed by diffusing the vessel mask derived from OOF-based analysis to the entire image volume. Subsequently, we utilize a non-linear, 3D OCT image registration method to provide localized comparisons of retinal vasculature across subjects. In our experimental results, we demonstrate an application of our method for longitudinal qualitative analysis of two pathological subjects with edema during the course of clinical care. Additionally, we quantitatively validate our method on synthetic data with simulated capillary dropout, a dataset obtained from a normal control (NC) population divided into two age groups and a dataset obtained from patients with diabetic retinopathy (DR). Our results show that we can successfully detect localized vascular changes caused by simulated capillary loss, normal aging, and DR pathology even in presence of edema. These results demonstrate the potential of the proposed framework in localized detection of microvascular changes and monitoring retinal disease progression.

Structure from motion photogrammetry in ecology: Does the choice of software matter?

Image-based modeling, and more precisely, Structure from Motion (SfM) and Multi-View Stereo (MVS), is emerging as a flexible, self-service, remote sensing tool for generating fine-grained digital surface models (DSMs) in the Earth sciences and ecology. However, drone-based SfM + MVS applications have developed at a rapid pace over the past decade and there are now many software options available for data processing. Consequently, understanding of reproducibility issues caused by variations in software choice and their influence on data quality is relatively poorly understood. This understanding is crucial for the development of SfM + MVS if it is to fulfill a role as a new quantitative remote sensing tool to inform management frameworks and species conservation schemes. To address this knowledge gap, a lightweight multirotor drone carrying a Ricoh GR II consumer-grade camera was used to capture replicate, centimeter-resolution image datasets of a temperate, intensively managed grassland ecosystem. These data allowed the exploration of method reproducibility and the impact of SfM + MVS software choice on derived vegetation canopy height measurement accuracy. The quality of DSM height measurements derived from four different, yet widely used SfM-MVS software-Photoscan, Pix4D, 3DFlow Zephyr, and MICMAC, was compared with in situ data captured on the same day as image capture. We used both traditional agronomic techniques for measuring sward height, and a high accuracy and precision differential GPS survey to generate independent measurements of the underlying ground surface elevation. Using the same replicate image dataset (n = 3) as input, we demonstrate that there are 1.7, 2.0, and 2.5 cm differences in RMSE (excluding one outlier) between the outputs from different SfM + MVS software using High, Medium, and Low quality settings, respectively. Furthermore, we show that there can be a significant difference, although of small overall magnitude between replicate image datasets (n = 3) processed using the same SfM + MVS software, following the same workflow, with a variance in RMSE of up to 1.3, 1.5, and 2.7 cm (excluding one outlier) for "High," "Medium," and "Low" quality settings, respectively. We conclude that SfM + MVS software choice does matter, although the differences between products processed using "High" and "Medium" quality settings are of small overall magnitude.

Optical Coherence Tomography Analysis Based Prediction of Humphrey 24-2 Visual Field Thresholds in Patients With Glaucoma

Purpose

A pilot study showed that prediction of individual Humphrey 24-2 visual field (HVF 24-2) sensitivity thresholds from optical coherence tomography (OCT) image analysis is possible. We evaluate performance of an improved approach as well as 3 other predictive algorithms on a new, fully independent set of glaucoma subjects.

Methods

Subjects underwent HVF 24-2 and 9-field OCT (Heidelberg Spectralis) testing. Nerve fiber (NFL), and ganglion cell and inner plexiform (GCL+IPL) layers were cosegmented and partitioned into 52 sectors matching HVF 24-2 test locations. The Wilcoxon rank sum test was applied to test correlation R, root mean square error (RMSE), and limits of agreement (LoA) between actual and predicted thresholds for four prediction models. The training data consisted of the 9-field OCT and HVF 24-2 thresholds of 111 glaucoma patients from our pilot study.

Results

We studied 112 subjects (112 eyes) with early, moderate, or advanced primary and secondary open angle glaucoma. Subjects with less than 9 scans (15/112) or insufficient quality segmentations (11/97) were excluded. Retinal ganglion cell axonal complex (RGC-AC) optimized had superior average R = 0.74 (95% confidence interval [CI], 0.67-0.76) and RMSE = 5.42 (95% CI, 5.1-5.7) dB, which was significantly better (P < 0.05/3) than the other three models: Naïve (R = 0.49; 95% CI, 0.44-0.54; RMSE = 7.24 dB; 95% CI, 6.6-7.8 dB), Garway-Heath (R = 0.66; 95% CI, 0.60-0.68; RMSE = 6.07 dB; 95% CI, 5.7-6.5 dB), and Donut (R = 0.67; 95% CI, 0.61-0.69; RMSE = 6.08 dB, 95% CI, 5.8-6.4 dB).

Conclusions

The proposed RGC-AC optimized predictive algorithm based on 9-field OCT image analysis and the RGC-AC concept is superior to previous methods and its performance is close to the reproducibility of HVF 24-2.

The Edge Detectors Suitable for Retinal OCT Image Segmentation

Retinal layer thickness measurement offers important information for reliable diagnosis of retinal diseases and for the evaluation of disease development and medical treatment responses. This task critically depends on the accurate edge detection of the retinal layers in OCT images. Here, we intended to search for the most suitable edge detectors for the retinal OCT image segmentation task. The three most promising edge detection algorithms were identified in the related literature: Canny edge detector, the two-pass method, and the EdgeFlow technique. The quantitative evaluation results show that the two-pass method outperforms consistently the Canny detector and the EdgeFlow technique in delineating the retinal layer boundaries in the OCT images. In addition, the mean localization deviation metrics show that the two-pass method caused the smallest edge shifting problem. These findings suggest that the two-pass method is the best among the three algorithms for detecting retinal layer boundaries. The overall better performance of Canny and two-pass methods over EdgeFlow technique implies that the OCT images contain more intensity gradient information than texture changes along the retinal layer boundaries. The results will guide our future efforts in the quantitative analysis of retinal OCT images for the effective use of OCT technologies in the field of ophthalmology.

Automatic Subretinal Fluid Segmentation of Retinal SD-OCT Images With Neurosensory Retinal Detachment Guided by Enface Fundus Imaging

OBJECTIVE

Accurate segmentation of neurosensory retinal detachment (NRD) associated subretinal fluid in spectral domain optical coherence tomography (SD-OCT) is vital for the assessment of central serous chorioretinopathy (CSC). A novel two-stage segmentation algorithm was proposed, guided by Enface fundus imaging.

METHODS

In the first stage, Enface fundus image was segmented using thickness map prior to detecting the fluid-associated abnormalities with diffuse boundaries. In the second stage, the locations of the abnormalities were used to restrict the spatial extent of the fluid region, and a fuzzy level set method with a spatial smoothness constraint was applied to subretinal fluid segmentation in the SD-OCT scans.

RESULTS

Experimental results from 31 retinal SD-OCT volumes with CSC demonstrate that our method can achieve a true positive volume fraction (TPVF), false positive volume fraction (FPVF), and positive predicative value (PPV) of 94.3%, 0.97%, and 93.6%, respectively, for NRD regions. Our approach can also discriminate NRD-associated subretinal fluid from subretinal pigment epithelium fluid associated with pigment epithelial detachment with a TPVF, FPVF, and PPV of 93.8%, 0.40%, and 90.5%, respectively.

CONCLUSION

We report a fully automatic method for the segmentation of subretinal fluid.

SIGNIFICANCE

Our method shows the potential to improve clinical therapy for CSC.

Multimodal registration of SD-OCT volumes and fundus photographs using histograms of oriented gradients

With availability of different retinal imaging modalities such as fundus photography and spectral domain optical coherence tomography (SD-OCT), having a robust and accurate registration scheme to enable utilization of this complementary information is beneficial. The few existing fundus-OCT registration approaches contain a vessel segmentation step, as the retinal blood vessels are the most dominant structures that are in common between the pair of images. However, errors in the vessel segmentation from either modality may cause corresponding errors in the registration. In this paper, we propose a feature-based registration method for registering fundus photographs and SD-OCT projection images that benefits from vasculature structural information without requiring blood vessel segmentation. In particular, after a preprocessing step, a set of control points (CPs) are identified by looking for the corners in the images. Next, each CP is represented by a feature vector which encodes the local structural information via computing the histograms of oriented gradients (HOG) from the neighborhood of each CP. The best matching CPs are identified by calculating the distance of their corresponding feature vectors. After removing the incorrect matches the best affine transform that registers fundus photographs to SD-OCT projection images is computed using the random sample consensus (RANSAC) method. The proposed method was tested on 44 pairs of fundus and SD-OCT projection images of glaucoma patients and the result showed that the proposed method successfully registers the multimodal images and produced a registration error of 25.34 ± 12.34 μm (0.84 ± 0.41 pixels).

Quantitative analysis of retinal OCT

Clinical acceptance of 3-D OCT retinal imaging brought rapid development of quantitative 3-D analysis of retinal layers, vasculature, retinal lesions as well as facilitated new research in retinal diseases. One of the cornerstones of many such analyses is segmentation and thickness quantification of retinal layers and the choroid, with an inherently 3-D simultaneous multi-layer LOGISMOS (Layered Optimal Graph Image Segmentation for Multiple Objects and Surfaces) segmentation approach being extremely well suited for the task. Once retinal layers are segmented, regional thickness, brightness, or texture-based indices of individual layers can be easily determined and thus contribute to our understanding of retinal or optic nerve head (ONH) disease processes and can be employed for determination of disease status, treatment responses, visual function, etc. Out of many applications, examples provided in this paper focus on image-guided therapy and outcome prediction in age-related macular degeneration and on assessing visual function from retinal layer structure in glaucoma.

Tissue thickness calculation in ocular optical coherence tomography

Thickness measurements derived from optical coherence tomography (OCT) images of the eye are a fundamental clinical and research metric, since they provide valuable information regarding the eye's anatomical and physiological characteristics, and can assist in the diagnosis and monitoring of numerous ocular conditions. Despite the importance of these measurements, limited attention has been given to the methods used to estimate thickness in OCT images of the eye. Most current studies employing OCT use an axial thickness metric, but there is evidence that axial thickness measures may be biased by tilt and curvature of the image. In this paper, standard axial thickness calculations are compared with a variety of alternative metrics for estimating tissue thickness. These methods were tested on a data set of wide-field chorio-retinal OCT scans (field of view (FOV) 60° x 25°) to examine their performance across a wide region of interest and to demonstrate the potential effect of curvature of the posterior segment of the eye on the thickness estimates. Similarly, the effect of image tilt was systematically examined with the same range of proposed metrics. The results demonstrate that image tilt and curvature of the posterior segment can affect axial tissue thickness calculations, while alternative metrics, which are not biased by these effects, should be considered. This study demonstrates the need to consider alternative methods to calculate tissue thickness in order to avoid measurement error due to image tilt and curvature.

Three dimensional data-driven multi scale atomic representation of optical coherence tomography

In this paper, we discuss about applications of different methods for decomposing a signal over elementary waveforms chosen in a family called a dictionary (atomic representations) in optical coherence tomography (OCT). If the representation is learned from the data, a nonparametric dictionary is defined with three fundamental properties of being data-driven, applicability on 3D, and working in multi-scale, which make it appropriate for processing of OCT images. We discuss about application of such representations including complex wavelet based K-SVD, and diffusion wavelets on OCT data. We introduce complex wavelet based K-SVD to take advantage of adaptability in dictionary learning methods to improve the performance of simple dual tree complex wavelets in speckle reduction of OCT datasets in 2D and 3D. The algorithm is evaluated on 144 randomly selected slices from twelve 3D OCTs taken by Topcon 3D OCT-1000 and Cirrus Zeiss Meditec. Improvement of contrast to noise ratio (CNR) (from 0.9 to 11.91 and from 3.09 to 88.9, respectively) is achieved. Furthermore, two approaches are proposed for image segmentation using diffusion. The first method is designing a competition between extended basis functions at each level and the second approach is defining a new distance for each level and clustering based on such distances. A combined algorithm, based on these two methods is then proposed for segmentation of retinal OCTs, which is able to localize 12 boundaries with unsigned border positioning error of 9.22 ±3.05 μm, on a test set of 20 slices selected from 13 3D OCTs.

Relationships of retinal structure and humphrey 24-2 visual field thresholds in patients with glaucoma

PURPOSE

To determine relationships between spectral-domain optical coherence tomography (SD-OCT) derived regional damage to the retinal ganglion cell-axonal complex (RGC-AC) and visual thresholds for each location of the Humphrey 24-2 visual field, in all stages of open-angle glaucoma.

METHODS

Patients with early, moderate, and advanced glaucoma were recruited from a tertiary glaucoma clinic. Humphrey 24-2 and 9-field Spectralis SD-OCT were acquired for each subject. Individual OCT volumes were aligned, nerve fiber layer (NFL), ganglion cell and inner plexiform layers (GCL+IPL) cosegmented. These layers were then partitioned into 54 sectors corresponding to the 24-2 grid. A Support Vector Machine was trained independently for each sector to predict the sector threshold, using these structural properties.

RESULTS

One hundred twenty-two consecutive subjects, 43 early, 39 moderate, and 40 advanced, glaucoma were included (122 eyes). Average correlation coefficient (R) was 0.68 (0.47-0.82), and average root mean square error (RMSE) was 6.92 dB (3.93-8.68 dB). Prediction performance averaged over the entire field, superior hemifield, and inferior hemifield had R (RMSE) values of 0.77 (3.76), 0.80 (5.05), and 0.84 (3.80) dB, respectively.

CONCLUSIONS

Predicting individual 24-2 visual field thresholds from structural information derived from nine-field SD-OCT local NFL and GCL+IPL thicknesses using the RGC-AC concept is feasible, showing the potential for the predictive ability of SD-OCT structural information for visual function. Ultimately, it may be feasible to complement and reduce the burden of subjective visual field testing in glaucoma patients with predicted function derived objectively from OCT.