Improving graph-based OCT segmentation for severe pathology in Retinitis Pigmentosa patients

Longitudinal deep network for consistent OCT layer segmentation

Retinal layer thickness is an important bio-marker for people with multiple sclerosis (PwMS). In clinical practice, retinal layer thickness changes in optical coherence tomography (OCT) are widely used for monitoring multiple sclerosis (MS) progression. Recent developments in automated retinal layer segmentation algorithms allow cohort-level retina thinning to be observed in a large study of PwMS. However, variability in these results make it difficult to identify patient-level trends; this prevents patient specific disease monitoring and treatment planning using OCT. Deep learning based retinal layer segmentation algorithms have achieved state-of-the-art accuracy, but the segmentation is performed on each individual scan without utilizing longitudinal information, which can be important in reducing segmentation error and reveal subtle changes in retinal layers. In this paper, we propose a longitudinal OCT segmentation network which achieves more accurate and consistent layer thickness measurements for PwMS.

Performance of Deep Learning Models in Automatic Measurement of Ellipsoid Zone Area on Baseline Optical Coherence Tomography (OCT) Images From the Rate of Progression of USH2A-Related Retinal Degeneration (RUSH2A) Study

Purpose

Previously, we have shown the capability of a hybrid deep learning (DL) model that combines a U-Net and a sliding-window (SW) convolutional neural network (CNN) for automatic segmentation of retinal layers from OCT scan images in retinitis pigmentosa (RP). We found that one of the shortcomings of the hybrid model is that it tends to underestimate ellipsoid zone (EZ) width or area, especially when EZ extends toward or beyond the edge of the macula. In this study, we trained the model with additional data which included more OCT scans having extended EZ. We evaluated its performance in automatic measurement of EZ area on SD-OCT volume scans obtained from the participants of the RUSH2A natural history study by comparing the model’s performance to the reading center’s manual grading.

Materials and Methods

De-identified Spectralis high-resolution 9-mm 121-line macular volume scans as well as their EZ area measurements by a reading center were transferred from the management center of the RUSH2A study under the data transfer and processing agreement. A total of 86 baseline volume scans from 86 participants of the RUSH2A study were included to evaluate two hybrid models: the original RP240 model trained on 480 mid-line B-scans from 220 patients with retinitis pigmentosa (RP) and 20 participants with normal vision from a single site, and the new RP340 model trained on a revised RP340 dataset which included RP240 dataset plus an additional 200 mid-line B-scans from another 100 patients with RP. There was no overlap of patients between training and evaluation datasets. EZ and apical RPE in each B-scan image were automatically segmented by the hybrid model. EZ areas were determined by interpolating the discrete 2-dimensional B-scan EZ-RPE layer over the scan area. Dice similarity, correlation, linear regression, and Bland-Altman analyses were conducted to assess the agreement between the EZ areas measured by the hybrid model and by the reading center.

Results

For EZ area > 1 mm2, average dice coefficients ± SD between the EZ band segmentations determined by the DL model and the manual grading were 0.835 ± 0.132 and 0.867 ± 0.105 for RP240 and RP340 hybrid models, respectively (p < 0.0005; n = 51). When compared to the manual grading, correlation coefficients (95% CI) were 0.991 (0.987–0.994) and 0.994 (0.991–0.996) for RP240 and RP340 hybrid models, respectively. Linear regression slopes (95% CI) were 0.918 (0.896–0.940) and 0.995 (0.975–1.014), respectively. Bland-Altman analysis revealed a mean difference ± SD of -0.137 ± 1.131 mm2 and 0.082 ± 0.825 mm2, respectively.

Conclusion

Additional training data improved the hybrid model’s performance, especially reducing the bias and narrowing the range of the 95% limit of agreement when compared to manual grading. The close agreement of DL models to manual grading suggests that DL may provide effective tools to significantly reduce the burden of reading centers to analyze OCT scan images. In addition to EZ area, our DL models can also provide the measurements of photoreceptor outer segment volume and thickness to further help assess disease progression and to facilitate the study of structure and function relationship in RP.

A Hybrid Model Composed of Two Convolutional Neural Networks (CNNs) for Automatic Retinal Layer Segmentation of OCT Images in Retinitis Pigmentosa (RP)

Purpose

We propose and evaluate a hybrid model composed of two convolutional neural networks (CNNs) with different architectures for automatic segmentation of retina layers in spectral domain optical coherence tomography (SD-OCT) B-scans of retinitis pigmentosa (RP).

Methods

The hybrid model consisted of a U-Net for initial semantic segmentation and a sliding-window (SW) CNN for refinement by correcting the segmentation errors of U-Net. The U-Net construction followed Ronneberger et al. (2015) with an input image size of 256 × 32. The SW model was similar to our previously reported approach. Training image patches were generated from 480 horizontal midline B-scans obtained from 220 patients with RP and 20 normal participants. Testing images were 160 midline B-scans from a separate group of 80 patients with RP. The Spectralis segmentation of B-scans was manually corrected for the boundaries of the inner limiting membrane, inner nuclear layer, ellipsoid zone (EZ), retinal pigment epithelium, and Bruch's membrane by one grader for the training set and two for the testing set. The trained U-Net and SW, as well as the hybrid model, were used to classify all pixels in the testing B-scans. Bland–Altman and correlation analyses were conducted to compare layer boundary lines, EZ width, and photoreceptor outer segment (OS) length and area determined by the models to those by human graders.

Results

The mean times to classify a B-scan image were 0.3, 65.7, and 2.4 seconds for U-Net, SW, and the hybrid model, respectively. The mean ± SD accuracies to segment retinal layers were 90.8% ± 4.8% and 90.7% ± 4.0% for U-Net and SW, respectively. The hybrid model improved mean ± SD accuracy to 91.5% ± 4.8% (P < 0.039 vs. U-Net), resulting in an improvement in layer boundary segmentation as revealed by Bland–Altman analyses. EZ width, OS length, and OS area measured by the models were highly correlated with those measured by the human graders (r > 0.95 for EZ width; r > 0.83 for OS length; r > 0.97 for OS area; P < 0.05). The hybrid model further improved the performance of measuring retinal layer thickness by correcting misclassification of retinal layers from U-Net.

Conclusions

While the performances of U-Net and the SW model were comparable in delineating various retinal layers, U-Net was much faster than the SW model to segment B-scan images. The hybrid model that combines the two improves automatic retinal layer segmentation from OCT images in RP.

Translational Relevance

A hybrid deep machine learning model composed of CNNs with different architectures can be more effective than either model separately for automatic analysis of SD-OCT scan images, which is becoming increasingly necessary with current high-resolution, high-density volume scans.

Structured layer surface segmentation for retina OCT using fully convolutional regression networks

Optical coherence tomography (OCT) is a noninvasive imaging modality with micrometer resolution which has been widely used for scanning the retina. Retinal layers are important biomarkers for many diseases. Accurate automated algorithms for segmenting smooth continuous layer surfaces with correct hierarchy (topology) are important for automated retinal thickness and surface shape analysis. State-of-the-art methods typically use a two step process. Firstly, a trained classifier is used to label each pixel into either background and layers or boundaries and non-boundaries. Secondly, the desired smooth surfaces with the correct topology are extracted by graph methods (e.g., graph cut). Data driven methods like deep networks have shown great ability for the pixel classification step, but to date have not been able to extract structured smooth continuous surfaces with topological constraints in the second step. In this paper, we combine these two steps into a unified deep learning framework by directly modeling the distribution of the surface positions. Smooth, continuous, and topologically correct surfaces are obtained in a single feed forward operation. The proposed method was evaluated on two publicly available data sets of healthy controls and subjects with either multiple sclerosis or diabetic macular edema, and is shown to achieve state-of-the art performance with sub-pixel accuracy.

Deep learning based topology guaranteed surface and MME segmentation of multiple sclerosis subjects from retinal OCT

Optical coherence tomography (OCT) is a noninvasive imaging modality that can be used to obtain depth images of the retina. Patients with multiple sclerosis (MS) have thinning retinal nerve fiber and ganglion cell layers, and approximately 5% of MS patients will develop microcystic macular edema (MME) within the retina. Segmentation of both the retinal layers and MME can provide important information to help monitor MS progression. Graph-based segmentation with machine learning preprocessing is the leading method for retinal layer segmentation, providing accurate surface delineations with the correct topological ordering. However, graph methods are time-consuming and they do not optimally incorporate joint MME segmentation. This paper presents a deep network that extracts continuous, smooth, and topology-guaranteed surfaces and MMEs. The network learns shape priors automatically during training rather than being hard-coded as in graph methods. In this new approach, retinal surfaces and MMEs are segmented together with two cascaded deep networks in a single feed forward propagation. The proposed framework obtains retinal surfaces (separating the layers) with sub-pixel surface accuracy comparable to the best existing graph methods and MMEs with better accuracy than the state-of-the-art method. The full segmentation operation takes only ten seconds for a 3D volume.

Layer boundary evolution method for macular OCT layer segmentation

Optical coherence tomography (OCT) is used to produce high resolution depth images of the retina and is now the standard of care for in-vivo ophthalmological assessment. It is also increasingly being used for evaluation of neurological disorders such as multiple sclerosis (MS). Automatic segmentation methods identify the retinal layers of the macular cube providing consistent results without intra- and inter-rater variation and is faster than manual segmentation. In this paper, we propose a fast multi-layer macular OCT segmentation method based on a fast level set method. Our framework uses contours in an optimized approach specifically for OCT layer segmentation over the whole macular cube. Our algorithm takes boundary probability maps from a trained random forest and iteratively refines the prediction to subvoxel precision. Evaluation on both healthy and multiple sclerosis subjects shows that our method is statistically better than a state-of-the-art graph-based method.

Retinal layer parcellation of optical coherence tomography images: Data resource for multiple sclerosis and healthy controls

This paper presents optical coherence tomography (OCT) images of the human retina and manual delineations of eight retinal layers. The data includes 35 human retina scans acquired on a Spectralis OCT system (Heidelberg Engineering, Heidelberg, Germany), 14 of which are healthy controls (HC) and 21 have a diagnosis of multiple sclerosis (MS). The provided data includes manually delineation of eight retina layers, which were independently reviewed and edited. The data presented in this article was used to validate automatic segmentation algorithms (Lang et al., 2013).

Prevalences of segmentation errors and motion artifacts in OCT-angiography differ among retinal diseases

PURPOSE

To assess the prevalences of segmentation errors and motion artifacts in optical coherence tomography angiography (OCT-A) in different retinal diseases METHODS: In a retrospective analysis, multimodal retinal imaging including OCT-A was performed in one eye of 57 healthy controls (50.96 ± 22.4 years) and 149 patients (66.42 ± 14.1 years) affected by different chorioretinal diseases: early/intermediate age-related macular degeneration (AMD; n = 26), neovascular AMD (nAMD; n = 22), geographic atrophy due to AMD (GA; n = 6), glaucoma (n = 28), central serous chorioretinopathy (CSC; n = 14), epiretinal membrane (EM; n = 26), retinal vein occlusion (RVO; n = 11), and retinitis pigmentosa (RP; n = 16). Central 3 × 3 mm² OCT-A imaging was performed with active eye-tracking (AngioVue, Optovue). Best-corrected visual acuity (BCVA) and signal strength index (SSI) were recorded. Images were independently evaluated by two graders using the OCT-A motion artifact score (MAS; scores I-IV) as well as a newly introduced segmentation accuracy score (SAS; score I-IIB).

RESULTS

Mean SSI was 63.67 ± 9.2 showing a negative correlation with increasing age (rSp = - 0.42, p < 0.001, n = 206). In the healthy cohort, mean MAS was 1.45 ± 0.8 and segmentation was accurate (SAS I) in all eyes. In eyes with retinal pathologies, mean MAS was 2.1 ± 0.9 (p < 0.001). Lowest MAS was observed in GA (2.67 ± 0.5) and RVO (2.45 ± 1.1). Compared to an accurate segmentation in 100% in healthy subjects, 34.2% (n = 51) of all patients showed highest segmentation quality (p < 0.001). 63.8% showed segmentation errors in more than 5% of all single b-scans in one (SAS IIA, n = 58) or at least two (SAS IIB, n = 40) segmentation boundaries. Highest percentages of inaccurate segmentation (SAS IIA or IIB) were observed in the nAMD group (90.1%). The inner plexiform layer was the segmentation boundary most prone to inaccurate segmentation in all pathologies compared to the inner limiting membrane (ILM) and retinal pigment epithelium (RPE) segmentation layer. Incorrect ILM segmentation was only seen in patients with EM.

CONCLUSIONS

Prior to both qualitative and quantitative analysis, OCT-A images must be carefully reviewed as motion artifacts and segmentation errors in current OCT-A technology are frequent particularly in pathologically altered maculae.

Deep learning approach for the detection and quantification of intraretinal cystoid fluid in multivendor optical coherence tomography

We developed a deep learning algorithm for the automatic segmentation and quantification of intraretinal cystoid fluid (IRC) in spectral domain optical coherence tomography (SD-OCT) volumes independent of the device used for acquisition. A cascade of neural networks was introduced to include prior information on the retinal anatomy, boosting performance significantly. The proposed algorithm approached human performance reaching an overall Dice coefficient of 0.754 ± 0.136 and an intraclass correlation coefficient of 0.936, for the task of IRC segmentation and quantification, respectively. The proposed method allows for fast quantitative IRC volume measurements that can be used to improve patient care, reduce costs, and allow fast and reliable analysis in large population studies.

Intensity inhomogeneity correction of SD-OCT data using macular flatspace

Images of the retina acquired using optical coherence tomography (OCT) often suffer from intensity inhomogeneity problems that degrade both the quality of the images and the performance of automated algorithms utilized to measure structural changes. This intensity variation has many causes, including off-axis acquisition, signal attenuation, multi-frame averaging, and vignetting, making it difficult to correct the data in a fundamental way. This paper presents a method for inhomogeneity correction by acting to reduce the variability of intensities within each layer. In particular, the N3 algorithm, which is popular in neuroimage analysis, is adapted to work for OCT data. N3 works by sharpening the intensity histogram, which reduces the variation of intensities within different classes. To apply it here, the data are first converted to a standardized space called macular flat space (MFS). MFS allows the intensities within each layer to be more easily normalized by removing the natural curvature of the retina. N3 is then run on the MFS data using a modified smoothing model, which improves the efficiency of the original algorithm. We show that our method more accurately corrects gain fields on synthetic OCT data when compared to running N3 on non-flattened data. It also reduces the overall variability of the intensities within each layer, without sacrificing contrast between layers, and improves the performance of registration between OCT images.