Automated segmentation of hyperreflective foci in spectral domain optical coherence tomography with diabetic retinopathy

Automatic exudate and aneurysm segmentation in OCT images using UNET++ and hyperreflective-foci feature based bagged tree ensemble

Diabetic retinopathy's signs, such as exudates (EXs) and aneurysms (ANs), initially develop from under the retinal surface detectable from optical coherence tomography (OCT) images. Detecting these signs helps ophthalmologists diagnose DR sooner. Detecting and segmenting exudates (EXs) and aneurysms (ANs) in medical images is challenging due to their small size, similarity to other hyperreflective regions, noise presence, and low background contrast. Furthermore, the scarcity of public OCT images featuring these abnormalities has limited the number of studies related to the automatic segmentation of EXs and ANs, and the reported performance of such studies has not been satisfactory. This work proposes an efficient algorithm that can automatically segment these anomalies by improving key steps in the process. The potential area where these hyper-reflective EXs and ANs occur was scoped by our method using a deep-learning U-Net++ program. From this area, the candidates for EX-AN were segmented using the adaptive thresholding method. Nine features based on appearances, locations, and shadow markers were extracted from these candidates. They were trained and tested using bagged tree ensemble classifiers to obtain only EX-AN blobs. The proposed method was tested on a collection of a public dataset comprising 80 images with hand-drawn ground truths. The experimental results showed that our method could segment EX-AN blobs with average recall, precision, and F1-measure as 87.9%, 86.1%, and 87.0%, respectively. Its F1-measure drastically outperformed two comparative methods, binary thresholding and watershed (BT-WS) and adaptive thresholding with shadow tracking (AT-ST), by 78.0% and 82.1%, respectively.

Automated evaluation of retinal hyperreflective foci changes in diabetic macular edema patients before and after intravitreal injection

Purpose

Fast and automated reconstruction of retinal hyperreflective foci (HRF) is of great importance for many eye-related disease understanding. In this paper, we introduced a new automated framework, driven by recent advances in deep learning to automatically extract 12 three-dimensional parameters from the segmented hyperreflective foci in optical coherence tomography (OCT).

Methods

Unlike traditional convolutional neural networks, which struggle with long-range feature correlations, we introduce a spatial and channel attention module within the bottleneck layer, integrated into the nnU-Net architecture. Spatial Attention Block aggregates features across spatial locations to capture related features, while Channel Attention Block heightens channel feature contrasts. The proposed model was trained and tested on 162 retinal OCT volumes of patients with diabetic macular edema (DME), yielding robust segmentation outcomes. We further investigate HRF's potential as a biomarker of DME.

Results

Results unveil notable discrepancies in the amount and volume of HRF subtypes. In the whole retinal layer (WR), the mean distance from HRF to the retinal pigmented epithelium was significantly reduced after treatment. In WR, the improvement in central macular thickness resulting from intravitreal injection treatment was positively correlated with the mean distance from HRF subtypes to the fovea.

Conclusion

Our study demonstrates the applicability of OCT for automated quantification of retinal HRF in DME patients, offering an objective, quantitative approach for clinical and research applications.

Segmentation of paracentral acute middle maculopathy lesions in spectral-domain optical coherence tomography images through weakly supervised deep convolutional networks

BACKGROUND AND OBJECTIVES

Spectral-domain optical coherence tomography (SD-OCT) is a valuable tool for non-invasive imaging of the retina, allowing the discovery and visualization of localized lesions, the presence of which is associated with eye diseases. The present study introduces X-Net, a weakly supervised deep-learning framework for automated segmentation of paracentral acute middle maculopathy (PAMM) lesions in retinal SD-OCT images. Despite recent advances in the development of automatic methods for clinical analysis of OCT scans, there remains a scarcity of studies focusing on the automated detection of small retinal focal lesions. Additionally, most existing solutions depend on supervised learning, which can be time-consuming and require extensive image labeling, whereas X-Net offers a solution to these challenges. As far as we can determine, no prior study has addressed the segmentation of PAMM lesions in SD-OCT images.

METHODS

This study leverages 133 SD-OCT retinal images, each containing instances of paracentral acute middle maculopathy lesions. A team of eye experts annotated the PAMM lesions in these images using bounding boxes. Then, labeled data were used to train a U-Net that performs pre-segmentation, producing region labels of pixel-level accuracy. To attain a highly-accurate final segmentation, we introduced X-Net, a novel neural network made up of a master and a slave U-Net. During training, it takes the expert annotated, and pixel-level pre-segment annotated images and employs sophisticated strategies to ensure the highest segmentation accuracy.

RESULTS

The proposed method was rigorously evaluated on clinical retinal images excluded from training and achieved an accuracy of 99% with a high level of similarity between the automatic segmentation and expert annotation, as demonstrated by a mean Intersection-over-Union of 0.8. Alternative methods were tested on the same data. Single-stage neural networks proved insufficient for achieving satisfactory results, confirming that more advanced solutions, such as the proposed method, are necessary. We also found that X-Net using Attention U-net for both the pre-segmentation and X-Net arms for the final segmentation shows comparable performance to the proposed method, suggesting that the proposed approach remains a viable solution even when implemented with variants of the classic U-Net.

CONCLUSIONS

The proposed method exhibits reasonably high performance, validated through quantitative and qualitative evaluations. Medical eye specialists have also verified its validity and accuracy. Thus, it could be a viable tool in the clinical assessment of the retina. Additionally, the demonstrated approach for annotating the training set has proven to be effective in reducing the expert workload.

LTF-NSI: a novel local transfer function based on neighborhood similarity index for medical image enhancement

Medical image is an essential tool used in quantitative and qualitative evaluation of different diseases. Medical imaging methods such as fluorescein angiography (FA), optical coherence tomography angiography (OCTA), computed tomography (CT), optical coherence tomography (OCT), and X-ray are used for diagnosis. These imaging modalities suffer from low contrast, which leads to deterioration in the image quality. Consequently, this causes limitation in the usage of medical images in clinical routine and hindered its potential by depriving clinicians from assessing useful information that are needed in disease monitoring, treatment, progression, and decision-making. To overcome this limitation, we propose a novel local transfer function for medical image enhancement algorithm using the pixel neighborhood constraint. The proposed algorithm uses block-wise intensity distribution to generate the regional similarity index. The regional similarity index transformed each centered pixel in the block, to generate a new similarity image. An intuitive optimization algorithm is utilized to optimize the proposed algorithm parameters. Experimentation results show that the proposed LTF-NSI performs better than the state-of-the-art methods and improves the interpretability and perception of the medical images, which can provide clinicians and computer vision program with good quantitative and qualitative information.

Automatic segmentation of hyperreflective dots via focal priors and visual saliency

PURPOSE

Hyperreflective dots (HRDs) can be observed in spectral domain optical coherence tomography (SD-OCT), which can provide a sensitive marker in the treatment decision process. Quantitative analyses of HRDs are the key to make appropriate decisions on observation, treatment, and retreatment. The purpose of this study is to automatically and accurately segment HRDs in SD-OCT B-scans with diabetic retinopathy (DR).

METHODS

The authors propose an automatic segmentation algorithm of HRDs via focal priors and visual saliency. The algorithm is divided into three stages: segmentation of retinal layers, calculation of the multiscale local contrast saliency map, and adaptive threshold segmentation. First, a method based on improved graph search is used to segment retinal layers to obtain the region of interest (ROI) and the reflectivity estimation of the retinal pigment epithelium (RPE) layer; then, the multiscale local contrast saliency map is obtained by using a local contrast measure, which measures the dissimilarity between the current pixels and corresponding neighborhoods; finally, an adaptive threshold is applied to segment HRDs.

RESULTS

Experimental results on 20 SD-OCT B-scans demonstrate that our method is effective for HRDs segmentation. The average dice similarity coefficient (DSC) and detection accuracy are 71.12% and 85.07%, respectively.

CONCLUSIONS

The proposed method can accurately segment HRDs in SD-OCT B-scans with DR and outperforms current state-of-the-art methods. Our method can provide reliable HRDs segmentation to assist ophthalmologists in clinical diagnosis, treatment, disease monitoring, and progression.

The Role of the Choroid in Stargardt Disease

Stargardt disease is the commonest juvenile macular dystrophy. It is caused by genetic mutations in the ABCA4 gene. Diagnosis is not always straightforward, and various phenocopies exist. Late-onset disease can be misdiagnosed with age-related macular disease. A correct diagnosis is particularly critical because of emergent gene therapies. Stargardt disease is known to affect retinal pigment epithelium and photoreceptors. Many studies have also highlighted the importance of the choroid in the diagnosis, pathophysiology, and progression of the disease. The choroid is in an integral relationship with the retinal pigment epithelium and photoreceptors, and its possible involvement during the disease should be considered. The purpose of this review is to analyze the current diagnostic tools for choroidal evaluation and the extrapolation of useful data for ophthalmologists and researchers studying the disease.

Artificial intelligence model driven by transfer learning for image-based medical diagnosis

Artificial intelligent (AI) systems for clinical-decision support are an important tool in clinical routine. It has become a crucial diagnostic tool with adequate reliability and interpretability in disease diagnosis and monitoring. Undoubtedly, these models are faced with insufficient data challenges for training, which often directly determines the model’s performance. In order word, insufficient data for model training leads to inefficiency in the model built. To overcome this problem, we propose an AI-driven model by transfer learning in accurate diagnosis for medical decision support. Our approach leverages the shortage of data with a pretrained model by training the neural network with a fraction of the new dataset. For this purpose, we utilized the VGG19 network as the backbone network to support our model in integrating known features with the newly learned features for accurate diagnosis and decision making. Integrating this trained model speeds up the training phase and improve the performance of the proposed model. Experimental results show that the proposed model is effective and efficient in diagnosing different medical diseases. As such, we anticipated that this diagnosis tool will ultimately aid in facilitating early treatment of these treatable diseases, which will improve clinical out-comes.

Joint segmentation of multi-class hyper-reflective foci in retinal optical coherence tomography images

Hyper-reflective foci (HRF) refers to the spot-shaped, block-shaped areas with characteristics of high local contrast and high reflectivity, which is mostly observed in retinal optical coherence tomography (OCT) images of patients with fundus diseases. HRF mainly appears hard exudates (HE) and microglia (MG) clinically. Accurate segmentation of HE and MG is essential to alleviate the harm in retinal diseases. However, it is still a challenge to segment HE and MG simultaneously due to similar pathological features, various shapes and location distribution, blurred boundaries, and small morphology dimensions. To tackle these problems, in this paper, we propose a novel global information fusion and dual decoder collaboration-based network (GD-Net), which can segment HE and MG in OCT images jointly. Specifically, to suppress the interference of similar pathological features, a novel global information fusion (GIF) module is proposed, which can aggregate the global semantic information efficiently. To further improve the segmentation performance, we design a dual decoder collaborative workspace (DDCW) to comprehensively utilize the semantic correlation between HE and MG while enhancing the mutual influence on them by feedback alternately. To further optimize GD-Net, we explore a joint loss function which integrates pixel-level with image-level. The dataset of this study comes from patients diagnosed with diabetic macular edema at the department of ophthalmology, University Medical Center Groningen, the Netherlands. Experimental results show that our proposed method performs better than other state-of-the-art methods, which suggests the effectiveness of the proposed method and provides research ideas for medical applications.

Algorithm for Detection and Quantification of Hyperreflective Dots on Optical Coherence Tomography in Diabetic Macular Edema

Purpose: To develop an algorithm to detect and quantify hyperreflective dots (HRDs) on optical coherence tomography (OCT) in patients with diabetic macular edema (DME).

Materials and Methods: Twenty OCTs (each OCT contains 128 b scans) from 20 patients diagnosed with DME were included in this study. Two types of HRDs, hard exudates and small HRDs (hypothesized to be activated microglia), were identified and labeled independently by two raters. An algorithm using deep learning technology was developed based on input (in total 2,560 OCT b scans) of manual labeling and differentiation of HRDs from rater 1. 4-fold cross-validation was used to train and validate the algorithm. Dice coefficient, intraclass coefficient (ICC), correlation coefficient, and Bland–Altman plot were used to evaluate agreement of the output parameters between two methods (either between two raters or between one rater and proposed algorithm).

Results: The Dice coefficients of total HRDs, hard exudates, and small HRDs area of the algorithm were 0.70 ± 0.10, 0.72 ± 0.11, and 0.46 ± 0.06, respectively. The correlations between rater 1 and proposed algorithm (range: 0.95–0.99, all p < 0.001) were stronger than the correlations between the two raters (range: 0.84–0.96, all p < 0.001) for all parameters. The ICCs were higher for all the parameters between rater 1 and proposed algorithm (range: 0.972–0.997) than those between the two raters (range: 0.860–0.953).

Conclusions: Our proposed algorithm is a good tool to detect and quantify HRDs and can provide objective and repeatable information of OCT for DME patients in clinical practice and studies.

Choroidal hyper-reflective foci and vascularity in retinal dystrophy

Purpose:

To investigate choroidal hyper-reflective foci (HRF) in subjects with retinal dystrophy [Stargardt's disease (SGD) and retinitis pigmentosa (RP)] and their association with demographics, visual acuity, choroidal thickness (CT), and choroidal vascularity index (CVI).

Methods:

Single center retrospective study of subjects with previously diagnosed SGD or RP. Swept-source optical coherence tomography images were analyzed for the presence of choroidal HRFs and CVI using previously validated automated algorithm. A Spearman's rank correlation coefficient was used to evaluate the correlation between the number of HRF and various baseline parameters including age, visual acuity, intraocular pressure, and other optical coherence tomography (OCT) parameters (CT, choroidal area, and CVI) were evaluated in these subjects.

Results:

This study included 46 eyes (23 subjects) and 55 eyes (28 subjects) with previously diagnosed RP and SGD, respectively. In the RP group, the mean number of HRFs was 247.9 ± 57.1 and mean CVI was 0.56 ± 0.04. In SGD group, mean HRF was 192.5 ± 44.3 and mean CVI was 0.41 ± 0.04. Mean HRF was significantly greater in the RP group (0.02), however, the mean CVI was not statistically different. In RP, mean HRF were correlated only with CVI (r = 0.49; P = 0.001), however, in SGD, it correlated with only choroidal area (r = 0.27; P = 0.04).

Conclusion:

Choroidal HRF were present in both RP and SGD subjects with more HRFs in those with RP. These HRFs were associated with alteration in choroidal vascularity, which further adds into the pathogenesis of these diseases.

Disease Progression in Patients with Autosomal Dominant Retinitis Pigmentosa due to a Mutation in Inosine Monophosphate Dehydrogenase 1 (IMPDH1)

Purpose

Mutations in the inosine monophosphate dehydrogenase 1 (IMPDH1) gene are a common cause of inherited retinal degeneration (IRD). Due to species- and tissue-dependent expression of IMPDH1, there are no appropriate models of human IMPDH1 disease. Therefore, a limited understanding remains of disease expression and rates of progression for IMPDH1-related IRD.

Methods

We evaluated semiautomated kinetic and chromatic static perimetry, spectral-domain optical coherence tomography (SD-OCT), and ultra-wide field fundus images with autofluorescence in a cohort of 12 patients (ages 11–58 at first visit). Ten patients had longitudinal data for which rates of progression were estimated.

Results

Visual acuities were relatively stable over time and the photoreceptors within the central retina remained intact. Perifoveal photoreceptor loss measured over a period of years coincided with visual fields, which were constricted and progressed over time in all patients. Rod sensitivity showed a similar pattern of defect to that of the kinetic perimetry and the autofluorescence ultra-wide field imaging. Full-field electroretinograms were severely reduced and the dark-adapted rod and mixed responses were extinguished at earlier visits than the light-adapted cone responses.

Conclusions

There was variability in disease severity at the first visit, but results show that the peripheral retina is more susceptible to the deleterious consequences of an IMPDH1 mutation. Given the pattern of degeneration and the alternatively spliced isoforms of IMPDH1, potential interventions may consider targeting the periphery early in disease, modulating transcript expression, and/or preserving central vision at late stages of the disease.

Translational Relevance

These results inform clinical prognosis and offer evidence strategies toward therapeutic intervention.

Fast and Automated Hyperreflective Foci Segmentation Based on Image Enhancement and Improved 3D U-Net in SD-OCT Volumes with Diabetic Retinopathy

Purpose

To design a robust and automated hyperreflective foci (HRF) segmentation framework for spectral-domain optical coherence tomography (SD-OCT) volumes, especially volumes with low HRF-background contrast.

Methods

HRF in retinal SD-OCT volumes appear with low-contrast characteristics that results in the difficulty of HRF segmentation. Therefore to effectively segment the HRF we proposed a fully automated method for HRF segmentation in SD-OCT volumes with diabetic retinopathy (DR). First, we generated the enhanced SD-OCT images from the denoised SD-OCT images with an enhancement method. Then the enhanced images were cascaded with the denoised images as the two-channel input to the network against the low-contrast HRF. Finally, we replaced the standard convolution with slice-wise dilated convolution in the last layer of the encoder path of 3D U-Net to obtain long-range information.

Results

We evaluated our method using two-fold cross-validation on 33 SD-OCT volumes from 27 patients. The average dice similarity coefficient was 70.73%, which was higher than that of the existing methods with significant difference (P < 0.01).

Conclusions

Experimental results demonstrated that the proposed method is faster and achieves more reliable segmentation results than the current HRF segmentation algorithms. We expect that this method will contribute to clinical diagnosis and disease surveillance.

Translational Relevance

Our framework for the automated HRF segmentation of SD-OCT volumes may improve the clinical diagnosis of DR.

Hyperreflective Foci Enhancement in a Combined Spatial-Transform Domain for SD-OCT Images

Purpose

Spectral-domain optical coherent tomography (SD-OCT) is a useful tool for visualizing, treating, and monitoring retinal abnormality in patients with different retinal diseases. However, the assessment of SD-OCT images is thwarted by the lack of image quality necessary for ophthalmologists to analyze and quantify the diseases. This has hindered the potential role of hyperreflective foci (HRF) as a prognostic indicator of visual outcome in patients with retinal diseases. We present a new multi-vendor algorithm that is robust to noise while enhancing the HRF in SD-OCT images.

Methods

The proposed algorithm processes the SD-OCT images in two parallel processes simultaneously. The two parallel processes are combined by histogram matching. An inverse of both logarithmic and orthogonal transforms is applied to the mapped data to produce the enhanced image.

Results

We evaluated our algorithm on a dataset composed of 40 SD-OCT volumes. The proposed method obtained high values for the measure of enhancement, peak signal-to-noise ratio, structure similarity, and correlation (ρ) and a low value for mean square error of 36.72, 38.87, 0.87, 0.98, and 25.12 for Cirrus; 40.77, 41.84, 0.89, 0.98, and 22.15 for Spectralis; and 30.81, 32.10, 0.81, 0.96, and 28.55 for Topcon SD-OCT devices, respectively.

Conclusions

The proposed algorithm can be used in the medical field to assist ophthalmologists and in the preprocessing of medical images.

Translational Relevance

The proposed enhancement algorithm facilitates the visualization and detection of HRF, which is a step forward in assisting clinicians with decision making about patient treatment planning and disease monitoring.

Retinal hyperreflective foci in Fabry disease

Background

Fabry disease (FD) is an X-linked inherited storage disorder caused by deficiency of lysosomal alpha-Galactosidase A. Here we describe new retinal findings in patients with FD assessed by Spectral domain optical coherence tomography (SD-OCT) and their possible clinical relevance.

Methods

54 eyes of 27 FD patients and 54 eyes of 27 control subjects were included. The ophthalmic examination included visual acuity testing, tonometry, slit lamp and fundus examination. SD-OCT imaging of the macula was performed in all subjects. Central retinal thickness and retinal nerve fiber layer analysis were quantified. Vessel tortuosity was obtained by a subjective scoring and mathematically calculated. Inner retinal hyperreflective foci (HRF) were quantified, clinically graded and correlated with a biomarker of Fabry disease (lyso-Gb3).

Results

In comparison to an age-matched control group, a significant amount of HRF was identified in macular SD-OCT images in FD patients. These HRF were localized within the inner retinal layers. Furthermore, lyso-Gb3 levels correlated significantly with the quantitative evaluation of HRF (p < 0,001). In addition, the vessel tortuosity was remarkably increased in FD patients compared to control persons and correlated significantly with lyso-G3 levels (p = 0.005). A further subanalysis revealed significantly higher HRF and vessel tortuosity scores in male patients with the classic FD phenotype.

Conclusions

The observational, cross sectional, comparative study describes novel intraretinal findings in patients with FD. We were able to identify suspicious HRF within the inner retinal layers. These findings were not accompanied by functional limitations, as visual acuity remained unchanged. However, HRF correlated well with lyso-Gb3, a degradation product of the accumulating protein Gb3 and might potentially indicate Gb3 accumulation within the highly metabolic and densely vascularized macula.

Automated Quantification of Hyperreflective Foci in SD-OCT With Diabetic Retinopathy

The presence of hyperreflective foci (HFs) is related to retinal disease progression, and the quantity has proven to be a prognostic factor of visual and anatomical outcome in various retinal diseases. However, lack of efficient quantitative tools for evaluating the HFs has deprived ophthalmologist of assessing the volume of HFs. For this reason, we propose an automated quantification algorithm to segment and quantify HFs in spectral domain optical coherence tomography (SD-OCT). The proposed algorithm consists of two parallel processes namely: region of interest (ROI) generation and HFs estimation. To generate the ROI, we use morphological reconstruction to obtain the reconstructed image and histogram constructed for data distributions and clustering. In parallel, we estimate the HFs by extracting the extremal regions from the connected regions obtained from a component tree. Finally, both the ROI and the HFs estimation process are merged to obtain the segmented HFs. The proposed algorithm was tested on 40 3D SD-OCT volumes from 40 patients diagnosed with non-proliferative diabetic retinopathy (NPDR), proliferative diabetic retinopathy (PDR), and diabetic macular edema (DME). The average dice similarity coefficient (DSC) and correlation coefficient (r) are 69.70%, 0.99 for NPDR, 70.31%, 0.99 for PDR, and 71.30%, 0.99 for DME, respectively. The proposed algorithm can provide ophthalmologist with good HFs quantitative information, such as volume, size, and location of the HFs.