Deep learning in retinal optical coherence tomography (OCT): A comprehensive survey

Wavelet-based selection-and-recalibration network for Parkinson's disease screening in OCT images

BACKGROUND AND OBJECTIVE

Parkinson's disease (PD) is one of the most prevalent neurodegenerative brain diseases worldwide. Therefore, accurate PD screening is crucial for early clinical intervention and treatment. Recent clinical research indicates that changes in pathology, such as the texture and thickness of the retinal layers, can serve as biomarkers for clinical PD diagnosis based on optical coherence tomography (OCT) images. However, the pathological manifestations of PD in the retinal layers are subtle compared to the more salient lesions associated with retinal diseases.

METHODS

Inspired by textural edge feature extraction in frequency domain learning, we aim to explore a potential approach to enhance the distinction between the feature distributions in retinal layers of PD cases and healthy controls. In this paper, we introduce a simple yet novel wavelet-based selection and recalibration module to effectively enhance the feature representations of the deep neural network by aggregating the unique clinical properties, such as the retinal layers in each frequency band. We combine this module with the residual block to form a deep network named Wavelet-based Selection and Recalibration Network (WaveSRNet) for automatic PD screening.

RESULTS

The extensive experiments on a clinical PD-OCT dataset and two publicly available datasets demonstrate that our approach outperforms state-of-the-art methods. Visualization analysis and ablation studies are conducted to enhance the explainability of WaveSRNet in the decision-making process.

CONCLUSIONS

Our results suggest the potential role of the retina as an assessment tool for PD. Visual analysis shows that PD-related elements include not only certain retinal layers but also the location of the fovea in OCT images.

Revolutionary Point-of-Care Wearable Diagnostics for Early Disease Detection and Biomarker Discovery through Intelligent Technologies

Early-stage disease detection, particularly in Point-Of-Care (POC) wearable formats, assumes pivotal role in advancing healthcare services and precision-medicine. Public benefits of early detection extend beyond cost-effectively promoting healthcare outcomes, to also include reducing the risk of comorbid diseases. Technological advancements enabling POC biomarker recognition empower discovery of new markers for various health conditions. Integration of POC wearables for biomarker detection with intelligent frameworks represents ground-breaking innovations enabling automation of operations, conducting advanced large-scale data analysis, generating predictive models, and facilitating remote and guided clinical decision-making. These advancements substantially alleviate socioeconomic burdens, creating a paradigm shift in diagnostics, and revolutionizing medical assessments and technology development. This review explores critical topics and recent progress in development of 1) POC systems and wearable solutions for early disease detection and physiological monitoring, as well as 2) discussing current trends in adoption of smart technologies within clinical settings and in developing biological assays, and ultimately 3) exploring utilities of POC systems and smart platforms for biomarker discovery. Additionally, the review explores technology translation from research labs to broader applications. It also addresses associated risks, biases, and challenges of widespread Artificial Intelligence (AI) integration in diagnostics systems, while systematically outlining potential prospects, current challenges, and opportunities.

A deep learning approach to explore the association of age-related macular degeneration polygenic risk score with retinal optical coherence tomography: A preliminary study

PURPOSE

Age-related macular degeneration (AMD) is a complex eye disorder affecting millions worldwide. This article uses deep learning techniques to investigate the relationship between AMD, genetics and optical coherence tomography (OCT) scans.

METHODS

The cohort consisted of 332 patients, of which 235 were diagnosed with AMD and 97 were controls with no signs of AMD. The genome-wide association studies summary statistics utilized to establish the polygenic risk score (PRS) in relation to AMD were derived from the GERA European study. A PRS estimation based on OCT volumes for both eyes was performed using a proprietary convolutional neural network (CNN) model supported by machine learning models. The method's performance was assessed using numerical evaluation metrics, and the Grad-CAM technique was used to evaluate the results by visualizing the features learned by the model.

RESULTS

The best results were obtained with the CNN and the Extra Tree regressor (MAE = 0.55, MSE = 0.49, RMSE = 0.70, R2 = 0.34). Extending the feature vector with additional information on AMD diagnosis, age and smoking history improved the results slightly, with mainly AMD diagnosis used by the model (MAE = 0.54, MSE = 0.44, RMSE = 0.66, R2 = 0.42). Grad-CAM heatmap evaluation showed that the model decisions rely on retinal morphology factors relevant to AMD diagnosis.

CONCLUSION

The developed method allows an efficient PRS estimation from OCT images. A new technique for analysing the association of OCT images with PRS of AMD, using a deep learning approach, may provide an opportunity to discover new associations between genotype-based AMD risk and retinal morphology.

Invariant Content Representation for Generalizable Medical Image Segmentation

Domain generalization (DG) for medical image segmentation due to privacy preservation prefers learning from a single-source domain and expects good robustness on unseen target domains. To achieve this goal, previous methods mainly use data augmentation to expand the distribution of samples and learn invariant content from them. However, most of these methods commonly perform global augmentation, leading to limited augmented sample diversity. In addition, the style of the augmented image is more scattered than the source domain, which may cause the model to overfit the style of the source domain. To address the above issues, we propose an invariant content representation network (ICRN) to enhance the learning of invariant content and suppress the learning of variability styles. Specifically, we first design a gamma correction-based local style augmentation (LSA) to expand the distribution of samples by augmenting foreground and background styles, respectively. Then, based on the augmented samples, we introduce invariant content learning (ICL) to learn generalizable invariant content from both augmented and source-domain samples. Finally, we design domain-specific batch normalization (DSBN) based style adversarial learning (SAL) to suppress the learning of preferences for source-domain styles. Experimental results show that our proposed method improves by 8.74% and 11.33% in overall dice coefficient (Dice) and reduces 15.88 mm and 3.87 mm in overall average surface distance (ASD) on two publicly available cross-domain datasets, Fundus and Prostate, compared to the state-of-the-art DG methods. The code is available at https://github.com/ZMC-IIIM/ICRN-DG .

Biomimicking Atherosclerotic Vessels: A Relevant and (Yet) Sub-Explored Topic

Atherosclerosis represents the etiologic source of several cardiovascular events, including myocardial infarction, cerebrovascular accidents, and peripheral artery disease, which remain the leading cause of mortality in the world. Numerous strategies are being delineated to revert the non-optimal projections of the World Health Organization, by both designing new diagnostic and therapeutic approaches or improving the interventional procedures performed by physicians. Deeply understanding the pathological process of atherosclerosis is, therefore, mandatory to accomplish improved results in these trials. Due to their availability, reproducibility, low expensiveness, and rapid production, biomimicking physical models are preferred over animal experimentation because they can overcome some limitations, mainly related to replicability and ethical issues. Their capability to represent any atherosclerotic stage and/or plaque type makes them valuable tools to investigate hemodynamical, pharmacodynamical, and biomechanical behaviors, as well as to optimize imaging systems and, thus, obtain meaningful prospects to improve the efficacy and effectiveness of treatment on a patient-specific basis. However, the broadness of possible applications in which these biomodels can be used is associated with a wide range of tissue-mimicking materials that are selected depending on the final purpose of the model and, consequently, prioritizing some materials' properties over others. This review aims to summarize the progress in fabricating biomimicking atherosclerotic models, mainly focusing on using materials according to the intended application.

Deep Learning-Assisted Measurements of Photoreceptor Ellipsoid Zone Area and Outer Segment Volume as Biomarkers for Retinitis Pigmentosa

The manual segmentation of retinal layers from OCT scan images is time-consuming and costly. The deep learning approach has potential for the automatic delineation of retinal layers to significantly reduce the burden of human graders. In this study, we compared deep learning model (DLM) segmentation with manual correction (DLM-MC) to conventional manual grading (MG) for the measurements of the photoreceptor ellipsoid zone (EZ) area and outer segment (OS) volume in retinitis pigmentosa (RP) to assess whether DLM-MC can be a new gold standard for retinal layer segmentation and for the measurement of retinal layer metrics. Ninety-six high-speed 9 mm 31-line volume scans obtained from 48 patients with RPGR-associated XLRP were selected based on the following criteria: the presence of an EZ band within the scan limit and a detectable EZ in at least three B-scans in a volume scan. All the B-scan images in each volume scan were manually segmented for the EZ and proximal retinal pigment epithelium (pRPE) by two experienced human graders to serve as the ground truth for comparison. The test volume scans were also segmented by a DLM and then manually corrected for EZ and pRPE by the same two graders to obtain DLM-MC segmentation. The EZ area and OS volume were determined by interpolating the discrete two-dimensional B-scan EZ-pRPE layer over the scan area. Dice similarity, Bland-Altman analysis, correlation, and linear regression analyses were conducted to assess the agreement between DLM-MC and MG for the EZ area and OS volume measurements. For the EZ area, the overall mean dice score (SD) between DLM-MC and MG was 0.8524 (0.0821), which was comparable to 0.8417 (0.1111) between two MGs. For the EZ area > 1 mm2, the average dice score increased to 0.8799 (0.0614). When comparing DLM-MC to MG, the Bland-Altman plots revealed a mean difference (SE) of 0.0132 (0.0953) mm2 and a coefficient of repeatability (CoR) of 1.8303 mm2 for the EZ area and a mean difference (SE) of 0.0080 (0.0020) mm3 and a CoR of 0.0381 mm3 for the OS volume. The correlation coefficients (95% CI) were 0.9928 (0.9892-0.9952) and 0.9938 (0.9906-0.9958) for the EZ area and OS volume, respectively. The linear regression slopes (95% CI) were 0.9598 (0.9399-0.9797) and 1.0104 (0.9909-1.0298), respectively. The results from this study suggest that the manual correction of deep learning model segmentation can generate EZ area and OS volume measurements in excellent agreement with those of conventional manual grading in RP. Because DLM-MC is more efficient for retinal layer segmentation from OCT scan images, it has the potential to reduce the burden of human graders in obtaining quantitative measurements of biomarkers for assessing disease progression and treatment outcomes in RP.

Optimal Deep Learning Architecture for Automated Segmentation of Cysts in OCT Images Using X-Let Transforms

The retina is a thin, light-sensitive membrane with a multilayered structure found in the back of the eyeball. There are many types of retinal disorders. The two most prevalent retinal illnesses are Age-Related Macular Degeneration (AMD) and Diabetic Macular Edema (DME). Optical Coherence Tomography (OCT) is a vital retinal imaging technology. X-lets (such as curvelet, DTCWT, contourlet, etc.) have several benefits in image processing and analysis. They can capture both local and non-local features of an image simultaneously. The aim of this paper is to propose an optimal deep learning architecture based on sparse basis functions for the automated segmentation of cystic areas in OCT images. Different X-let transforms were used to produce different network inputs, including curvelet, Dual-Tree Complex Wavelet Transform (DTCWT), circlet, and contourlet. Additionally, three different combinations of these transforms are suggested to achieve more accurate segmentation results. Various metrics, including Dice coefficient, sensitivity, false positive ratio, Jaccard index, and qualitative results, were evaluated to find the optimal networks and combinations of the X-let's sub-bands. The proposed network was tested on both original and noisy datasets. The results show the following facts: (1) contourlet achieves the optimal results between different combinations; (2) the five-channel decomposition using high-pass sub-bands of contourlet transform achieves the best performance; and (3) the five-channel decomposition using high-pass sub-bands formations out-performs the state-of-the-art methods, especially in the noisy dataset. The proposed method has the potential to improve the accuracy and speed of the segmentation process in clinical settings, facilitating the diagnosis and treatment of retinal diseases.

Generalist Vision Foundation Models for Medical Imaging: A Case Study of Segment Anything Model on Zero-Shot Medical Segmentation

Medical image analysis plays an important role in clinical diagnosis. In this paper, we examine the recent Segment Anything Model (SAM) on medical images, and report both quantitative and qualitative zero-shot segmentation results on nine medical image segmentation benchmarks, covering various imaging modalities, such as optical coherence tomography (OCT), magnetic resonance imaging (MRI), and computed tomography (CT), as well as different applications including dermatology, ophthalmology, and radiology. Those benchmarks are representative and commonly used in model development. Our experimental results indicate that while SAM presents remarkable segmentation performance on images from the general domain, its zero-shot segmentation ability remains restricted for out-of-distribution images, e.g., medical images. In addition, SAM exhibits inconsistent zero-shot segmentation performance across different unseen medical domains. For certain structured targets, e.g., blood vessels, the zero-shot segmentation of SAM completely failed. In contrast, a simple fine-tuning of it with a small amount of data could lead to remarkable improvement of the segmentation quality, showing the great potential and feasibility of using fine-tuned SAM to achieve accurate medical image segmentation for a precision diagnostics. Our study indicates the versatility of generalist vision foundation models on medical imaging, and their great potential to achieve desired performance through fine-turning and eventually address the challenges associated with accessing large and diverse medical datasets in support of clinical diagnostics.

Deep-learning segmentation of fascicles from microCT of the human vagus nerve

Introduction

MicroCT of the three-dimensional fascicular organization of the human vagus nerve provides essential data to inform basic anatomy as well as the development and optimization of neuromodulation therapies. To process the images into usable formats for subsequent analysis and computational modeling, the fascicles must be segmented. Prior segmentations were completed manually due to the complex nature of the images, including variable contrast between tissue types and staining artifacts.

Methods

Here, we developed a U-Net convolutional neural network (CNN) to automate segmentation of fascicles in microCT of human vagus nerve.

Results

The U-Net segmentation of ~500 images spanning one cervical vagus nerve was completed in 24 s, versus ~40 h for manual segmentation, i.e., nearly four orders of magnitude faster. The automated segmentations had a Dice coefficient of 0.87, a measure of pixel-wise accuracy, thus suggesting a rapid and accurate segmentation. While Dice coefficients are a commonly used metric to assess segmentation performance, we also adapted a metric to assess fascicle-wise detection accuracy, which showed that our network accurately detects the majority of fascicles, but may under-detect smaller fascicles.

Discussion

This network and the associated performance metrics set a benchmark, using a standard U-Net CNN, for the application of deep-learning algorithms to segment fascicles from microCT images. The process may be further optimized by refining tissue staining methods, modifying network architecture, and expanding the ground-truth training data. The resulting three-dimensional segmentations of the human vagus nerve will provide unprecedented accuracy to define nerve morphology in computational models for the analysis and design of neuromodulation therapies.

Robotic Navigation Autonomy for Subretinal Injection via Intelligent Real-Time Virtual iOCT Volume Slicing

In the last decade, various robotic platforms have been introduced that could support delicate retinal surgeries. Concurrently, to provide semantic understanding of the surgical area, recent advances have enabled microscope-integrated intraoperative Optical Coherent Tomography (iOCT) with high-resolution 3D imaging at near video rate. The combination of robotics and semantic understanding enables task autonomy in robotic retinal surgery, such as for subretinal injection. This procedure requires precise needle insertion for best treatment outcomes. However, merging robotic systems with iOCT introduces new challenges. These include, but are not limited to high demands on data processing rates and dynamic registration of these systems during the procedure. In this work, we propose a framework for autonomous robotic navigation for subretinal injection, based on intelligent real-time processing of iOCT volumes. Our method consists of an instrument pose estimation method, an online registration between the robotic and the iOCT system, and trajectory planning tailored for navigation to an injection target. We also introduce intelligent virtual B-scans, a volume slicing approach for rapid instrument pose estimation, which is enabled by Convolutional Neural Networks (CNNs). Our experiments on ex-vivo porcine eyes demonstrate the precision and repeatability of the method. Finally, we discuss identified challenges in this work and suggest potential solutions to further the development of such systems.

Dual-branch collaborative learning network for crop disease identification

Crop diseases seriously affect the quality, yield, and food security of crops. redBesides, traditional manual monitoring methods can no longer meet intelligent agriculture's efficiency and accuracy requirements. Recently, deep learning methods have been rapidly developed in computer vision. To cope with these issues, we propose a dual-branch collaborative learning network for crop disease identification, called DBCLNet. Concretely, we propose a dual-branch collaborative module using convolutional kernels of different scales to extract global and local features of images, which can effectively utilize both global and local features. Meanwhile, we embed a channel attention mechanism in each branch module to refine the global and local features. Whereafter, we cascade multiple dual-branch collaborative modules to design a feature cascade module, which further learns features at more abstract levels via the multi-layer cascade design strategy. Extensive experiments on the Plant Village dataset demonstrated the best classification performance of our DBCLNet method compared to the state-of-the-art methods for the identification of 38 categories of crop diseases. Besides, the Accuracy, Precision, Recall, and F-score of our DBCLNet for the identification of 38 categories of crop diseases are 99.89%, 99.97%, 99.67%, and 99.79%, respectively. 811.

CTS-Net: A Segmentation Network for Glaucoma Optical Coherence Tomography Retinal Layer Images

Optical Coherence Tomography (OCT) technology is essential to obtain glaucoma diagnostic data non-invasively and rapidly. Early diagnosis of glaucoma can be achieved by analyzing the thickness and shape of retinal layers. Accurate retinal layer segmentation assists ophthalmologists in improving the efficiency of disease diagnosis. Deep learning technology is one of the most effective methods for processing OCT retinal layer images, which can segment different retinal layers and effectively obtain the topological structure of the boundary. This paper proposes a neural network method for retinal layer segmentation based on the CSWin Transformer (CTS-Net), which can achieve pixel-level segmentation and obtain smooth boundaries. A Dice loss function based on boundary areas (BADice Loss) is proposed to make CTS-Net learn more features of edge regions and improve the accuracy of boundary segmentation. We applied the model to the publicly available dataset of glaucoma retina, and the test results showed that mean absolute distance (MAD), root mean square error (RMSE), and dice-similarity coefficient (DSC) metrics were 1.79 pixels, 2.15 pixels, and 92.79%, respectively, which are better than those of the compared model. In the cross-validation experiment, the ranges of MAD, RMSE, and DSC are 0.05 pixels, 0.03 pixels, and 0.33%, respectively, with a slight difference, which further verifies the generalization ability of CTS-Net.

Spatial-contextual variational autoencoder with attention correction for anomaly detection in retinal OCT images

Anomaly detection refers to leveraging only normal data to train a model for identifying unseen abnormal cases, which is extensively studied in various fields. Most previous methods are based on reconstruction models, and use anomaly score calculated by the reconstruction error as the metric to tackle anomaly detection. However, these methods just employ single constraint on latent space to construct reconstruction model, resulting in limited performance in anomaly detection. To address this problem, we propose a Spatial-Contextual Variational Autoencoder with Attention Correction for anomaly detection in retinal OCT images. Specifically, we first propose a self-supervised segmentation network to extract retinal regions, which can effectively eliminate interference of background regions. Next, by introducing both multi-dimensional and one-dimensional latent space, our proposed framework can then learn the spatial and contextual manifolds of normal images, which is conducive to enlarging the difference between reconstruction errors of normal images and those of abnormal ones. Furthermore, an ablation-based method is proposed to localize anomalous regions by computing the importance of feature maps, which is used to correct anomaly score calculated by reconstruction error. Finally, a novel anomaly score is constructed to separate the abnormal images from the normal ones. Extensive experiments on two retinal OCT datasets are conducted to evaluate our proposed method, and the experimental results demonstrate the effectiveness of our approach.