N2NSR-OCT: Simultaneous denoising and super-resolution in optical coherence tomography images using semisupervised deep learning

Noise-imitation learning: unpaired speckle noise reduction for optical coherence tomography

Objective.Optical coherence tomography (OCT) is widely used in clinical practice for its non-invasive, high-resolution imaging capabilities. However, speckle noise inherent to its low coherence principle can degrade image quality and compromise diagnostic accuracy. While deep learning methods have shown promise in reducing speckle noise, obtaining well-registered image pairs remains challenging, leading to the development of unpaired methods. Despite their potential, existing unpaired methods suffer from redundancy in network structures or interaction mechanisms. Therefore, a more streamlined method for unpaired OCT denoising is essential.Approach.In this work, we propose a novel unpaired method for OCT image denoising, referred to as noise-imitation learning (NIL). NIL comprises three primary modules: the noise extraction module, which extracts noise features by denoising noisy images; the noise imitation module, which synthesizes noisy images and generates fake clean images; and the adversarial learning module, which differentiates between real and fake clean images through adversarial training. The complexity of NIL is significantly lower than that of previous unpaired methods, utilizing only one generator and one discriminator for training.Main results.By efficiently fusing unpaired images and employing adversarial training, NIL can extract more speckle noise information to enhance denoising performance. Building on NIL, we propose an OCT image denoising pipeline, NIL-NAFNet. This pipeline achieved PSNR, SSIM, and RMSE values of 31.27 dB, 0.865, and 7.00, respectively, on the PKU37 dataset. Extensive experiments suggest that our method outperforms state-of-the-art unpaired methods both qualitatively and quantitatively.Significance.These findings indicate that the proposed NIL is a simple yet effective method for unpaired OCT speckle noise reduction. The OCT denoising pipeline based on NIL demonstrates exceptional performance and efficiency. By addressing speckle noise without requiring well-registered image pairs, this method can enhance image quality and diagnostic accuracy in clinical practice.

High-resolution in vivo 4D-OCT fish-eye imaging using 3D-UNet with multi-level residue decoder

Optical coherence tomography (OCT) allows high-resolution volumetric imaging of biological tissues in vivo. However, 3D-image acquisition often suffers from motion artifacts due to slow frame rates and involuntary and physiological movements of living tissue. To solve these issues, we implement a real-time 4D-OCT system capable of reconstructing near-distortion-free volumetric images based on a deep learning-based reconstruction algorithm. The system initially collects undersampled volumetric images at a high speed and then upsamples the images in real-time by a convolutional neural network (CNN) that generates high-frequency features using a deep learning algorithm. We compare and analyze both dual-2D- and 3D-UNet-based networks for the OCT 3D high-resolution image reconstruction. We refine the network architecture by incorporating multi-level information to accelerate convergence and improve accuracy. The network is optimized by utilizing the 16-bit floating-point precision for network parameters to conserve GPU memory and enhance efficiency. The result shows that the refined and optimized 3D-network is capable of retrieving the tissue structure more precisely and enable real-time 4D-OCT imaging at a rate greater than 10 Hz with a root mean square error (RMSE) of ∼0.03.

Toward Ground-Truth Optical Coherence Tomography via Three-Dimensional Unsupervised Deep Learning Processing and Data

Optical coherence tomography (OCT) can perform non-invasive high-resolution three-dimensional (3D) imaging and has been widely used in biomedical fields, while it is inevitably affected by coherence speckle noise which degrades OCT imaging performance and restricts its applications. Here we present a novel speckle-free OCT imaging strategy, named toward-ground-truth OCT ( t GT-OCT), that utilizes unsupervised 3D deep-learning processing and leverages OCT 3D imaging features to achieve speckle-free OCT imaging. Specifically, our proposed t GT-OCT utilizes an unsupervised 3D-convolution deep-learning network trained using random 3D volumetric data to distinguish and separate speckle from real structures in 3D imaging volumetric space; moreover, t GT-OCT effectively further reduces speckle noise and reveals structures that would otherwise be obscured by speckle noise while preserving spatial resolution. Results derived from different samples demonstrated the high-quality speckle-free 3D imaging performance of t GT-OCT and its advancement beyond the previous state-of-the-art. The code is available online: https://github.com/Voluntino/tGT-OCT.

PSCAT: a lightweight transformer for simultaneous denoising and super-resolution of OCT images

Optical coherence tomography (OCT), owing to its non-invasive nature, has demonstrated tremendous potential in clinical practice and has become a prevalent diagnostic method. Nevertheless, the inherent speckle noise and low sampling rate in OCT imaging often limit the quality of OCT images. In this paper, we propose a lightweight Transformer to efficiently reconstruct high-quality images from noisy and low-resolution OCT images acquired by short scans. Our method, PSCAT, parallelly employs spatial window self-attention and channel attention in the Transformer block to aggregate features from both spatial and channel dimensions. It explores the potential of the Transformer in denoising and super-resolution for OCT, reducing computational costs and enhancing the speed of image processing. To effectively assist in restoring high-frequency details, we introduce a hybrid loss function in both spatial and frequency domains. Extensive experiments demonstrate that our PSCAT has fewer network parameters and lower computational costs compared to state-of-the-art methods while delivering a competitive performance both qualitatively and quantitatively.

Denoising of Optical Coherence Tomography Images in Ophthalmology Using Deep Learning: A Systematic Review

Imaging from optical coherence tomography (OCT) is widely used for detecting retinal diseases, localization of intra-retinal boundaries, etc. It is, however, degraded by speckle noise. Deep learning models can aid with denoising, allowing clinicians to clearly diagnose retinal diseases. Deep learning models can be considered as an end-to-end framework. We selected denoising studies that used deep learning models with retinal OCT imagery. Each study was quality-assessed through image quality metrics (including the peak signal-to-noise ratio-PSNR, contrast-to-noise ratio-CNR, and structural similarity index metric-SSIM). Meta-analysis could not be performed due to heterogeneity in the methods of the studies and measurements of their performance. Multiple databases (including Medline via PubMed, Google Scholar, Scopus, Embase) and a repository (ArXiv) were screened for publications published after 2010, without any limitation on language. From the 95 potential studies identified, a total of 41 were evaluated thoroughly. Fifty-four of these studies were excluded after full text assessment depending on whether deep learning (DL) was utilized or the dataset and results were not effectively explained. Numerous types of OCT images are mentioned in this review consisting of public retinal image datasets utilized purposefully for denoising OCT images (n = 37) and the Optic Nerve Head (ONH) (n = 4). A wide range of image quality metrics was used; PSNR and SNR that ranged between 8 and 156 dB. The minority of studies (n = 8) showed a low risk of bias in all domains. Studies utilizing ONH images produced either a PSNR or SNR value varying from 8.1 to 25.7 dB, and that of public retinal datasets was 26.4 to 158.6 dB. Further analysis on denoising models was not possible due to discrepancies in reporting that did not allow useful pooling. An increasing number of studies have investigated denoising retinal OCT images using deep learning, with a range of architectures being implemented. The reported increase in image quality metrics seems promising, while study and reporting quality are currently low.

Unsupervised OCT image despeckling with ground-truth- and repeated-scanning-free features

Optical coherence tomography (OCT) can resolve biological three-dimensional tissue structures, but it is inevitably plagued by speckle noise that degrades image quality and obscures biological structure. Recently unsupervised deep learning methods are becoming more popular in OCT despeckling but they still have to use unpaired noisy-clean images or paired noisy-noisy images. To address the above problem, we propose what we believe to be a novel unsupervised deep learning method for OCT despeckling, termed Double-free Net, which eliminates the need for ground truth data and repeated scanning by sub-sampling noisy images and synthesizing noisier images. In comparison to existing unsupervised methods, Double-free Net obtains superior denoising performance when trained on datasets comprising retinal and human tissue images without clean images. The efficacy of Double-free Net in denoising holds significant promise for diagnostic applications in retinal pathologies and enhances the accuracy of retinal layer segmentation. Results demonstrate that Double-free Net outperforms state-of-the-art methods and exhibits strong convenience and adaptability across different OCT images.

Axial super-resolution optical coherence tomography via complex-valued network

Optical coherence tomography (OCT) is a fast and non-invasive optical interferometric imaging technique that can provide high-resolution cross-sectional images of biological tissues. OCT's key strength is its depth resolving capability which remains invariant along the imaging depth and is determined by the axial resolution. The axial resolution is inversely proportional to the bandwidth of the OCT light source. Thus, the use of broadband light sources can effectively improve the axial resolution and however leads to an increased cost. In recent years, real-valued deep learning technique has been introduced to obtain super-resolution optical imaging. In this study, we proposed a complex-valued super-resolution network (CVSR-Net) to achieve an axial super-resolution for OCT by fully utilizing the amplitude and phase of OCT signal. The method was evaluated on three OCT datasets. The results show that the CVSR-Net outperforms its real-valued counterpart with a better depth resolving capability. Furthermore, comparisons were made between our network, six prevailing real-valued networks and their complex-valued counterparts. The results demonstrate that the complex-valued network exhibited a better super-resolution performance than its real-valued counterpart and our proposed CVSR-Net achieved the best performance. In addition, the CVSR-Net was tested on out-of-distribution domain datasets and its super-resolution performance was well maintained as compared to that on source domain datasets, indicating a good generalization capability.

Deep learning in optical coherence tomography: Where are the gaps?

Optical coherence tomography (OCT) is a non-invasive optical imaging modality, which provides rapid, high-resolution and cross-sectional morphology of macular area and optic nerve head for diagnosis and managing of different eye diseases. However, interpreting OCT images requires experts in both OCT images and eye diseases since many factors such as artefacts and concomitant diseases can affect the accuracy of quantitative measurements made by post-processing algorithms. Currently, there is a growing interest in applying deep learning (DL) methods to analyse OCT images automatically. This review summarises the trends in DL-based OCT image analysis in ophthalmology, discusses the current gaps, and provides potential research directions. DL in OCT analysis shows promising performance in several tasks: (1) layers and features segmentation and quantification; (2) disease classification; (3) disease progression and prognosis; and (4) referral triage level prediction. Different studies and trends in the development of DL-based OCT image analysis are described and the following challenges are identified and described: (1) public OCT data are scarce and scattered; (2) models show performance discrepancies in real-world settings; (3) models lack of transparency; (4) there is a lack of societal acceptance and regulatory standards; and (5) OCT is still not widely available in underprivileged areas. More work is needed to tackle the challenges and gaps, before DL is further applied in OCT image analysis for clinical use.

Frequency-aware optical coherence tomography image super-resolution via conditional generative adversarial neural network

Optical coherence tomography (OCT) has stimulated a wide range of medical image-based diagnosis and treatment in fields such as cardiology and ophthalmology. Such applications can be further facilitated by deep learning-based super-resolution technology, which improves the capability of resolving morphological structures. However, existing deep learning-based method only focuses on spatial distribution and disregards frequency fidelity in image reconstruction, leading to a frequency bias. To overcome this limitation, we propose a frequency-aware super-resolution framework that integrates three critical frequency-based modules (i.e., frequency transformation, frequency skip connection, and frequency alignment) and frequency-based loss function into a conditional generative adversarial network (cGAN). We conducted a large-scale quantitative study from an existing coronary OCT dataset to demonstrate the superiority of our proposed framework over existing deep learning frameworks. In addition, we confirmed the generalizability of our framework by applying it to fish corneal images and rat retinal images, demonstrating its capability to super-resolve morphological details in eye imaging.

A hand-held optical coherence tomography angiography scanner based on angiography reconstruction transformer networks

Optical coherence tomography angiography (OCTA) has successfully demonstrated its viability for clinical applications in dermatology. Due to the high optical scattering property of skin, extracting high-quality OCTA images from skin tissues requires at least six-repeated scans. While the motion artifacts from the patient and the free hand-held probe can lead to a low-quality OCTA image. Our deep-learning-based scan pipeline enables fast and high-quality OCTA imaging with 0.3-s data acquisition. We utilize a fast scanning protocol with a 60 μm/pixel spatial interval rate and introduce angiography-reconstruction-transformer (ART) for 4× super-resolution of low transverse resolution OCTA images. The ART outperforms state-of-the-art networks in OCTA image super-resolution and provides a lighter network size. ART can restore microvessels while reducing the processing time by 85%, and maintaining improvements in structural similarity and peak-signal-to-noise ratio. This study represents that ART can achieve fast and flexible skin OCTA imaging while maintaining image quality.

MAS-Net OCT: a deep-learning-based speckle-free multiple aperture synthetic optical coherence tomography

High-resolution spectral domain optical coherence tomography (SD-OCT) is a vital clinical technique that suffers from the inherent compromise between transverse resolution and depth of focus (DOF). Meanwhile, speckle noise worsens OCT imaging resolving power and restricts potential resolution-enhancement techniques. Multiple aperture synthetic (MAS) OCT transmits light signals and records sample echoes along a synthetic aperture to extend DOF, acquired by time-encoding or optical path length encoding. In this work, a deep-learning-based multiple aperture synthetic OCT termed MAS-Net OCT, which integrated a speckle-free model based on self-supervised learning, was proposed. MAS-Net was trained on datasets generated by the MAS OCT system. Here we performed experiments on homemade microparticle samples and various biological tissues. Results demonstrated that the proposed MAS-Net OCT could effectively improve the transverse resolution in a large imaging depth as well as reduced most speckle noise.

Live 4D-OCT denoising with self-supervised deep learning

By providing three-dimensional visualization of tissues and instruments at high resolution, live volumetric optical coherence tomography (4D-OCT) has the potential to revolutionize ophthalmic surgery. However, the necessary imaging speed is accompanied by increased noise levels. A high data rate and the requirement for minimal latency impose major limitations for real-time noise reduction. In this work, we propose a low complexity neural network for denoising, directly incorporated into the image reconstruction pipeline of a microscope-integrated 4D-OCT prototype with an A-scan rate of 1.2 MHz. For this purpose, we trained a blind-spot network on unpaired OCT images using a self-supervised learning approach. With an optimized U-Net, only a few milliseconds of additional latency were introduced. Simultaneously, these architectural adaptations improved the numerical denoising performance compared to the basic setup, outperforming non-local filtering algorithms. Layers and edges of anatomical structures in B-scans were better preserved than with Gaussian filtering despite comparable processing time. By comparing scenes with and without denoising employed, we show that neural networks can be used to improve visual appearance of volumetric renderings in real time. Enhancing the rendering quality is an important step for the clinical acceptance and translation of 4D-OCT as an intra-surgical guidance tool.

On Machine Learning in Clinical Interpretation of Retinal Diseases Using OCT Images

Optical coherence tomography (OCT) is a noninvasive imaging technique that provides high-resolution cross-sectional retina images, enabling ophthalmologists to gather crucial information for diagnosing various retinal diseases. Despite its benefits, manual analysis of OCT images is time-consuming and heavily dependent on the personal experience of the analyst. This paper focuses on using machine learning to analyse OCT images in the clinical interpretation of retinal diseases. The complexity of understanding the biomarkers present in OCT images has been a challenge for many researchers, particularly those from nonclinical disciplines. This paper aims to provide an overview of the current state-of-the-art OCT image processing techniques, including image denoising and layer segmentation. It also highlights the potential of machine learning algorithms to automate the analysis of OCT images, reducing time consumption and improving diagnostic accuracy. Using machine learning in OCT image analysis can mitigate the limitations of manual analysis methods and provide a more reliable and objective approach to diagnosing retinal diseases. This paper will be of interest to ophthalmologists, researchers, and data scientists working in the field of retinal disease diagnosis and machine learning. By presenting the latest advancements in OCT image analysis using machine learning, this paper will contribute to the ongoing efforts to improve the diagnostic accuracy of retinal diseases.

Mixed multiscale BM4D for three-dimensional optical coherence tomography denoising

A multiscale extension for the well-known block matching and 4D filtering (BM4D) method is proposed by analyzing and extending the wavelet subbands denoising method in such a way that the proposed method avoids directly denoising detail subbands, which considerably simplifies the computations and makes the multiscale processing feasible in 3D. To this end, we first derive the multiscale construction method in 2D and propose multiscale extensions for three 2D natural image denoising methods. Then, the derivation is extended to 3D by proposing mixed multiscale BM4D (mmBM4D) for optical coherence tomography (OCT) image denoising. We tested mmBM4D on three public OCT datasets captured by various imaging devices. The experiments revealed that mmBM4D significantly outperforms its original counterpart and performs on par with the state-of-the-art OCT denoising methods. In terms of peak-signal-to-noise-ratio (PSNR), mmBM4D surpasses the original BM4D by more than 0.68 decibels over the first dataset. In the second and third datasets, significant improvements in the mean to standard deviation ratio, contrast to noise ratio, and equivalent number of looks were achieved. Furthermore, on the downstream task of retinal layer segmentation, the layer quality preservation of the compared OCT denoising methods is evaluated.

Multi-Scale Reconstruction of Undersampled Spectral-Spatial OCT Data for Coronary Imaging Using Deep Learning

Coronary artery disease (CAD) is a cardiovascular condition with high morbidity and mortality. Intravascular optical coherence tomography (IVOCT) has been considered as an optimal imagining system for the diagnosis and treatment of CAD. Constrained by Nyquist theorem, dense sampling in IVOCT attains high resolving power to delineate cellular structures/features. There is a trade-off between high spatial resolution and fast scanning rate for coronary imaging. In this paper, we propose a viable spectral-spatial acquisition method that down-scales the sampling process in both spectral and spatial domain while maintaining high quality in image reconstruction. The down-scaling schedule boosts data acquisition speed without any hardware modifications. Additionally, we propose a unified multi-scale reconstruction framework, namely Multiscale-Spectral-Spatial-Magnification Network (MSSMN), to resolve highly down-scaled (compressed) OCT images with flexible magnification factors. We incorporate the proposed methods into Spectral Domain OCT (SD-OCT) imaging of human coronary samples with clinical features such as stent and calcified lesions. Our experimental results demonstrate that spectral-spatial down-scaled data can be better reconstructed than data that are down-scaled solely in either spectral or spatial domain. Moreover, we observe better reconstruction performance using MSSMN than using existing reconstruction methods. Our acquisition method and multi-scale reconstruction framework, in combination, may allow faster SD-OCT inspection with high resolution during coronary intervention.

Speckle denoising based on deep learning via a conditional generative adversarial network in digital holographic interferometry

Speckle denoising can improve digital holographic interferometry phase measurements but may affect experimental accuracy. A deep-learning-based speckle denoising algorithm is developed using a conditional generative adversarial network. Two subnetworks, namely discriminator and generator networks, which refer to the U-Net and DenseNet layer structures are used to supervise network learning quality and denoising. Datasets obtained from speckle simulations are shown to provide improved noise feature extraction. The loss function is designed by considering the peak signal-to-noise ratio parameters to improve efficiency and accuracy. The proposed method thus shows better performance than other denoising algorithms for processing experimental strain data from digital holography.

Lateral image reconstruction of optical coherence tomography using one-dimensional deep deconvolution network

BACKGROUND AND OBJECTIVES

Optical coherence tomography (OCT) is a cross-sectional imaging method utilizing a low coherence interferometry. The lateral resolution of the OCT is limited by the numerical aperture (NA) of the imaging lens. Using a high NA lens improves the lateral resolution but reduces the depth of focus (DOF). In this study, we propose a method to improve the lateral resolution of OCT images by end-to-end training of a deep 1-D deconvolution network without use of high-resolution images.

MATERIALS AND METHODS

To improve the lateral resolution of the OCT, we trained the 1-D deconvolution network using lateral profiles of OCT images and the beam spot size. We used our image-guided laparoscopic surgical tool (IGLaST) to acquire OCT images of nonbiological and biological samples ex vivo. The OCT images were then blurred by applying Gaussian functions with various full width half maximums ranging from 40 to 160 µm. The network was trained using the blurred OCT images as input and the non-blurred original OCT images as output. We quantitatively evaluated the developed network in terms of similarity and signal-to-ratio (SNR), using in-vivo images of mesenteric tissue from a porcine model that was not used for training. In addition, we performed knife-edge tests and qualitative evaluation of the network to show the lateral resolution improvement of ex-vivo and in-vivo OCT images.

RESULTS

The proposed method showed an improvement of image quality on both blurred images and non-blurred images. When the proposed deconvolution network was applied, the similarity to the non-blurred image was improved by 1.29 times, and the SNR was improved by 1.76 dB compared to the artificially blurred images, which was superior to the conventional deconvolution method. The knife-edge tests at distances at 200 to 1000 µm from the imaging probe showed an approximately 1.2 times improvement in lateral resolution. In addition, through qualitative evaluation, it was found that the image quality of both ex-vivo and in-vivo tissue images was improved with clear structure and less noise.

CONCLUSIONS

This study showed the ability of the 1-D deconvolution network to improve the image quality of OCT images with variable lateral resolution. We were able to train the network with a small amount of data by constraining the network in 1-D. The quantitative evaluation showed better results than conventional deconvolution methods for various amount of blurring. Qualitative evaluation showed analogous results with quantitative results. This simple yet powerful image restoration method provides improved lateral resolution and suppresses background noise, making it applicable to a variety of OCT imaging applications.

Comparative study of deep neural networks with unsupervised Noise2Noise strategy for noise reduction of optical coherence tomography images

As a powerful diagnostic tool, optical coherence tomography (OCT) has been widely used in various clinical setting. However, OCT images are susceptible to inherent speckle noise that may contaminate subtle structure information, due to low-coherence interferometric imaging procedure. Many supervised learning-based models have achieved impressive performance in reducing speckle noise of OCT images trained with a large number of noisy-clean paired OCT images, which are not commonly feasible in clinical practice. In this article, we conducted a comparative study to investigate the denoising performance of OCT images over different deep neural networks through an unsupervised Noise2Noise (N2N) strategy, which only trained with noisy OCT samples. Four representative network architectures including U-shaped model, multi-information stream model, straight-information stream model and GAN-based model were investigated on an OCT image dataset acquired from healthy human eyes. The results demonstrated all four unsupervised N2N models offered denoised OCT images with a performance comparable with that of supervised learning models, illustrating the effectiveness of unsupervised N2N models in denoising OCT images. Furthermore, U-shaped models and GAN-based models using UNet network as generator are two preferred and suitable architectures for reducing speckle noise of OCT images and preserving fine structure information of retinal layers under unsupervised N2N circumstances.

Resolution-enhanced OCT and expanded framework of information capacity and resolution in coherent imaging

Spatial resolution in conventional optical microscopy has traditionally been treated as a fixed parameter of the optical system. Here, we present an approach to enhance transverse resolution in beam-scanned optical coherence tomography (OCT) beyond its aberration-free resolution limit, without any modification to the optical system. Based on the theorem of invariance of information capacity, resolution-enhanced (RE)-OCT navigates the exchange of information between resolution and signal-to-noise ratio (SNR) by exploiting efficient noise suppression via coherent averaging and a simple computational bandwidth expansion procedure. We demonstrate a resolution enhancement of 1.5 × relative to the aberration-free limit while maintaining comparable SNR in silicone phantom. We show that RE-OCT can significantly enhance the visualization of fine microstructural features in collagen gel and ex vivo mouse brain. Beyond RE-OCT, our analysis in the spatial-frequency domain leads to an expanded framework of information capacity and resolution in coherent imaging that contributes new implications to the theory of coherent imaging. RE-OCT can be readily implemented on most OCT systems worldwide, immediately unlocking information that is beyond their current imaging capabilities, and so has the potential for widespread impact in the numerous areas in which OCT is utilized, including the basic sciences and translational medicine.

Imaging depth adaptive resolution enhancement for optical coherence tomography via deep neural network with external attention

Optical coherence tomography (OCT) is a promising non-invasive imaging technique that owns many biomedical applications. In this paper, a deep neural network is proposed for enhancing the spatial resolution of OCTen faceimages. Different from the previous reports, the proposed can recover high-resolutionen faceimages from low-resolutionen faceimages at arbitrary imaging depth. This kind of imaging depth adaptive resolution enhancement is achieved through an external attention mechanism, which takes advantage of morphological similarity between the arbitrary-depth and full-depthen faceimages. Firstly, the deep feature maps are extracted by a feature extraction network from the arbitrary-depth and full-depthen faceimages. Secondly, the morphological similarity between the deep feature maps is extracted and utilized to emphasize the features strongly correlated to the vessel structures by using the external attention network. Finally, the SR image is recovered from the enhanced feature map through an up-sampling network. The proposed network is tested on a clinical skin OCT data set and an open-access retinal OCT dataset. The results show that the proposed external attention mechanism can suppress invalid features and enhance significant features in our tasks. For all tests, the proposed SR network outperformed the traditional image interpolation method, e.g. bi-cubic method, and the state-of-the-art image super-resolution networks, e.g. enhanced deep super-resolution network, residual channel attention network, and second-order attention network. The proposed method may increase the quantitative clinical assessment of micro-vascular diseases which is limited by OCT imaging device resolution.