1
|
Komninos C, Pissas T, Flores B, Bloch E, Vercauteren T, Ourselin S, Da Cruz L, Bergeles C. Unpaired intra-operative OCT (iOCT) video super-resolution with contrastive learning. BIOMEDICAL OPTICS EXPRESS 2024; 15:772-788. [PMID: 38404298 PMCID: PMC10890864 DOI: 10.1364/boe.501743] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 07/28/2023] [Revised: 08/30/2023] [Accepted: 09/22/2023] [Indexed: 02/27/2024]
Abstract
Regenerative therapies show promise in reversing sight loss caused by degenerative eye diseases. Their precise subretinal delivery can be facilitated by robotic systems alongside with Intra-operative Optical Coherence Tomography (iOCT). However, iOCT's real-time retinal layer information is compromised by inferior image quality. To address this limitation, we introduce an unpaired video super-resolution methodology for iOCT quality enhancement. A recurrent network is proposed to leverage temporal information from iOCT sequences, and spatial information from pre-operatively acquired OCT images. Additionally, a patchwise contrastive loss enables unpaired super-resolution. Extensive quantitative analysis demonstrates that our approach outperforms existing state-of-the-art iOCT super-resolution models. Furthermore, ablation studies showcase the importance of temporal aggregation and contrastive loss in elevating iOCT quality. A qualitative study involving expert clinicians also confirms this improvement. The comprehensive evaluation demonstrates our method's potential to enhance the iOCT image quality, thereby facilitating successful guidance for regenerative therapies.
Collapse
Affiliation(s)
- Charalampos Komninos
- School of Biomedical Engineering & Imaging Sciences, King’s College London, SE1 7EU, London, UK
| | - Theodoros Pissas
- School of Biomedical Engineering & Imaging Sciences, King’s College London, SE1 7EU, London, UK
| | | | | | - Tom Vercauteren
- School of Biomedical Engineering & Imaging Sciences, King’s College London, SE1 7EU, London, UK
| | - Sébastien Ourselin
- School of Biomedical Engineering & Imaging Sciences, King’s College London, SE1 7EU, London, UK
| | - Lyndon Da Cruz
- Moorfields Eye Hospital, EC1V 2PD, London, UK
- Institute of Ophthalmology, University College London, EC1V 9EL, London, UK
| | - Christos Bergeles
- School of Biomedical Engineering & Imaging Sciences, King’s College London, SE1 7EU, London, UK
| |
Collapse
|
2
|
Liao J, Yang S, Zhang T, Li C, Huang Z. A hand-held optical coherence tomography angiography scanner based on angiography reconstruction transformer networks. JOURNAL OF BIOPHOTONICS 2023; 16:e202300100. [PMID: 37264544 DOI: 10.1002/jbio.202300100] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 03/24/2023] [Revised: 05/18/2023] [Accepted: 05/24/2023] [Indexed: 06/03/2023]
Abstract
Optical coherence tomography angiography (OCTA) has successfully demonstrated its viability for clinical applications in dermatology. Due to the high optical scattering property of skin, extracting high-quality OCTA images from skin tissues requires at least six-repeated scans. While the motion artifacts from the patient and the free hand-held probe can lead to a low-quality OCTA image. Our deep-learning-based scan pipeline enables fast and high-quality OCTA imaging with 0.3-s data acquisition. We utilize a fast scanning protocol with a 60 μm/pixel spatial interval rate and introduce angiography-reconstruction-transformer (ART) for 4× super-resolution of low transverse resolution OCTA images. The ART outperforms state-of-the-art networks in OCTA image super-resolution and provides a lighter network size. ART can restore microvessels while reducing the processing time by 85%, and maintaining improvements in structural similarity and peak-signal-to-noise ratio. This study represents that ART can achieve fast and flexible skin OCTA imaging while maintaining image quality.
Collapse
Affiliation(s)
- Jinpeng Liao
- School of Science and Engineering, University of Dundee, Scotland, UK
| | - Shufan Yang
- School of Computing, Engineering and Built Environment, Edinburgh Napier University, Edinburgh, UK
- Research Department of Orthopaedics and Musculoskeletal Science, University College London, UK
| | - Tianyu Zhang
- School of Science and Engineering, University of Dundee, Scotland, UK
| | - Chunhui Li
- School of Science and Engineering, University of Dundee, Scotland, UK
| | - Zhihong Huang
- School of Science and Engineering, University of Dundee, Scotland, UK
| |
Collapse
|
3
|
Fang Y, Shao X, Liu B, Lv H. Optical coherence tomography image despeckling based on tensor singular value decomposition and fractional edge detection. Heliyon 2023; 9:e17735. [PMID: 37449117 PMCID: PMC10336597 DOI: 10.1016/j.heliyon.2023.e17735] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/24/2022] [Revised: 06/26/2023] [Accepted: 06/27/2023] [Indexed: 07/18/2023] Open
Abstract
Optical coherence tomography (OCT) imaging is a technique that is frequently used to diagnose medical conditions. However, coherent noise, sometimes referred to as speckle noise, can dramatically reduce the quality of OCT images, which has an adverse effect on how OCT images are used. In order to enhance the quality of OCT images, a speckle noise reduction technique is developed, and this method is modelled as a low-rank tensor approximation issue. The grouped 3D tensors are first transformed into the transform domain using tensor singular value decomposition (t-SVD). Then, to cut down on speckle noise, transform coefficients are thresholded. Finally, the inverse transform can be used to produce images with speckle suppression. To further enhance the despeckling results, a feature-guided thresholding approach based on fractional edge detection and an adaptive backward projection technique are also presented. Experimental results indicate that the presented algorithm outperforms several comparison methods in relation to speckle suppression, objective metrics, and edge preservation.
Collapse
Affiliation(s)
- Ying Fang
- School of Information Technology, Shangqiu Normal University, Shangqiu, 476000, China
| | - Xia Shao
- School of Information Technology, Shangqiu Normal University, Shangqiu, 476000, China
| | - Bangquan Liu
- College of Digital Technology and Engineering, Ningbo University of Finance and Economics, Ningbo, 315100, China
| | - Hongli Lv
- School of Information Technology, Shangqiu Normal University, Shangqiu, 476000, China
- College of Big Data and Software Engineering, Zhejiang Wanli University, Ningbo, 315100, China
| |
Collapse
|
4
|
Huang S, Wang R, Wu R, Zhong J, Ge X, Liu Y, Ni G. SNR-Net OCT: brighten and denoise low-light optical coherence tomography images via deep learning. OPTICS EXPRESS 2023; 31:20696-20714. [PMID: 37381187 DOI: 10.1364/oe.491391] [Citation(s) in RCA: 2] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Received: 03/23/2023] [Accepted: 05/23/2023] [Indexed: 06/30/2023]
Abstract
Low-light optical coherence tomography (OCT) images generated when using low input power, low-quantum-efficiency detection units, low exposure time, or facing high-reflective surfaces, have low bright and signal-to-noise rates (SNR), and restrict OCT technique and clinical applications. While low input power, low quantum efficiency, and low exposure time can help reduce the hardware requirements and accelerate imaging speed; high-reflective surfaces are unavoidable sometimes. Here we propose a deep-learning-based technique to brighten and denoise low-light OCT images, termed SNR-Net OCT. The proposed SNR-Net OCT deeply integrated a conventional OCT setup and a residual-dense-block U-Net generative adversarial network with channel-wise attention connections trained using a customized large speckle-free SNR-enhanced brighter OCT dataset. Results demonstrated that the proposed SNR-Net OCT can brighten low-light OCT images and remove the speckle noise effectively, with enhancing SNR and maintaining the tissue microstructures well. Moreover, compared to the hardware-based techniques, the proposed SNR-Net OCT can be of lower cost and better performance.
Collapse
|
5
|
Lv H. Speckle attenuation for optical coherence tomography images using the generalized low rank approximations of matrices. OPTICS EXPRESS 2023; 31:11745-11759. [PMID: 37155802 DOI: 10.1364/oe.485097] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 05/10/2023]
Abstract
A frequently used technology in medical diagnosis is optical coherence tomography (OCT). However, coherent noise, also known as speckle noise, has the potential to severely reduce the quality of OCT images, which would be detrimental to the use of OCT images for disease diagnosis. In this paper, a despeckling method is proposed to effectively reduce the speckle noise in OCT images using the generalized low rank approximations of matrices (GLRAM). Specifically, the Manhattan distance (MD)-based block matching method is first used to find nonlocal similar blocks for the reference one. The left and right projection matrices shared by these image blocks are then found using the GLRAM approach, and an adaptive method based on asymptotic matrix reconstruction is proposed to determine how many eigenvectors are present in the left and right projection matrices. Finally, all the reconstructed image blocks are aggregated to create the despeckled OCT image. In addition, an edge-guided adaptive back-projection strategy is used to improve the despeckling performance of the proposed method. Experiments with synthetic and real OCT images show that the presented method performs well in both objective measurements and visual evaluation.
Collapse
|
6
|
A generative adversarial network with multi-scale convolution and dilated convolution res-network for OCT retinal image despeckling. Biomed Signal Process Control 2023. [DOI: 10.1016/j.bspc.2022.104231] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/05/2022]
|
7
|
Chen H, Gao J. Non-Local Mean Denoising Algorithm Based on Fractional Compact Finite Difference Scheme Effectively Reduces Speckle Noise in Optical Coherence Tomography Images. MICROMACHINES 2022; 13:2039. [PMID: 36557339 PMCID: PMC9781262 DOI: 10.3390/mi13122039] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 10/14/2022] [Revised: 11/18/2022] [Accepted: 11/19/2022] [Indexed: 06/17/2023]
Abstract
Optical coherence tomography (OCT) is used in various fields such, as medical diagnosis and material inspection, as a non-invasive and high-resolution optical imaging modality. However, an OCT image is damaged by speckle noise during its generation, thus reducing the image quality. To address this problem, a non-local means (NLM) algorithm based on the fractional compact finite difference scheme (FCFDS) is proposed to remove the speckle noise in OCT images. FCFDS uses more local pixel information when compared to integer-order difference operators. The FCFDS operator is introduced into the NLM algorithm to construct a high-precision weight calculation so that the proposed algorithm can effectively reduce the speckle noise in the OCT images. Experiments on simulations and real OCT images show that the proposed method is comparable to other state-of-the-art despeckling methods and can substantially reduce noise and preserve image details such as edges and structures. Speckle noise removal can further promote the application of the proposed algorithm in medical diagnosis and industrial detection, as it has key research value.
Collapse
Affiliation(s)
- Huaiguang Chen
- School of Science, Shandong Jianzhu University, Jinan 250101, China
- Center for Engineering Computation and Software Development, Shandong Jianzhu University, Jinan 250101, China
| | - Jing Gao
- School of Science, Shandong Jianzhu University, Jinan 250101, China
| |
Collapse
|
8
|
Ni G, Wu R, Zhong J, Chen Y, Wan L, Xie Y, Mei J, Liu Y. Hybrid-structure network and network comparative study for deep-learning-based speckle-modulating optical coherence tomography. OPTICS EXPRESS 2022; 30:18919-18938. [PMID: 36221682 DOI: 10.1364/oe.454504] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 01/23/2022] [Accepted: 04/26/2022] [Indexed: 06/16/2023]
Abstract
Optical coherence tomography (OCT), a promising noninvasive bioimaging technique, can resolve sample three-dimensional microstructures. However, speckle noise imposes obvious limitations on OCT resolving capabilities. Here we proposed a deep-learning-based speckle-modulating OCT based on a hybrid-structure network, residual-dense-block U-Net generative adversarial network (RDBU-Net GAN), and further conducted a comprehensively comparative study to explore multi-type deep-learning architectures' abilities to extract speckle pattern characteristics and remove speckle, and resolve microstructures. This is the first time that network comparative study has been performed on a customized dataset containing mass more-general speckle patterns obtained from a custom-built speckle-modulating OCT, but not on retinal OCT datasets with limited speckle patterns. Results demonstrated that the proposed RDBU-Net GAN has a more excellent ability to extract speckle pattern characteristics and remove speckle, and resolve microstructures. This work will be useful for future studies on OCT speckle removing and deep-learning-based speckle-modulating OCT.
Collapse
|