1
|
Ni G, Wu R, Zheng F, Li M, Huang S, Ge X, Liu L, Liu Y. Toward Ground-Truth Optical Coherence Tomography via Three-Dimensional Unsupervised Deep Learning Processing and Data. IEEE TRANSACTIONS ON MEDICAL IMAGING 2024; 43:2395-2407. [PMID: 38324426 DOI: 10.1109/tmi.2024.3363416] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 02/09/2024]
Abstract
Optical coherence tomography (OCT) can perform non-invasive high-resolution three-dimensional (3D) imaging and has been widely used in biomedical fields, while it is inevitably affected by coherence speckle noise which degrades OCT imaging performance and restricts its applications. Here we present a novel speckle-free OCT imaging strategy, named toward-ground-truth OCT ( t GT-OCT), that utilizes unsupervised 3D deep-learning processing and leverages OCT 3D imaging features to achieve speckle-free OCT imaging. Specifically, our proposed t GT-OCT utilizes an unsupervised 3D-convolution deep-learning network trained using random 3D volumetric data to distinguish and separate speckle from real structures in 3D imaging volumetric space; moreover, t GT-OCT effectively further reduces speckle noise and reveals structures that would otherwise be obscured by speckle noise while preserving spatial resolution. Results derived from different samples demonstrated the high-quality speckle-free 3D imaging performance of t GT-OCT and its advancement beyond the previous state-of-the-art. The code is available online: https://github.com/Voluntino/tGT-OCT.
Collapse
|
2
|
Ahmed H, Zhang Q, Donnan R, Alomainy A. Transformer enhanced autoencoder rendering cleaning of noisy optical coherence tomography images. J Med Imaging (Bellingham) 2024; 11:034008. [PMID: 38694626 PMCID: PMC11058346 DOI: 10.1117/1.jmi.11.3.034008] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/11/2023] [Revised: 04/09/2024] [Accepted: 04/10/2024] [Indexed: 05/04/2024] Open
Abstract
Purpose Optical coherence tomography (OCT) is an emerging imaging tool in healthcare with common applications in ophthalmology for detection of retinal diseases, as well as other medical domains. The noise in OCT images presents a great challenge as it hinders the clinician's ability to diagnosis in extensive detail. Approach In this work, a region-based, deep-learning, denoising framework is proposed for adaptive cleaning of noisy OCT-acquired images. The core of the framework is a hybrid deep-learning model named transformer enhanced autoencoder rendering (TEAR). Attention gates are utilized to ensure focus on denoising the foreground and to remove the background. TEAR is designed to remove the different types of noise artifacts commonly present in OCT images and to enhance the visual quality. Results Extensive quantitative evaluations are performed to evaluate the performance of TEAR and compare it against both deep-learning and traditional state-of-the-art denoising algorithms. The proposed method improved the peak signal-to-noise ratio to 27.9 dB, CNR to 6.3 dB, SSIM to 0.9, and equivalent number of looks to 120.8 dB for a dental dataset. For a retinal dataset, the performance metrics in the same sequence are: 24.6, 14.2, 0.64, and 1038.7 dB, respectively. Conclusions The results show that the approach verifiably removes speckle noise and achieves superior quality over several well-known denoisers.
Collapse
Affiliation(s)
- Hanya Ahmed
- Queen Mary University of London, School of Electronic Engineering and Computer Science, London, United Kingdom
| | - Qianni Zhang
- Queen Mary University of London, School of Electronic Engineering and Computer Science, London, United Kingdom
| | - Robert Donnan
- Queen Mary University of London, School of Engineering and Materials Science, London, United Kingdom
| | - Akram Alomainy
- Queen Mary University of London, School of Electronic Engineering and Computer Science, London, United Kingdom
| |
Collapse
|
3
|
Ahmed H, Zhang Q, Donnan R, Alomainy A. Denoising of Optical Coherence Tomography Images in Ophthalmology Using Deep Learning: A Systematic Review. J Imaging 2024; 10:86. [PMID: 38667984 PMCID: PMC11050869 DOI: 10.3390/jimaging10040086] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/27/2024] [Revised: 03/22/2024] [Accepted: 03/24/2024] [Indexed: 04/28/2024] Open
Abstract
Imaging from optical coherence tomography (OCT) is widely used for detecting retinal diseases, localization of intra-retinal boundaries, etc. It is, however, degraded by speckle noise. Deep learning models can aid with denoising, allowing clinicians to clearly diagnose retinal diseases. Deep learning models can be considered as an end-to-end framework. We selected denoising studies that used deep learning models with retinal OCT imagery. Each study was quality-assessed through image quality metrics (including the peak signal-to-noise ratio-PSNR, contrast-to-noise ratio-CNR, and structural similarity index metric-SSIM). Meta-analysis could not be performed due to heterogeneity in the methods of the studies and measurements of their performance. Multiple databases (including Medline via PubMed, Google Scholar, Scopus, Embase) and a repository (ArXiv) were screened for publications published after 2010, without any limitation on language. From the 95 potential studies identified, a total of 41 were evaluated thoroughly. Fifty-four of these studies were excluded after full text assessment depending on whether deep learning (DL) was utilized or the dataset and results were not effectively explained. Numerous types of OCT images are mentioned in this review consisting of public retinal image datasets utilized purposefully for denoising OCT images (n = 37) and the Optic Nerve Head (ONH) (n = 4). A wide range of image quality metrics was used; PSNR and SNR that ranged between 8 and 156 dB. The minority of studies (n = 8) showed a low risk of bias in all domains. Studies utilizing ONH images produced either a PSNR or SNR value varying from 8.1 to 25.7 dB, and that of public retinal datasets was 26.4 to 158.6 dB. Further analysis on denoising models was not possible due to discrepancies in reporting that did not allow useful pooling. An increasing number of studies have investigated denoising retinal OCT images using deep learning, with a range of architectures being implemented. The reported increase in image quality metrics seems promising, while study and reporting quality are currently low.
Collapse
Affiliation(s)
- Hanya Ahmed
- Department of Electronic Engineering and Computer Science, Queen Mary University of London, London E1 4NS, UK
| | - Qianni Zhang
- Department of Electronic Engineering and Computer Science, Queen Mary University of London, London E1 4NS, UK
| | - Robert Donnan
- Department of Engineering and Materials Science, Queen Mary University of London, London E1 4NS, UK
| | - Akram Alomainy
- Department of Electronic Engineering and Computer Science, Queen Mary University of London, London E1 4NS, UK
| |
Collapse
|
4
|
Wu R, Huang S, Zhong J, Zheng F, Li M, Ge X, Zhong J, Liu L, Ni G, Liu Y. Unsupervised OCT image despeckling with ground-truth- and repeated-scanning-free features. OPTICS EXPRESS 2024; 32:11934-11951. [PMID: 38571030 DOI: 10.1364/oe.510696] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 10/30/2023] [Accepted: 03/05/2024] [Indexed: 04/05/2024]
Abstract
Optical coherence tomography (OCT) can resolve biological three-dimensional tissue structures, but it is inevitably plagued by speckle noise that degrades image quality and obscures biological structure. Recently unsupervised deep learning methods are becoming more popular in OCT despeckling but they still have to use unpaired noisy-clean images or paired noisy-noisy images. To address the above problem, we propose what we believe to be a novel unsupervised deep learning method for OCT despeckling, termed Double-free Net, which eliminates the need for ground truth data and repeated scanning by sub-sampling noisy images and synthesizing noisier images. In comparison to existing unsupervised methods, Double-free Net obtains superior denoising performance when trained on datasets comprising retinal and human tissue images without clean images. The efficacy of Double-free Net in denoising holds significant promise for diagnostic applications in retinal pathologies and enhances the accuracy of retinal layer segmentation. Results demonstrate that Double-free Net outperforms state-of-the-art methods and exhibits strong convenience and adaptability across different OCT images.
Collapse
|
5
|
Jovanov L, Philips W. Denoising in optical coherence tomography volumes for improved 3D visualization. OPTICS EXPRESS 2024; 32:10302-10316. [PMID: 38571246 DOI: 10.1364/oe.478957] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Received: 10/31/2023] [Accepted: 01/14/2024] [Indexed: 04/05/2024]
Abstract
Optical coherence tomography (OCT) has already become one of the most important diagnostic tools in different fields of medicine, as well as in various industrial applications. The most important characteristic of OCT is its high resolution, both in depth and the transverse direction. Together with the information on the tissue density, OCT offers highly precise information on tissue geometry. However, the detectability of small and low-intensity features in OCT scans is limited by the presence of speckle noise. In this paper we present a new volumetric method for noise removal in OCT volumes, which aims at improving the quality of rendered 3D volumes. In order to remove noise uniformly, while preserving important details, the proposed algorithm simultaneously observes the estimated amounts of noise and the sharpness measure, and iteratively enhances the volume until it reaches the required quality. We evaluate the proposed method using four quality measures as well as visually, by evaluating the visualization of OCT volumes on an auto-stereoscopic 3D screen. The results show that the proposed method outperforms reference methods both visually and in terms of objective measures.
Collapse
|
6
|
Graas A, Coban SB, Batenburg KJ, Lucka F. Just-in-time deep learning for real-time X-ray computed tomography. Sci Rep 2023; 13:20070. [PMID: 37973801 PMCID: PMC10654383 DOI: 10.1038/s41598-023-46028-9] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/15/2023] [Accepted: 10/26/2023] [Indexed: 11/19/2023] Open
Abstract
Real-time X-ray tomography pipelines, such as implemented by RECAST3D, compute and visualize tomographic reconstructions in milliseconds, and enable the observation of dynamic experiments in synchrotron beamlines and laboratory scanners. For extending real-time reconstruction by image processing and analysis components, Deep Neural Networks (DNNs) are a promising technology, due to their strong performance and much faster run-times compared to conventional algorithms. DNNs may prevent experiment repetition by simplifying real-time steering and optimization of the ongoing experiment. The main challenge of integrating DNNs into real-time tomography pipelines, however, is that they need to learn their task from representative data before the start of the experiment. In scientific environments, such training data may not exist, and other uncertain and variable factors, such as the set-up configuration, reconstruction parameters, or user interaction, cannot easily be anticipated beforehand, either. To overcome these problems, we developed just-in-time learning, an online DNN training strategy that takes advantage of the spatio-temporal continuity of consecutive reconstructions in the tomographic pipeline. This allows training and deploying comparatively small DNNs during the experiment. We provide software implementations, and study the feasibility and challenges of the approach by training the self-supervised Noise2Inverse denoising task with X-ray data replayed from real-world dynamic experiments.
Collapse
Affiliation(s)
- Adriaan Graas
- Computational Imaging, CWI, 1098 XG, Amsterdam, The Netherlands.
| | - Sophia Bethany Coban
- Computational Imaging, CWI, 1098 XG, Amsterdam, The Netherlands
- Frazer-Nash Consultancy, Leatherhead, KT22 7NL, Surrey, UK
| | - K Joost Batenburg
- Computational Imaging, CWI, 1098 XG, Amsterdam, The Netherlands
- Leiden Institute of Advanced Computer Science, Leiden University, Niels Bohrweg 1, 2333 CA, Leiden, The Netherlands
| | - Felix Lucka
- Computational Imaging, CWI, 1098 XG, Amsterdam, The Netherlands.
| |
Collapse
|
7
|
Song Z, Lin Y, Xiong L, Li Z. Super-resolution algorithm for the characterization of sweat glands in fingerprint OCT images. JOURNAL OF THE OPTICAL SOCIETY OF AMERICA. A, OPTICS, IMAGE SCIENCE, AND VISION 2023; 40:2068-2077. [PMID: 38038073 DOI: 10.1364/josaa.503212] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 08/22/2023] [Accepted: 10/07/2023] [Indexed: 12/02/2023]
Abstract
Optical coherence tomography (OCT) is a noninvasive optical imaging technique that can be used to produce three-dimensional images of fingerprints. However, the low quality and poor resolution of the regions of interest (ROIs) in OCT images make it challenging to segment small tissues accurately. To address this issue, a super-resolution (SR) network called ESRNet has been developed to enhance the quality of OCT images, facilitating their applications in research. Firstly, the performance of the SR images produced by ESRNet is evaluated by comparing it to those generated by five other SR methods. Specifically, the SR performance is evaluated using three upscale factors (2×, 3×, and 4×) to assess the quality of the enhanced images. Based on the results obtained from the three datasets, it is evident that ESRNet outperforms current advanced networks in terms of SR performance. Furthermore, the segmentation accuracy of sweat glands has been significantly improved by the SR images. The number of sweat glands in the top view increased from 102 to 117, further substantiating the performance of the ESRNet network. The spiral structure of sweat glands is clear to human eyes and has been verified by showing similar left-right-handed spiral numbers. Finally, a sweat gland recognition method for the SR 3D images is proposed.
Collapse
|
8
|
Wu X, Gao W, Bian H. Self-denoising method for OCT images with single spectrogram-based deep learning. OPTICS LETTERS 2023; 48:4945-4948. [PMID: 37773356 DOI: 10.1364/ol.499966] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Received: 07/07/2023] [Accepted: 08/29/2023] [Indexed: 10/01/2023]
Abstract
The presence of noise in images reconstructed with optical coherence tomography (OCT) is a key issue which limits the further improvement of the image quality. In this Letter, for the first time, to the best of our knowledge, a self-denoising method for OCT images is presented with single spectrogram-based deep learning. Different noises in different images could be customized with an extremely low computation. The deep-learning model consists of two fully connected layers, two convolution layers, and one deconvolution layer, with the input being the raw interference spectrogram and the label being its reconstructed image using the Fourier transform. The denoising image could be calculated by subtracting the noise predicted by our model from the label image. The OCT images of the TiO2 phantom, the orange, and the zebrafish obtained with our spectral-domain OCT system are used as examples to demonstrate the capability of our method. The results demonstrate its effectiveness in reducing noises such as speckle patterns and horizontal and vertical stripes. Compared with the label image, the signal-to-noise ratio could be improved by 35.0 dB, and the image contrast could be improved by a factor of two. Compared with the results denoised by the average method, the mean peak signal-to-noise ratio is 26.2 dB.
Collapse
|
9
|
Wu R, Huang S, Zhong J, Li M, Zheng F, Bo E, Liu L, Liu Y, Ge X, Ni G. MAS-Net OCT: a deep-learning-based speckle-free multiple aperture synthetic optical coherence tomography. BIOMEDICAL OPTICS EXPRESS 2023; 14:2591-2607. [PMID: 37342716 PMCID: PMC10278634 DOI: 10.1364/boe.483740] [Citation(s) in RCA: 2] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Received: 12/19/2022] [Revised: 02/26/2023] [Accepted: 04/28/2023] [Indexed: 06/23/2023]
Abstract
High-resolution spectral domain optical coherence tomography (SD-OCT) is a vital clinical technique that suffers from the inherent compromise between transverse resolution and depth of focus (DOF). Meanwhile, speckle noise worsens OCT imaging resolving power and restricts potential resolution-enhancement techniques. Multiple aperture synthetic (MAS) OCT transmits light signals and records sample echoes along a synthetic aperture to extend DOF, acquired by time-encoding or optical path length encoding. In this work, a deep-learning-based multiple aperture synthetic OCT termed MAS-Net OCT, which integrated a speckle-free model based on self-supervised learning, was proposed. MAS-Net was trained on datasets generated by the MAS OCT system. Here we performed experiments on homemade microparticle samples and various biological tissues. Results demonstrated that the proposed MAS-Net OCT could effectively improve the transverse resolution in a large imaging depth as well as reduced most speckle noise.
Collapse
Affiliation(s)
- Renxiong Wu
- School of Optoelectronic Science and Engineering, University of Electronic Science and Technology of China, Chengdu 610054, China
| | - Shaoyan Huang
- School of Optoelectronic Science and Engineering, University of Electronic Science and Technology of China, Chengdu 610054, China
| | - Junming Zhong
- School of Optoelectronic Science and Engineering, University of Electronic Science and Technology of China, Chengdu 610054, China
| | - Meixuan Li
- School of Optoelectronic Science and Engineering, University of Electronic Science and Technology of China, Chengdu 610054, China
| | - Fei Zheng
- School of Optoelectronic Science and Engineering, University of Electronic Science and Technology of China, Chengdu 610054, China
| | - En Bo
- School of Electrical and Electronic Engineering, Nanyang Technological University, Singapore 639798, Singapore
| | - Linbo Liu
- School of Electrical and Electronic Engineering, Nanyang Technological University, Singapore 639798, Singapore
| | - Yong Liu
- School of Optoelectronic Science and Engineering, University of Electronic Science and Technology of China, Chengdu 610054, China
| | - Xin Ge
- School of Science, Shenzhen Campus of Sun Yat-sen University, Shenzhen 510275, China
| | - Guangming Ni
- School of Optoelectronic Science and Engineering, University of Electronic Science and Technology of China, Chengdu 610054, China
| |
Collapse
|
10
|
Nienhaus J, Matten P, Britten A, Scherer J, Höck E, Freytag A, Drexler W, Leitgeb RA, Schlegl T, Schmoll T. Live 4D-OCT denoising with self-supervised deep learning. Sci Rep 2023; 13:5760. [PMID: 37031338 PMCID: PMC10082772 DOI: 10.1038/s41598-023-32695-1] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/16/2022] [Accepted: 03/31/2023] [Indexed: 04/10/2023] Open
Abstract
By providing three-dimensional visualization of tissues and instruments at high resolution, live volumetric optical coherence tomography (4D-OCT) has the potential to revolutionize ophthalmic surgery. However, the necessary imaging speed is accompanied by increased noise levels. A high data rate and the requirement for minimal latency impose major limitations for real-time noise reduction. In this work, we propose a low complexity neural network for denoising, directly incorporated into the image reconstruction pipeline of a microscope-integrated 4D-OCT prototype with an A-scan rate of 1.2 MHz. For this purpose, we trained a blind-spot network on unpaired OCT images using a self-supervised learning approach. With an optimized U-Net, only a few milliseconds of additional latency were introduced. Simultaneously, these architectural adaptations improved the numerical denoising performance compared to the basic setup, outperforming non-local filtering algorithms. Layers and edges of anatomical structures in B-scans were better preserved than with Gaussian filtering despite comparable processing time. By comparing scenes with and without denoising employed, we show that neural networks can be used to improve visual appearance of volumetric renderings in real time. Enhancing the rendering quality is an important step for the clinical acceptance and translation of 4D-OCT as an intra-surgical guidance tool.
Collapse
Affiliation(s)
- Jonas Nienhaus
- Center for Medical Physics and Biomedical Engineering, Medical University of Vienna, Vienna, Austria.
| | - Philipp Matten
- Center for Medical Physics and Biomedical Engineering, Medical University of Vienna, Vienna, Austria
| | - Anja Britten
- Center for Medical Physics and Biomedical Engineering, Medical University of Vienna, Vienna, Austria
| | - Julius Scherer
- Center for Medical Physics and Biomedical Engineering, Medical University of Vienna, Vienna, Austria
| | | | | | - Wolfgang Drexler
- Center for Medical Physics and Biomedical Engineering, Medical University of Vienna, Vienna, Austria
| | - Rainer A Leitgeb
- Center for Medical Physics and Biomedical Engineering, Medical University of Vienna, Vienna, Austria
| | - Thomas Schlegl
- Center for Medical Physics and Biomedical Engineering, Medical University of Vienna, Vienna, Austria
| | - Tilman Schmoll
- Center for Medical Physics and Biomedical Engineering, Medical University of Vienna, Vienna, Austria
- Carl Zeiss Meditec, Inc., Dublin, USA
| |
Collapse
|
11
|
Karn PK, Abdulla WH. On Machine Learning in Clinical Interpretation of Retinal Diseases Using OCT Images. Bioengineering (Basel) 2023; 10:bioengineering10040407. [PMID: 37106594 PMCID: PMC10135895 DOI: 10.3390/bioengineering10040407] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/17/2023] [Revised: 03/21/2023] [Accepted: 03/22/2023] [Indexed: 03/29/2023] Open
Abstract
Optical coherence tomography (OCT) is a noninvasive imaging technique that provides high-resolution cross-sectional retina images, enabling ophthalmologists to gather crucial information for diagnosing various retinal diseases. Despite its benefits, manual analysis of OCT images is time-consuming and heavily dependent on the personal experience of the analyst. This paper focuses on using machine learning to analyse OCT images in the clinical interpretation of retinal diseases. The complexity of understanding the biomarkers present in OCT images has been a challenge for many researchers, particularly those from nonclinical disciplines. This paper aims to provide an overview of the current state-of-the-art OCT image processing techniques, including image denoising and layer segmentation. It also highlights the potential of machine learning algorithms to automate the analysis of OCT images, reducing time consumption and improving diagnostic accuracy. Using machine learning in OCT image analysis can mitigate the limitations of manual analysis methods and provide a more reliable and objective approach to diagnosing retinal diseases. This paper will be of interest to ophthalmologists, researchers, and data scientists working in the field of retinal disease diagnosis and machine learning. By presenting the latest advancements in OCT image analysis using machine learning, this paper will contribute to the ongoing efforts to improve the diagnostic accuracy of retinal diseases.
Collapse
|
12
|
Applegate MB, Kose K, Ghimire S, Rajadhyaksha M, Dy J. Self-supervised denoising of Nyquist-sampled volumetric images via deep learning. J Med Imaging (Bellingham) 2023; 10:024005. [PMID: 36992871 PMCID: PMC10042483 DOI: 10.1117/1.jmi.10.2.024005] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/27/2022] [Accepted: 03/06/2023] [Indexed: 03/29/2023] Open
Abstract
Purpose Deep learning has demonstrated excellent performance enhancing noisy or degraded biomedical images. However, many of these models require access to a noise-free version of the images to provide supervision during training, which limits their utility. Here, we develop an algorithm (noise2Nyquist) that leverages the fact that Nyquist sampling provides guarantees about the maximum difference between adjacent slices in a volumetric image, which allows denoising to be performed without access to clean images. We aim to show that our method is more broadly applicable and more effective than other self-supervised denoising algorithms on real biomedical images, and provides comparable performance to algorithms that need clean images during training. Approach We first provide a theoretical analysis of noise2Nyquist and an upper bound for denoising error based on sampling rate. We go on to demonstrate its effectiveness in denoising in a simulated example as well as real fluorescence confocal microscopy, computed tomography, and optical coherence tomography images. Results We find that our method has better denoising performance than existing self-supervised methods and is applicable to datasets where clean versions are not available. Our method resulted in peak signal to noise ratio (PSNR) within 1 dB and structural similarity (SSIM) index within 0.02 of supervised methods. On medical images, it outperforms existing self-supervised methods by an average of 3 dB in PSNR and 0.1 in SSIM. Conclusion noise2Nyquist can be used to denoise any volumetric dataset sampled at at least the Nyquist rate making it useful for a wide variety of existing datasets.
Collapse
Affiliation(s)
- Matthew B. Applegate
- Northeastern University, Department of Electrical and Computer Engineering, Boston, Massachusetts, United States
| | - Kivanc Kose
- Dermatology Service at Memorial Sloan Kettering Cancer Center, New York, United States
| | - Sandesh Ghimire
- Northeastern University, Department of Electrical and Computer Engineering, Boston, Massachusetts, United States
| | - Milind Rajadhyaksha
- Dermatology Service at Memorial Sloan Kettering Cancer Center, New York, United States
| | - Jennifer Dy
- Northeastern University, Department of Electrical and Computer Engineering, Boston, Massachusetts, United States
- Address all correspondence to Jennifer Dy,
| |
Collapse
|
13
|
Zhou Q, Wen M, Yu B, Lou C, Ding M, Zhang X. Self-supervised transformer based non-local means despeckling of optical coherence tomography images. Biomed Signal Process Control 2023. [DOI: 10.1016/j.bspc.2022.104348] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/11/2022]
|
14
|
Li Y, Fan Y, Liao H. Self-supervised speckle noise reduction of optical coherence tomography without clean data. BIOMEDICAL OPTICS EXPRESS 2022; 13:6357-6372. [PMID: 36589594 PMCID: PMC9774848 DOI: 10.1364/boe.471497] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Received: 07/25/2022] [Revised: 10/12/2022] [Accepted: 10/27/2022] [Indexed: 06/17/2023]
Abstract
Optical coherence tomography (OCT) is widely used in clinical diagnosis due to its non-invasive, real-time, and high-resolution characteristics. However, the inherent speckle noise seriously degrades the image quality, which might damage the fine structures in OCT, thus affecting the diagnosis results. In recent years, supervised deep learning-based denoising methods have shown excellent denoising ability. To train a deep denoiser, a large number of paired noisy-clean images are required, which is difficult to achieve in clinical practice, since acquiring a speckle-free OCT image requires dozens of repeated scans and image registration. In this research, we propose a self-supervised strategy that helps build a despeckling model by training it to map neighboring pixels in a single noisy OCT image. Adjacent pixel patches are randomly selected from the original OCT image to generate two similar undersampled images, which are respectively used as the input and target images for training a deep neural network. To ensure both the despeckling and the structure-preserving effects, a multi-scale pixel patch sampler and corresponding loss functions are adopted in our practice. Through quantitative evaluation and qualitative visual comparison, we found that the proposed method performs better than state-of-the-art methods regarding despeckling effects and structure preservation. Besides, the proposed method is much easier to train and deploy without the need for clean OCT images, which has great significance in clinical practice.
Collapse
Affiliation(s)
- Yangxi Li
- Department of Biomedical Engineering, School of Medicine, Tsinghua University, Beijing, 100084, China
| | - Yingwei Fan
- Institute of Engineering Medicine, Beijing Institute of Technology, Beijing, 100081, China
| | - Hongen Liao
- Department of Biomedical Engineering, School of Medicine, Tsinghua University, Beijing, 100084, China
| |
Collapse
|
15
|
Bayhaqi YA, Hamidi A, Canbaz F, Navarini AA, Cattin PC, Zam A. Deep-Learning-Based Fast Optical Coherence Tomography (OCT) Image Denoising for Smart Laser Osteotomy. IEEE TRANSACTIONS ON MEDICAL IMAGING 2022; 41:2615-2628. [PMID: 35442883 DOI: 10.1109/tmi.2022.3168793] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/14/2023]
Abstract
Laser osteotomy promises precise cutting and minor bone tissue damage. We proposed Optical Coherence Tomography (OCT) to monitor the ablation process toward our smart laser osteotomy approach. The OCT image is helpful to identify tissue type and provide feedback for the ablation laser to avoid critical tissues such as bone marrow and nerve. Furthermore, in the implementation, the tissue classifier's accuracy is dependent on the quality of the OCT image. Therefore, image denoising plays an important role in having an accurate feedback system. A common OCT image denoising technique is the frame-averaging method. Inherent to this method is the need for multiple images, i.e., the more images used, the better the resulting image quality. However, this approach comes at the price of increased acquisition time and sensitivity to motion artifacts. To overcome these limitations, we applied a deep-learning denoising method capable of imitating the frame-averaging method. The resulting image had a similar image quality to the frame-averaging and was better than the classical digital filtering methods. We also evaluated if this method affects the tissue classifier model's accuracy that will provide feedback to the ablation laser. We found that image denoising significantly increased the accuracy of the tissue classifier. Furthermore, we observed that the classifier trained using the deep learning denoised images achieved similar accuracy to the classifier trained using frame-averaged images. The results suggest the possibility of using the deep learning method as a pre-processing step for real-time tissue classification in smart laser osteotomy.
Collapse
|
16
|
Varadarajan D, Magnain C, Fogarty M, Boas DA, Fischl B, Wang H. A novel algorithm for multiplicative speckle noise reduction in ex vivo human brain OCT images. Neuroimage 2022; 257:119304. [PMID: 35568350 PMCID: PMC10018743 DOI: 10.1016/j.neuroimage.2022.119304] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/11/2022] [Revised: 05/06/2022] [Accepted: 05/10/2022] [Indexed: 10/18/2022] Open
Abstract
Optical coherence tomography (OCT) images of ex vivo human brain tissue are corrupted by multiplicative speckle noise that degrades the contrast to noise ratio (CNR) of microstructural compartments. This work proposes a novel algorithm to reduce noise corruption in OCT images that minimizes the penalized negative log likelihood of gamma distributed speckle noise. The proposed method is formulated as a majorize-minimize problem that reduces to solving an iterative regularized least squares optimization. We demonstrate the usefulness of the proposed method by removing speckle in simulated data, phantom data and real OCT images of human brain tissue. We compare the proposed method with state of the art filtering and non-local means based denoising methods. We demonstrate that our approach removes speckle accurately, improves CNR between different tissue types and better preserves small features and edges in human brain tissue.
Collapse
Affiliation(s)
- Divya Varadarajan
- Athinoula A. Martinos Center for Biomedical Imaging, Massachusetts General Hospital, Charlestown, MA 02129, USA; Radiology, Harvard Medical School, Boston, MA 02115, USA.
| | - Caroline Magnain
- Athinoula A. Martinos Center for Biomedical Imaging, Massachusetts General Hospital, Charlestown, MA 02129, USA; Radiology, Harvard Medical School, Boston, MA 02115, USA
| | - Morgan Fogarty
- Imaging Science Program, Washington University McKelvey School of Engineering, St. Louis, MO 63130, USA; Radiology, Washington University School of Medicine, St. Louis, MO 63110, USA
| | - David A Boas
- Biomedical Engineering and Electrical and Computer Engineering, Boston University, Boston, MA 02215, USA
| | - Bruce Fischl
- Athinoula A. Martinos Center for Biomedical Imaging, Massachusetts General Hospital, Charlestown, MA 02129, USA; Radiology, Harvard Medical School, Boston, MA 02115, USA; Harvard-MIT Health Science and Technology, Massachusetts Institute of Technology, Cambridge, MA 02139, USA
| | - Hui Wang
- Athinoula A. Martinos Center for Biomedical Imaging, Massachusetts General Hospital, Charlestown, MA 02129, USA; Radiology, Harvard Medical School, Boston, MA 02115, USA
| |
Collapse
|
17
|
Zhou Q, Wen M, Ding M, Zhang X. Unsupervised despeckling of optical coherence tomography images by combining cross-scale CNN with an intra-patch and inter-patch based transformer. OPTICS EXPRESS 2022; 30:18800-18820. [PMID: 36221673 DOI: 10.1364/oe.459477] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 03/29/2022] [Accepted: 05/03/2022] [Indexed: 06/16/2023]
Abstract
Optical coherence tomography (OCT) has found wide application to the diagnosis of ophthalmic diseases, but the quality of OCT images is degraded by speckle noise. The convolutional neural network (CNN) based methods have attracted much attention in OCT image despeckling. However, these methods generally need noisy-clean image pairs for training and they are difficult to capture the global context information effectively. To address these issues, we have proposed a novel unsupervised despeckling method. This method uses the cross-scale CNN to extract the local features and uses the intra-patch and inter-patch based transformer to extract and merge the local and global feature information. Based on these extracted features, a reconstruction network is used to produce the final denoised result. The proposed network is trained using a hybrid unsupervised loss function, which is defined by the loss produced from Nerighbor2Neighbor, the structural similarity between the despeckled results of the probabilistic non-local means method and our method as well as the mean squared error between their features extracted by the VGG network. Experiments on two clinical OCT image datasets show that our method performs better than several popular despeckling algorithms in terms of visual evaluation and quantitative indexes.
Collapse
|
18
|
Ni G, Wu R, Zhong J, Chen Y, Wan L, Xie Y, Mei J, Liu Y. Hybrid-structure network and network comparative study for deep-learning-based speckle-modulating optical coherence tomography. OPTICS EXPRESS 2022; 30:18919-18938. [PMID: 36221682 DOI: 10.1364/oe.454504] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 01/23/2022] [Accepted: 04/26/2022] [Indexed: 06/16/2023]
Abstract
Optical coherence tomography (OCT), a promising noninvasive bioimaging technique, can resolve sample three-dimensional microstructures. However, speckle noise imposes obvious limitations on OCT resolving capabilities. Here we proposed a deep-learning-based speckle-modulating OCT based on a hybrid-structure network, residual-dense-block U-Net generative adversarial network (RDBU-Net GAN), and further conducted a comprehensively comparative study to explore multi-type deep-learning architectures' abilities to extract speckle pattern characteristics and remove speckle, and resolve microstructures. This is the first time that network comparative study has been performed on a customized dataset containing mass more-general speckle patterns obtained from a custom-built speckle-modulating OCT, but not on retinal OCT datasets with limited speckle patterns. Results demonstrated that the proposed RDBU-Net GAN has a more excellent ability to extract speckle pattern characteristics and remove speckle, and resolve microstructures. This work will be useful for future studies on OCT speckle removing and deep-learning-based speckle-modulating OCT.
Collapse
|
19
|
Rico-Jimenez JJ, Hu D, Tang EM, Oguz I, Tao YK. Real-time OCT image denoising using a self-fusion neural network. BIOMEDICAL OPTICS EXPRESS 2022; 13:1398-1409. [PMID: 35415003 PMCID: PMC8973187 DOI: 10.1364/boe.451029] [Citation(s) in RCA: 6] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 12/13/2021] [Revised: 01/20/2022] [Accepted: 02/06/2022] [Indexed: 06/07/2023]
Abstract
Optical coherence tomography (OCT) has become the gold standard for ophthalmic diagnostic imaging. However, clinical OCT image-quality is highly variable and limited visualization can introduce errors in the quantitative analysis of anatomic and pathologic features-of-interest. Frame-averaging is a standard method for improving image-quality, however, frame-averaging in the presence of bulk-motion can degrade lateral resolution and prolongs total acquisition time. We recently introduced a method called self-fusion, which reduces speckle noise and enhances OCT signal-to-noise ratio (SNR) by using similarity between from adjacent frames and is more robust to motion-artifacts than frame-averaging. However, since self-fusion is based on deformable registration, it is computationally expensive. In this study a convolutional neural network was implemented to offset the computational overhead of self-fusion and perform OCT denoising in real-time. The self-fusion network was pretrained to fuse 3 frames to achieve near video-rate frame-rates. Our results showed a clear gain in peak SNR in the self-fused images over both the raw and frame-averaged OCT B-scans. This approach delivers a fast and robust OCT denoising alternative to frame-averaging without the need for repeated image acquisition. Real-time self-fusion image enhancement will enable improved localization of OCT field-of-view relative to features-of-interest and improved sensitivity for anatomic features of disease.
Collapse
Affiliation(s)
- Jose J. Rico-Jimenez
- Department of Biomedical Engineering, Vanderbilt University, Nashville, TN 37232, USA
| | - Dewei Hu
- Department of Electrical Engineering and Computer Science, Vanderbilt University, Nashville, TN 37235 USA, USA
| | - Eric M. Tang
- Department of Biomedical Engineering, Vanderbilt University, Nashville, TN 37232, USA
| | - Ipek Oguz
- Department of Electrical Engineering and Computer Science, Vanderbilt University, Nashville, TN 37235 USA, USA
| | - Yuankai K. Tao
- Department of Biomedical Engineering, Vanderbilt University, Nashville, TN 37232, USA
| |
Collapse
|
20
|
Smitha A, Febin I, Jidesh P. A retinex based non-local total generalized variation framework for OCT image restoration. Biomed Signal Process Control 2022. [DOI: 10.1016/j.bspc.2021.103234] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/02/2022]
|
21
|
Yu TT, Ma D, Lo J, Ju MJ, Beg MF, Sarunic MV. Effect of optical coherence tomography and angiography sampling rate towards diabetic retinopathy severity classification. BIOMEDICAL OPTICS EXPRESS 2021; 12:6660-6673. [PMID: 34745763 PMCID: PMC8547994 DOI: 10.1364/boe.431992] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Received: 06/08/2021] [Revised: 08/12/2021] [Accepted: 08/16/2021] [Indexed: 06/13/2023]
Abstract
Optical coherence tomography (OCT) and OCT angiography (OCT-A) may benefit the screening of diabetic retinopathy (DR). This study investigated the effect of laterally subsampling OCT/OCT-A en face scans by up to a factor of 8 when using deep neural networks for automated referable DR classification. There was no significant difference in the classification performance across all evaluation metrics when subsampling up to a factor of 3, and only minimal differences up to a factor of 8. Our findings suggest that OCT/OCT-A can reduce the number of samples (and hence the acquisition time) for a volume for a given field of view on the retina that is acquired for rDR classification.
Collapse
Affiliation(s)
- Timothy T. Yu
- Engineering Science, Simon Fraser University, Burnaby BC V5A1S6, Canada
| | - Da Ma
- Engineering Science, Simon Fraser University, Burnaby BC V5A1S6, Canada
| | - Julian Lo
- Engineering Science, Simon Fraser University, Burnaby BC V5A1S6, Canada
| | - Myeong Jin Ju
- Dept. of Ophthalmology and Visual Sciences, University of British Columbia, Vancouver, BC, V5Z 3N9, Canada
- School of Biomedical Engineering, University of British Columbia, Vancouver, BC, V5Z 3N9, Canada
| | - Mirza Faisal Beg
- Engineering Science, Simon Fraser University, Burnaby BC V5A1S6, Canada
| | | |
Collapse
|