1
|
Das V, Zhang F, Bower AJ, Li J, Liu T, Aguilera N, Alvisio B, Liu Z, Hammer DX, Tam J. Revealing speckle obscured living human retinal cells with artificial intelligence assisted adaptive optics optical coherence tomography. COMMUNICATIONS MEDICINE 2024; 4:68. [PMID: 38600290 PMCID: PMC11006674 DOI: 10.1038/s43856-024-00483-1] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/18/2023] [Accepted: 03/13/2024] [Indexed: 04/12/2024] Open
Abstract
BACKGROUND In vivo imaging of the human retina using adaptive optics optical coherence tomography (AO-OCT) has transformed medical imaging by enabling visualization of 3D retinal structures at cellular-scale resolution, including the retinal pigment epithelial (RPE) cells, which are essential for maintaining visual function. However, because noise inherent to the imaging process (e.g., speckle) makes it difficult to visualize RPE cells from a single volume acquisition, a large number of 3D volumes are typically averaged to improve contrast, substantially increasing the acquisition duration and reducing the overall imaging throughput. METHODS Here, we introduce parallel discriminator generative adversarial network (P-GAN), an artificial intelligence (AI) method designed to recover speckle-obscured cellular features from a single AO-OCT volume, circumventing the need for acquiring a large number of volumes for averaging. The combination of two parallel discriminators in P-GAN provides additional feedback to the generator to more faithfully recover both local and global cellular structures. Imaging data from 8 eyes of 7 participants were used in this study. RESULTS We show that P-GAN not only improves RPE cell contrast by 3.5-fold, but also improves the end-to-end time required to visualize RPE cells by 99-fold, thereby enabling large-scale imaging of cells in the living human eye. RPE cell spacing measured across a large set of AI recovered images from 3 participants were in agreement with expected normative ranges. CONCLUSIONS The results demonstrate the potential of AI assisted imaging in overcoming a key limitation of RPE imaging and making it more accessible in a routine clinical setting.
Collapse
Affiliation(s)
- Vineeta Das
- National Eye Institute, National Institutes of Health, Bethesda, MD, 20892, USA
| | - Furu Zhang
- National Eye Institute, National Institutes of Health, Bethesda, MD, 20892, USA
| | - Andrew J Bower
- National Eye Institute, National Institutes of Health, Bethesda, MD, 20892, USA
| | - Joanne Li
- National Eye Institute, National Institutes of Health, Bethesda, MD, 20892, USA
| | - Tao Liu
- National Eye Institute, National Institutes of Health, Bethesda, MD, 20892, USA
| | - Nancy Aguilera
- National Eye Institute, National Institutes of Health, Bethesda, MD, 20892, USA
| | - Bruno Alvisio
- National Eye Institute, National Institutes of Health, Bethesda, MD, 20892, USA
| | - Zhuolin Liu
- Center for Devices and Radiological Health, U.S. Food and Drug Administration, Silver Spring, MD, 20993, USA
| | - Daniel X Hammer
- Center for Devices and Radiological Health, U.S. Food and Drug Administration, Silver Spring, MD, 20993, USA
| | - Johnny Tam
- National Eye Institute, National Institutes of Health, Bethesda, MD, 20892, USA.
| |
Collapse
|
2
|
Li M, Huang K, Xu Q, Yang J, Zhang Y, Ji Z, Xie K, Yuan S, Liu Q, Chen Q. OCTA-500: A retinal dataset for optical coherence tomography angiography study. Med Image Anal 2024; 93:103092. [PMID: 38325155 DOI: 10.1016/j.media.2024.103092] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/23/2022] [Revised: 11/10/2023] [Accepted: 01/22/2024] [Indexed: 02/09/2024]
Abstract
Optical coherence tomography angiography (OCTA) is a novel imaging modality that has been widely utilized in ophthalmology and neuroscience studies to observe retinal vessels and microvascular systems. However, publicly available OCTA datasets remain scarce. In this paper, we introduce the largest and most comprehensive OCTA dataset dubbed OCTA-500, which contains OCTA imaging under two fields of view (FOVs) from 500 subjects. The dataset provides rich images and annotations including two modalities (OCT/OCTA volumes), six types of projections, four types of text labels (age/gender/eye/disease) and seven types of segmentation labels (large vessel/capillary/artery/vein/2D FAZ/3D FAZ/retinal layers). Then, we propose a multi-object segmentation task called CAVF, which integrates capillary segmentation, artery segmentation, vein segmentation, and FAZ segmentation under a unified framework. In addition, we optimize the 3D-to-2D image projection network (IPN) to IPN-V2 to serve as one of the segmentation baselines. Experimental results demonstrate that IPN-V2 achieves an about 10% mIoU improvement over IPN on CAVF task. Finally, we further study the impact of several dataset characteristics: the training set size, the model input (OCT/OCTA, 3D volume/2D projection), the baseline networks, and the diseases. The dataset and code are publicly available at: https://ieee-dataport.org/open-access/octa-500.
Collapse
Affiliation(s)
- Mingchao Li
- School of Computer Science and Engineering, Nanjing University of Science and Technology, NanJing 210094, China.
| | - Kun Huang
- School of Computer Science and Engineering, Nanjing University of Science and Technology, NanJing 210094, China.
| | - Qiuzhuo Xu
- School of Computer Science and Engineering, Nanjing University of Science and Technology, NanJing 210094, China.
| | - Jiadong Yang
- School of Computer Science and Engineering, Nanjing University of Science and Technology, NanJing 210094, China.
| | - Yuhan Zhang
- School of Computer Science and Engineering, Nanjing University of Science and Technology, NanJing 210094, China.
| | - Zexuan Ji
- School of Computer Science and Engineering, Nanjing University of Science and Technology, NanJing 210094, China.
| | - Keren Xie
- Department of Ophthalmology, The First Affiliated Hospital with Nanjing Medical University, NanJing 210029, China.
| | - Songtao Yuan
- Department of Ophthalmology, The First Affiliated Hospital with Nanjing Medical University, NanJing 210029, China.
| | - Qinghuai Liu
- Department of Ophthalmology, The First Affiliated Hospital with Nanjing Medical University, NanJing 210029, China.
| | - Qiang Chen
- School of Computer Science and Engineering, Nanjing University of Science and Technology, NanJing 210094, China.
| |
Collapse
|
3
|
Salimi M, Roshanfar M, Tabatabaei N, Mosadegh B. Machine Learning-Assisted Short-Wave InfraRed (SWIR) Techniques for Biomedical Applications: Towards Personalized Medicine. J Pers Med 2023; 14:33. [PMID: 38248734 PMCID: PMC10817559 DOI: 10.3390/jpm14010033] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/24/2023] [Revised: 12/08/2023] [Accepted: 12/20/2023] [Indexed: 01/23/2024] Open
Abstract
Personalized medicine transforms healthcare by adapting interventions to individuals' unique genetic, molecular, and clinical profiles. To maximize diagnostic and/or therapeutic efficacy, personalized medicine requires advanced imaging devices and sensors for accurate assessment and monitoring of individual patient conditions or responses to therapeutics. In the field of biomedical optics, short-wave infrared (SWIR) techniques offer an array of capabilities that hold promise to significantly enhance diagnostics, imaging, and therapeutic interventions. SWIR techniques provide in vivo information, which was previously inaccessible, by making use of its capacity to penetrate biological tissues with reduced attenuation and enable researchers and clinicians to delve deeper into anatomical structures, physiological processes, and molecular interactions. Combining SWIR techniques with machine learning (ML), which is a powerful tool for analyzing information, holds the potential to provide unprecedented accuracy for disease detection, precision in treatment guidance, and correlations of complex biological features, opening the way for the data-driven personalized medicine field. Despite numerous biomedical demonstrations that utilize cutting-edge SWIR techniques, the clinical potential of this approach has remained significantly underexplored. This paper demonstrates how the synergy between SWIR imaging and ML is reshaping biomedical research and clinical applications. As the paper showcases the growing significance of SWIR imaging techniques that are empowered by ML, it calls for continued collaboration between researchers, engineers, and clinicians to boost the translation of this technology into clinics, ultimately bridging the gap between cutting-edge technology and its potential for personalized medicine.
Collapse
Affiliation(s)
| | - Majid Roshanfar
- Department of Mechanical Engineering, Concordia University, Montreal, QC H3G 1M8, Canada;
| | - Nima Tabatabaei
- Department of Mechanical Engineering, York University, Toronto, ON M3J 1P3, Canada;
| | - Bobak Mosadegh
- Dalio Institute of Cardiovascular Imaging, Department of Radiology, Weill Cornell Medicine, New York, NY 10021, USA
| |
Collapse
|
4
|
Waheed NK, Rosen RB, Jia Y, Munk MR, Huang D, Fawzi A, Chong V, Nguyen QD, Sepah Y, Pearce E. Optical coherence tomography angiography in diabetic retinopathy. Prog Retin Eye Res 2023; 97:101206. [PMID: 37499857 DOI: 10.1016/j.preteyeres.2023.101206] [Citation(s) in RCA: 9] [Impact Index Per Article: 9.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/04/2023] [Revised: 07/24/2023] [Accepted: 07/25/2023] [Indexed: 07/29/2023]
Abstract
There remain many unanswered questions on how to assess and treat the pathology and complications that arise from diabetic retinopathy (DR). Optical coherence tomography angiography (OCTA) is a novel and non-invasive three-dimensional imaging method that can visualize capillaries in all retinal layers. Numerous studies have confirmed that OCTA can identify early evidence of microvascular changes and provide quantitative assessment of the extent of diseases such as DR and its complications. A number of informative OCTA metrics could be used to assess DR in clinical trials, including measurements of the foveal avascular zone (FAZ; area, acircularity, 3D para-FAZ vessel density), vessel density, extrafoveal avascular zones, and neovascularization. Assessing patients with DR using a full-retinal slab OCTA image can limit segmentation errors and confounding factors such as those related to center-involved diabetic macular edema. Given emerging data suggesting the importance of the peripheral retinal vasculature in assessing and predicting DR progression, wide-field OCTA imaging should also be used. Finally, the use of automated methods and algorithms for OCTA image analysis, such as those that can distinguish between areas of true and false signals, reconstruct images, and produce quantitative metrics, such as FAZ area, will greatly improve the efficiency and standardization of results between studies. Most importantly, clinical trial protocols should account for the relatively high frequency of poor-quality data related to sub-optimal imaging conditions in DR and should incorporate time for assessing OCTA image quality and re-imaging patients where necessary.
Collapse
Affiliation(s)
- Nadia K Waheed
- New England Eye Center, Tufts University School of Medicine, Boston, MA, USA.
| | - Richard B Rosen
- New York Eye and Ear Infirmary of Mount Sinai, Icahn School of Medicine at Mount Sinai, New York, NY, USA
| | - Yali Jia
- School of Medicine, Casey Eye Institute, Oregon Health and Science University, Portland, OR, USA
| | - Marion R Munk
- Augenarzt-Praxisgemeinschaft Gutblick AG, Pfäffikon, Switzerland
| | - David Huang
- School of Medicine, Casey Eye Institute, Oregon Health and Science University, Portland, OR, USA
| | - Amani Fawzi
- Department of Ophthalmology, Feinberg School of Medicine, Northwestern University, Chicago, IL, USA
| | - Victor Chong
- Institute of Ophthalmology, University College London, London, UK
| | - Quan Dong Nguyen
- Byers Eye Institute, Stanford University School of Medicine, Stanford, CA, USA
| | - Yasir Sepah
- Byers Eye Institute, Stanford University School of Medicine, Stanford, CA, USA
| | | |
Collapse
|
5
|
Liao J, Zhang T, Li C, Huang Z. U-shaped fusion convolutional transformer based workflow for fast optical coherence tomography angiography generation in lips. BIOMEDICAL OPTICS EXPRESS 2023; 14:5583-5601. [PMID: 38021117 PMCID: PMC10659781 DOI: 10.1364/boe.502085] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Received: 08/01/2023] [Revised: 09/15/2023] [Accepted: 09/23/2023] [Indexed: 12/01/2023]
Abstract
Oral disorders, including oral cancer, pose substantial diagnostic challenges due to late-stage diagnosis, invasive biopsy procedures, and the limitations of existing non-invasive imaging techniques. Optical coherence tomography angiography (OCTA) shows potential in delivering non-invasive, real-time, high-resolution vasculature images. However, the quality of OCTA images are often compromised due to motion artifacts and noise, necessitating more robust and reliable image reconstruction approaches. To address these issues, we propose a novel model, a U-shaped fusion convolutional transformer (UFCT), for the reconstruction of high-quality, low-noise OCTA images from two-repeated OCT scans. UFCT integrates the strengths of convolutional neural networks (CNNs) and transformers, proficiently capturing both local and global image features. According to the qualitative and quantitative analysis in normal and pathological conditions, the performance of the proposed pipeline outperforms that of the traditional OCTA generation methods when only two repeated B-scans are performed. We further provide a comparative study with various CNN and transformer models and conduct ablation studies to validate the effectiveness of our proposed strategies. Based on the results, the UFCT model holds the potential to significantly enhance clinical workflow in oral medicine by facilitating early detection, reducing the need for invasive procedures, and improving overall patient outcomes.
Collapse
Affiliation(s)
- Jinpeng Liao
- School of Science and Engineering, University of Dundee, DD1 4HN, Scotland, United Kingdom
| | - Tianyu Zhang
- School of Science and Engineering, University of Dundee, DD1 4HN, Scotland, United Kingdom
| | - Chunhui Li
- School of Science and Engineering, University of Dundee, DD1 4HN, Scotland, United Kingdom
| | - Zhihong Huang
- School of Science and Engineering, University of Dundee, DD1 4HN, Scotland, United Kingdom
| |
Collapse
|
6
|
Ristic D, Resan M, Pancevski I, Ristic P, Vukosavljevic M, Cvetkovic M, Pajic B. Correlation of the OCT Double-Layer Sign with Type 1 Non-Exudative Neovascularization on OCT-A in Age-Related Macular Degeneration. MEDICINA (KAUNAS, LITHUANIA) 2023; 59:1829. [PMID: 37893547 PMCID: PMC10608565 DOI: 10.3390/medicina59101829] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 09/05/2023] [Revised: 10/09/2023] [Accepted: 10/11/2023] [Indexed: 10/29/2023]
Abstract
Background and Objectives: Early diagnosis of the exudative form of age-related macular degeneration (AMD) is very important for a timely first treatment, which is directly related to the preservation of functional visual acuity over a long period. The goal of this paper was to examine the correlation between the double-layer sign (DLS) and the presence of non-exudative macular neovascularization (MNV). Materials and Methods: Our research included 60 patients with AMD, exudative in one eye and non-exudative in the other eye. We analyzed only the non-exudative form using optical coherence tomography (OCT) and optical coherence tomography angiography (OCT-A). The patients were classified into three groups, depending on the duration of the disease (<2 years, 2 to 5 years, >5 years). The onset of the disease was deemed the moment of establishing a diagnosis of exudative AMD in one eye. We defined the presence or absence of a DLS using OCT and the presence of non-exudative MNV using OCT-A, both on 3 × 3 mm and 6 × 6 mm sections. DLS was used as a projection biomarker for non-exudative MNV, with the aim of establishing a rapid diagnosis and achieving early treatment of the disease. Results: We found that there was a statistically significant correlation between the DLS diagnosed using OCT and non-exudative MNV diagnosed by OCT-A for both 3 × 3 mm (p < 0.001) and 6 × 6 mm (p < 0.001) imaging. There was a statistically significant difference between the frequencies of both DLS and MNV in Groups I and III on both 3 × 3 and 6 × 6 mm imaging. A statistically significant difference was also noted in the frequencies of DLS and MNV on 6 × 6 mm imaging, but not on 3 × 3 mm imaging, between Groups I and II. No differences were found between the frequencies of DLS and MNV between Groups II and III. Conclusions: The DLS on OCT can be used as a projection biomarker to assess the presence of a non-exudative MNV.
Collapse
Affiliation(s)
- Dragana Ristic
- Eye Clinic, Military Medical Academy, 11000 Belgrade, Serbia; (M.R.); (I.P.); (M.C.)
- Faculty of Medicine of the Military Medical Academy, University of Defense, 11000 Belgrade, Serbia; (P.R.); (M.V.); (B.P.)
| | - Mirko Resan
- Eye Clinic, Military Medical Academy, 11000 Belgrade, Serbia; (M.R.); (I.P.); (M.C.)
- Faculty of Medicine of the Military Medical Academy, University of Defense, 11000 Belgrade, Serbia; (P.R.); (M.V.); (B.P.)
- Department of Physics, Faculty of Sciences, University of Novi Sad, 21000 Novi Sad, Serbia
| | - Igor Pancevski
- Eye Clinic, Military Medical Academy, 11000 Belgrade, Serbia; (M.R.); (I.P.); (M.C.)
- Faculty of Medicine of the Military Medical Academy, University of Defense, 11000 Belgrade, Serbia; (P.R.); (M.V.); (B.P.)
| | - Petar Ristic
- Faculty of Medicine of the Military Medical Academy, University of Defense, 11000 Belgrade, Serbia; (P.R.); (M.V.); (B.P.)
- Endocrinology Clinic, Military Medical Academy, 11000 Belgrade, Serbia
| | - Miroslav Vukosavljevic
- Faculty of Medicine of the Military Medical Academy, University of Defense, 11000 Belgrade, Serbia; (P.R.); (M.V.); (B.P.)
- Military Medical Academy Management, 11000 Belgrade, Serbia
| | - Milos Cvetkovic
- Eye Clinic, Military Medical Academy, 11000 Belgrade, Serbia; (M.R.); (I.P.); (M.C.)
- Faculty of Medicine of the Military Medical Academy, University of Defense, 11000 Belgrade, Serbia; (P.R.); (M.V.); (B.P.)
| | - Bojan Pajic
- Faculty of Medicine of the Military Medical Academy, University of Defense, 11000 Belgrade, Serbia; (P.R.); (M.V.); (B.P.)
- Department of Physics, Faculty of Sciences, University of Novi Sad, 21000 Novi Sad, Serbia
- Division of Ophthalmology, Department of Clinical Neurosciences, Geneva University Hospitals, 1205 Geneva, Switzerland
- Experimental Ophthalmology, University of Geneva, 1205 Geneva, Switzerland
- Eye Clinic ORASIS, Swiss Eye Research Foundation, 5734 Reinach, Switzerland
| |
Collapse
|
7
|
Abu-Qamar O, Lewis W, Mendonca LSM, De Sisternes L, Chin A, Alibhai AY, Gendelman I, Reichel E, Magazzeni S, Kubach S, Durbin M, Witkin AJ, Baumal CR, Duker JS, Waheed NK. Pseudoaveraging for denoising of OCT angiography: a deep learning approach for image quality enhancement in healthy and diabetic eyes. Int J Retina Vitreous 2023; 9:62. [PMID: 37822004 PMCID: PMC10568842 DOI: 10.1186/s40942-023-00486-5] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/02/2023] [Accepted: 08/02/2023] [Indexed: 10/13/2023] Open
Abstract
BACKGROUND This study aimed to develop a deep learning (DL) algorithm that enhances the quality of a single-frame enface OCTA scan to make it comparable to 4-frame averaged scan without the need for the repeated acquisitions required for averaging. METHODS Each of the healthy eyes and eyes from diabetic subjects that were prospectively enrolled in this cross-sectional study underwent four repeated 6 × 6 mm macular scans (PLEX Elite 9000 SS-OCT), and the repeated scans of each eye were co-registered to produce 4-frame averages. This prospective dataset of original (single-frame) enface scans and their corresponding averaged scans was divided into a training dataset and a validation dataset. In the training dataset, a DL algorithm (named pseudoaveraging) was trained using original scans as input and 4-frame averages as target. In the validation dataset, the pseudoaveraging algorithm was applied to single-frame scans to produce pseudoaveraged scans, and the single-frame and its corresponding averaged and pseudoaveraged scans were all qualitatively compared. In a separate retrospectively collected dataset of single-frame scans from eyes of diabetic subjects, the DL algorithm was applied, and the produced pseudoaveraged scan was qualitatively compared against its corresponding original. RESULTS This study included 39 eyes that comprised the prospective dataset (split into 5 eyes for training and 34 eyes for validating the DL algorithm), and 105 eyes that comprised the retrospective test dataset. Of the total 144 study eyes, 58% had any level of diabetic retinopathy (with and without diabetic macular edema), and the rest were from healthy eyes or eyes of diabetic subjects but without diabetic retinopathy and without macular edema. Grading results in the validation dataset showed that the pseudoaveraged enface scan ranked best in overall scan quality, background noise reduction, and visibility of microaneurysms (p < 0.05). Averaged scan ranked best for motion artifact reduction (p < 0.05). Grading results in the test dataset showed that pseudoaveraging resulted in enhanced small vessels, reduction of background noise, and motion artifact in 100%, 82%, and 98% of scans, respectively. Rates of false-positive/-negative perfusion were zero. CONCLUSION Pseudoaveraging is a feasible DL approach to more efficiently improve enface OCTA scan quality without introducing notable image artifacts.
Collapse
Affiliation(s)
- Omar Abu-Qamar
- New England Eye Center, Tufts Medical Center, 800 Washington St., Box 450, Boston, MA, 02111, USA
| | - Warren Lewis
- Research and Development, Carl Zeiss Meditec, Dublin, CA, 94568, USA
| | - Luisa S M Mendonca
- New England Eye Center, Tufts Medical Center, 800 Washington St., Box 450, Boston, MA, 02111, USA
- Department of Ophthalmology, Federal University of Sao Paulo, Sao Paulo, Brazil
| | - Luis De Sisternes
- Research and Development, Carl Zeiss Meditec, Dublin, CA, 94568, USA
| | - Adam Chin
- New England Eye Center, Tufts Medical Center, 800 Washington St., Box 450, Boston, MA, 02111, USA
| | - A Yasin Alibhai
- Boston Image Reading Center, 55 Causeway street, Boston, MA, 02114, USA
| | - Isaac Gendelman
- New England Eye Center, Tufts Medical Center, 800 Washington St., Box 450, Boston, MA, 02111, USA
| | - Elias Reichel
- New England Eye Center, Tufts Medical Center, 800 Washington St., Box 450, Boston, MA, 02111, USA
| | | | - Sophie Kubach
- Research and Development, Carl Zeiss Meditec, Dublin, CA, 94568, USA
| | - Mary Durbin
- Research and Development, Carl Zeiss Meditec, Dublin, CA, 94568, USA
| | - Andre J Witkin
- New England Eye Center, Tufts Medical Center, 800 Washington St., Box 450, Boston, MA, 02111, USA
| | - Caroline R Baumal
- New England Eye Center, Tufts Medical Center, 800 Washington St., Box 450, Boston, MA, 02111, USA
| | - Jay S Duker
- New England Eye Center, Tufts Medical Center, 800 Washington St., Box 450, Boston, MA, 02111, USA
| | - Nadia K Waheed
- New England Eye Center, Tufts Medical Center, 800 Washington St., Box 450, Boston, MA, 02111, USA.
| |
Collapse
|
8
|
Xu J, Yuan X, Huang Y, Qin J, Lan G, Qiu H, Yu B, Jia H, Tan H, Zhao S, Feng Z, An L, Wei X. Deep-learning visualization enhancement method for optical coherence tomography angiography in dermatology. JOURNAL OF BIOPHOTONICS 2023; 16:e202200366. [PMID: 37289020 DOI: 10.1002/jbio.202200366] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 11/24/2022] [Revised: 05/20/2023] [Accepted: 05/21/2023] [Indexed: 06/09/2023]
Abstract
Optical coherence tomography angiography (OCTA) in dermatology usually suffers from low image quality due to the highly scattering property of the skin, the complexity of cutaneous vasculature, and limited acquisition time. Deep-learning methods have achieved great success in many applications. However, the deep learning approach to improve dermatological OCTA images has not been investigated due to the requirement of high-performance OCTA systems and difficulty of obtaining high-quality images as ground truth. This study aims to generate proper datasets and develop a robust deep learning method to enhance the skin OCTA images. A swept-source skin OCTA system was employed to create low-quality and high-quality OCTA images with different scanning protocols. We propose a model named vascular visualization enhancement generative adversarial network and adopt an optimized data augmentation strategy and perceptual content loss function to achieve better image enhancement effect with small amount of training data. We demonstrate the superiority of the proposed method in skin OCTA image enhancement by quantitative and qualitative comparisons.
Collapse
Affiliation(s)
- Jingjiang Xu
- Guangdong-Hong Kong-Macao Joint Laboratory for Intelligent Micro-Nano Optoelectronic Technology, School of Physics and Optoelectronic Engineering, Foshan University, Foshan, China
- Innovation and Entrepreneurship Teams Project of Guangdong Provincial Pearl River Talents Program, Guangdong Weiren Meditech Co. Ltd, Foshan, Guangdong, China
| | - Xing Yuan
- School of Mechatronic Engineering and Automation, Foshan University, Foshan, Guangdong, China
| | - Yanping Huang
- Guangdong-Hong Kong-Macao Joint Laboratory for Intelligent Micro-Nano Optoelectronic Technology, School of Physics and Optoelectronic Engineering, Foshan University, Foshan, China
- Innovation and Entrepreneurship Teams Project of Guangdong Provincial Pearl River Talents Program, Guangdong Weiren Meditech Co. Ltd, Foshan, Guangdong, China
| | - Jia Qin
- Innovation and Entrepreneurship Teams Project of Guangdong Provincial Pearl River Talents Program, Guangdong Weiren Meditech Co. Ltd, Foshan, Guangdong, China
- Department of Ophthalmology, Clinical Medical Institute, Affiliated Hospital, Weifang Medical University, Weifang, Shandong, China
| | - Gongpu Lan
- Guangdong-Hong Kong-Macao Joint Laboratory for Intelligent Micro-Nano Optoelectronic Technology, School of Physics and Optoelectronic Engineering, Foshan University, Foshan, China
- Innovation and Entrepreneurship Teams Project of Guangdong Provincial Pearl River Talents Program, Guangdong Weiren Meditech Co. Ltd, Foshan, Guangdong, China
| | - Haixia Qiu
- Department of Laser Medicine, The First Medical Centre, Chinese PLA General Hospital, Beijing, China
| | - Bo Yu
- Department of Cardiovascular Medicine, The Second Affiliated Hospital of Harbin Medical University, Harbin, Heilongjiang, China
| | - Haibo Jia
- Department of Cardiovascular Medicine, The Second Affiliated Hospital of Harbin Medical University, Harbin, Heilongjiang, China
| | - Haishu Tan
- Guangdong-Hong Kong-Macao Joint Laboratory for Intelligent Micro-Nano Optoelectronic Technology, School of Physics and Optoelectronic Engineering, Foshan University, Foshan, China
| | - Shiyong Zhao
- Tianjin Hengyu Medical Technology Co., Ltd., Tianjin, China
| | - Zhongwu Feng
- School of Mechatronic Engineering and Automation, Foshan University, Foshan, Guangdong, China
| | - Lin An
- Innovation and Entrepreneurship Teams Project of Guangdong Provincial Pearl River Talents Program, Guangdong Weiren Meditech Co. Ltd, Foshan, Guangdong, China
- Department of Ophthalmology, Clinical Medical Institute, Affiliated Hospital, Weifang Medical University, Weifang, Shandong, China
| | - Xunbin Wei
- Biomedical Engineering Department, Peking University, Beijing, China
| |
Collapse
|
9
|
Hormel TT, Jia Y. OCT angiography and its retinal biomarkers [Invited]. BIOMEDICAL OPTICS EXPRESS 2023; 14:4542-4566. [PMID: 37791289 PMCID: PMC10545210 DOI: 10.1364/boe.495627] [Citation(s) in RCA: 4] [Impact Index Per Article: 4.0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 05/15/2023] [Revised: 07/13/2023] [Accepted: 07/13/2023] [Indexed: 10/05/2023]
Abstract
Optical coherence tomography angiography (OCTA) is a high-resolution, depth-resolved imaging modality with important applications in ophthalmic practice. An extension of structural OCT, OCTA enables non-invasive, high-contrast imaging of retinal and choroidal vasculature that are amenable to quantification. As such, OCTA offers the capability to identify and characterize biomarkers important for clinical practice and therapeutic research. Here, we review new methods for analyzing biomarkers and discuss new insights provided by OCTA.
Collapse
Affiliation(s)
- Tristan T. Hormel
- Casey Eye Institute, Oregon Health & Science University, Portland, Oregon, USA
| | - Yali Jia
- Casey Eye Institute, Oregon Health & Science University, Portland, Oregon, USA
- Department of Biomedical Engineering, Oregon Health & Science University, Portland, Oregon, USA
| |
Collapse
|
10
|
Liao J, Yang S, Zhang T, Li C, Huang Z. Fast optical coherence tomography angiography image acquisition and reconstruction pipeline for skin application. BIOMEDICAL OPTICS EXPRESS 2023; 14:3899-3913. [PMID: 37799685 PMCID: PMC10549725 DOI: 10.1364/boe.486933] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Received: 02/03/2023] [Revised: 04/19/2023] [Accepted: 04/19/2023] [Indexed: 10/07/2023]
Abstract
Traditional high-quality OCTA images require multi-repeated scans (e.g., 4-8 repeats) in the same position, which may cause the patient to be uncomfortable. We propose a deep-learning-based pipeline that can extract high-quality OCTA images from only two-repeat OCT scans. The performance of the proposed image reconstruction U-Net (IRU-Net) outperforms the state-of-the-art UNet vision transformer and UNet in OCTA image reconstruction from a two-repeat OCT signal. The results demonstrated a mean peak-signal-to-noise ratio increased from 15.7 to 24.2; the mean structural similarity index measure improved from 0.28 to 0.59, while the OCT data acquisition time was reduced from 21 seconds to 3.5 seconds (reduced by 83%).
Collapse
Affiliation(s)
- Jinpeng Liao
- School of Science and Engineering,
University of Dundee, DD1 4HN, Scotland, UK
| | - Shufan Yang
- Engineering and Built Environment, Edinburgh Napier University, Edinburgh, UK
- Research Department of Orthopaedics and Musculoskeletal Science, University College London, UK
| | - Tianyu Zhang
- School of Science and Engineering,
University of Dundee, DD1 4HN, Scotland, UK
| | - Chunhui Li
- School of Science and Engineering,
University of Dundee, DD1 4HN, Scotland, UK
| | - Zhihong Huang
- School of Science and Engineering,
University of Dundee, DD1 4HN, Scotland, UK
| |
Collapse
|
11
|
Verbeek S, Dalvin LA. Advances in multimodal imaging for diagnosis of pigmented ocular fundus lesions. CANADIAN JOURNAL OF OPHTHALMOLOGY 2023:S0008-4182(23)00209-0. [PMID: 37480939 PMCID: PMC10796850 DOI: 10.1016/j.jcjo.2023.07.005] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 09/19/2022] [Revised: 05/25/2023] [Accepted: 07/06/2023] [Indexed: 07/24/2023]
Abstract
Pigmented ocular fundus lesions can range from benign to malignant. While observation is reasonable for asymptomatic benign lesions, early recognition of tumours that are vision or life threatening is critical for long-term prognosis. With recent advances and increased accessibility of multimodal imaging, it is important that providers understand how to best use these tools to detect tumours that require early referral to subspecialty centres. This review aims to provide an overview of pigmented ocular fundus lesions and their defining characteristics using multimodal imaging. We cover the spectrum of pigmented ocular fundus lesions, including freckle and focal aggregates of normal or near-normal uveal melanocytes, retinal pigment epithelium (RPE) hyperplasia, congenital hypertrophy of the RPE, RPE hamartoma associated with familial adenomatous polyposis, congenital simple hamartoma of the RPE, combined hamartoma of the retina and RPE (congenital hypertrophy of the RPE), choroidal nevus, melanocytosis, melanocytoma, melanoma, adenoma, and RPE adenocarcinoma. We describe key diagnostic features using multimodal imaging modalities of ultra-widefield fundus photography, fundus autofluorescence, optical coherence tomography (OCT), enhanced-depth imaging OCT, ultrasonography, fluorescein angiography, indocyanine green angiography, and OCT angiography (OCTA), with particular attention to diagnostic features that could be missed on fundus examination alone. Finally, we review what is on the horizon, including applications of artificial intelligence. Through skilled application of current and emerging imaging technologies, earlier detection of sight- and life-threatening melanocytic ocular fundus tumours can lead to improved patient prognosis.
Collapse
Affiliation(s)
- Sara Verbeek
- Department of Ophthalmology, Mayo Clinic, Rochester, MN
| | | |
Collapse
|
12
|
Ong CJT, Wong MYZ, Cheong KX, Zhao J, Teo KYC, Tan TE. Optical Coherence Tomography Angiography in Retinal Vascular Disorders. Diagnostics (Basel) 2023; 13:diagnostics13091620. [PMID: 37175011 PMCID: PMC10178415 DOI: 10.3390/diagnostics13091620] [Citation(s) in RCA: 2] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/07/2023] [Revised: 04/28/2023] [Accepted: 05/01/2023] [Indexed: 05/15/2023] Open
Abstract
Traditionally, abnormalities of the retinal vasculature and perfusion in retinal vascular disorders, such as diabetic retinopathy and retinal vascular occlusions, have been visualized with dye-based fluorescein angiography (FA). Optical coherence tomography angiography (OCTA) is a newer, alternative modality for imaging the retinal vasculature, which has some advantages over FA, such as its dye-free, non-invasive nature, and depth resolution. The depth resolution of OCTA allows for characterization of the retinal microvasculature in distinct anatomic layers, and commercial OCTA platforms also provide automated quantitative vascular and perfusion metrics. Quantitative and qualitative OCTA analysis in various retinal vascular disorders has facilitated the detection of pre-clinical vascular changes, greater understanding of known clinical signs, and the development of imaging biomarkers to prognosticate and guide treatment. With further technological improvements, such as a greater field of view and better image quality processing algorithms, it is likely that OCTA will play an integral role in the study and management of retinal vascular disorders. Artificial intelligence methods-in particular, deep learning-show promise in refining the insights to be gained from the use of OCTA in retinal vascular disorders. This review aims to summarize the current literature on this imaging modality in relation to common retinal vascular disorders.
Collapse
Affiliation(s)
- Charles Jit Teng Ong
- Singapore Eye Research Institute, Singapore National Eye Centre, Singapore 168751, Singapore
| | - Mark Yu Zheng Wong
- Singapore Eye Research Institute, Singapore National Eye Centre, Singapore 168751, Singapore
| | - Kai Xiong Cheong
- Singapore Eye Research Institute, Singapore National Eye Centre, Singapore 168751, Singapore
| | - Jinzhi Zhao
- Singapore Eye Research Institute, Singapore National Eye Centre, Singapore 168751, Singapore
| | - Kelvin Yi Chong Teo
- Singapore Eye Research Institute, Singapore National Eye Centre, Singapore 168751, Singapore
- Ophthalmology and Visual Sciences Academic Clinical Program (EYE ACP), Duke-NUS Medical School, Singapore 169857, Singapore
| | - Tien-En Tan
- Singapore Eye Research Institute, Singapore National Eye Centre, Singapore 168751, Singapore
- Ophthalmology and Visual Sciences Academic Clinical Program (EYE ACP), Duke-NUS Medical School, Singapore 169857, Singapore
| |
Collapse
|
13
|
Zhou Y, Lin G, Yu X, Cao Y, Cheng H, Shi C, Jiang J, Gao H, Lu F, Shen M. Deep learning segmentation of the tear fluid reservoir under the sclera lens in optical coherence tomography images. BIOMEDICAL OPTICS EXPRESS 2023; 14:1848-1861. [PMID: 37206122 PMCID: PMC10191653 DOI: 10.1364/boe.480247] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Received: 12/19/2022] [Revised: 01/31/2023] [Accepted: 02/05/2023] [Indexed: 05/21/2023]
Abstract
The tear fluid reservoir (TFR) under the sclera lens is a unique characteristic providing optical neutralization of any aberrations from corneal irregularities. Anterior segment optical coherence tomography (AS-OCT) has become an important imaging modality for sclera lens fitting and visual rehabilitation therapy in both optometry and ophthalmology. Herein, we aimed to investigate whether deep learning can be used to segment the TFR from healthy and keratoconus eyes, with irregular corneal surfaces, in OCT images. Using AS-OCT, a dataset of 31850 images from 52 healthy and 46 keratoconus eyes, during sclera lens wear, was obtained and labeled with our previously developed algorithm of semi-automatic segmentation. A custom-improved U-shape network architecture with a full-range multi-scale feature-enhanced module (FMFE-Unet) was designed and trained. A hybrid loss function was designed to focus training on the TFR, to tackle the class imbalance problem. The experiments on our database showed an IoU, precision, specificity, and recall of 0.9426, 0.9678, 0.9965, and 0.9731, respectively. Furthermore, FMFE-Unet was found to outperform the other two state-of-the-art methods and ablation models, suggesting its strength in segmenting the TFR under the sclera lens depicted on OCT images. The application of deep learning for TFR segmentation in OCT images provides a powerful tool to assess changes in the dynamic tear film under the sclera lens, improving the efficiency and accuracy of lens fitting, and thus supporting the promotion of sclera lenses in clinical practice.
Collapse
Affiliation(s)
- Yuheng Zhou
- Eye Hospital and School of Ophthalmology and Optometry, Wenzhou Medical University, Wenzhou, Zhejiang 325000, China
| | - Guangqing Lin
- Eye Hospital and School of Ophthalmology and Optometry, Wenzhou Medical University, Wenzhou, Zhejiang 325000, China
| | - Xiangle Yu
- Eye Hospital and School of Ophthalmology and Optometry, Wenzhou Medical University, Wenzhou, Zhejiang 325000, China
| | - Yang Cao
- Eye Hospital and School of Ophthalmology and Optometry, Wenzhou Medical University, Wenzhou, Zhejiang 325000, China
| | - Hongling Cheng
- Eye Hospital and School of Ophthalmology and Optometry, Wenzhou Medical University, Wenzhou, Zhejiang 325000, China
| | - Ce Shi
- Eye Hospital and School of Ophthalmology and Optometry, Wenzhou Medical University, Wenzhou, Zhejiang 325000, China
| | - Jun Jiang
- Eye Hospital and School of Ophthalmology and Optometry, Wenzhou Medical University, Wenzhou, Zhejiang 325000, China
| | - Hebei Gao
- Oujiang Laboratory (Zhejiang Lab for Regenerative Medicine, Vision and Brain Health), Eye Hospital and School of Ophthalmology and Optometry, Wenzhou Medical University, Wenzhou, 325000, China
| | - Fan Lu
- Eye Hospital and School of Ophthalmology and Optometry, Wenzhou Medical University, Wenzhou, Zhejiang 325000, China
- Oujiang Laboratory (Zhejiang Lab for Regenerative Medicine, Vision and Brain Health), Eye Hospital and School of Ophthalmology and Optometry, Wenzhou Medical University, Wenzhou, 325000, China
| | - Meixiao Shen
- Eye Hospital and School of Ophthalmology and Optometry, Wenzhou Medical University, Wenzhou, Zhejiang 325000, China
- Oujiang Laboratory (Zhejiang Lab for Regenerative Medicine, Vision and Brain Health), Eye Hospital and School of Ophthalmology and Optometry, Wenzhou Medical University, Wenzhou, 325000, China
| |
Collapse
|
14
|
Optical Coherence Tomography Angiography of the Intestine: How to Prevent Motion Artifacts in Open and Laparoscopic Surgery? Life (Basel) 2023; 13:life13030705. [PMID: 36983861 PMCID: PMC10055682 DOI: 10.3390/life13030705] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/07/2023] [Revised: 02/25/2023] [Accepted: 02/28/2023] [Indexed: 03/08/2023] Open
Abstract
(1) Introduction. The problem that limits the intraoperative use of OCTA for the intestinal circulation diagnostics is the low informative value of OCTA images containing too many motion artifacts. The aim of this study is to evaluate the efficiency and safety of the developed unit for the prevention of the appearance of motion artifacts in the OCTA images of the intestine in both open and laparoscopic surgery in the experiment; (2) Methods. A high-speed spectral-domain multimodal optical coherence tomograph (IAP RAS, Russia) operating at a wavelength of 1310 nm with a spectral width of 100 μm and a power of 2 mW was used. The developed unit was tested in two groups of experimental animals—on minipigs (group I, n = 10, open abdomen) and on rabbits (group II, n = 10, laparoscopy). Acute mesenteric ischemia was modeled and then 1 h later the small intestine underwent OCTA evaluation. A total of 400 OCTA images of the intact and ischemic small intestine were obtained and analyzed. The quality of the obtained OCTA images was evaluated based on the score proposed in 2020 by the group of Magnin M. (3) Results. Without stabilization, OCTA images of the intestine tissues were informative only in 32–44% of cases in open surgery and in 14–22% of cases in laparoscopic surgery. A vacuum bowel stabilizer with a pressure deficit of 22–25 mm Hg significantly reduced the number of motion artifacts. As a result, the proportion of informative OCTA images in open surgery increased up to 86.5% (Χ2 = 200.2, p = 0.001), and in laparoscopy up to 60% (Χ2 = 148.3, p = 0.001). (4) Conclusions. The used vacuum tissue stabilizer enabled a significant increase in the proportion of informative OCTA images by significantly reducing the motion artifacts.
Collapse
|
15
|
Deep Learning in Optical Coherence Tomography Angiography: Current Progress, Challenges, and Future Directions. Diagnostics (Basel) 2023; 13:diagnostics13020326. [PMID: 36673135 PMCID: PMC9857993 DOI: 10.3390/diagnostics13020326] [Citation(s) in RCA: 12] [Impact Index Per Article: 12.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/28/2022] [Revised: 01/11/2023] [Accepted: 01/12/2023] [Indexed: 01/18/2023] Open
Abstract
Optical coherence tomography angiography (OCT-A) provides depth-resolved visualization of the retinal microvasculature without intravenous dye injection. It facilitates investigations of various retinal vascular diseases and glaucoma by assessment of qualitative and quantitative microvascular changes in the different retinal layers and radial peripapillary layer non-invasively, individually, and efficiently. Deep learning (DL), a subset of artificial intelligence (AI) based on deep neural networks, has been applied in OCT-A image analysis in recent years and achieved good performance for different tasks, such as image quality control, segmentation, and classification. DL technologies have further facilitated the potential implementation of OCT-A in eye clinics in an automated and efficient manner and enhanced its clinical values for detecting and evaluating various vascular retinopathies. Nevertheless, the deployment of this combination in real-world clinics is still in the "proof-of-concept" stage due to several limitations, such as small training sample size, lack of standardized data preprocessing, insufficient testing in external datasets, and absence of standardized results interpretation. In this review, we introduce the existing applications of DL in OCT-A, summarize the potential challenges of the clinical deployment, and discuss future research directions.
Collapse
|
16
|
Ma D, Pasquale LR, Girard MJA, Leung CKS, Jia Y, Sarunic MV, Sappington RM, Chan KC. Reverse translation of artificial intelligence in glaucoma: Connecting basic science with clinical applications. FRONTIERS IN OPHTHALMOLOGY 2023; 2:1057896. [PMID: 36866233 PMCID: PMC9976697 DOI: 10.3389/fopht.2022.1057896] [Citation(s) in RCA: 3] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 09/30/2022] [Accepted: 12/05/2022] [Indexed: 04/16/2023]
Abstract
Artificial intelligence (AI) has been approved for biomedical research in diverse areas from bedside clinical studies to benchtop basic scientific research. For ophthalmic research, in particular glaucoma, AI applications are rapidly growing for potential clinical translation given the vast data available and the introduction of federated learning. Conversely, AI for basic science remains limited despite its useful power in providing mechanistic insight. In this perspective, we discuss recent progress, opportunities, and challenges in the application of AI in glaucoma for scientific discoveries. Specifically, we focus on the research paradigm of reverse translation, in which clinical data are first used for patient-centered hypothesis generation followed by transitioning into basic science studies for hypothesis validation. We elaborate on several distinctive areas of research opportunities for reverse translation of AI in glaucoma including disease risk and progression prediction, pathology characterization, and sub-phenotype identification. We conclude with current challenges and future opportunities for AI research in basic science for glaucoma such as inter-species diversity, AI model generalizability and explainability, as well as AI applications using advanced ocular imaging and genomic data.
Collapse
Affiliation(s)
- Da Ma
- School of Medicine, Wake Forest University, Winston-Salem, NC, United States
- Atrium Health Wake Forest Baptist Medical Center, Winston-Salem, NC, United States
- School of Engineering Science, Simon Fraser University, Burnaby, BC, Canada
| | - Louis R. Pasquale
- Department of Ophthalmology, Icahn School of Medicine at Mount Sinai, New York, NY, United States
| | - Michaël J. A. Girard
- Ophthalmic Engineering & Innovation Laboratory (OEIL), Singapore Eye Research Institute, Singapore National Eye Centre, Singapore, Singapore
- Duke-NUS Medical School, Singapore, Singapore
- Institute for Molecular and Clinical Ophthalmology, Basel, Switzerland
| | | | - Yali Jia
- Casey Eye Institute, Oregon Health & Science University, Portland, OR, United States
| | - Marinko V. Sarunic
- School of Engineering Science, Simon Fraser University, Burnaby, BC, Canada
- Institute of Ophthalmology, University College London, London, United Kingdom
| | - Rebecca M. Sappington
- School of Medicine, Wake Forest University, Winston-Salem, NC, United States
- Atrium Health Wake Forest Baptist Medical Center, Winston-Salem, NC, United States
| | - Kevin C. Chan
- Departments of Ophthalmology and Radiology, Neuroscience Institute, NYU Grossman School of Medicine, NYU Langone Health, New York University, New York, NY, United States
- Department of Biomedical Engineering, Tandon School of Engineering, New York University, New York, NY, United States
| |
Collapse
|
17
|
Schottenhamml J, Hohberger B, Mardin CY. Applications of Artificial Intelligence in Optical Coherence Tomography Angiography Imaging. Klin Monbl Augenheilkd 2022; 239:1412-1426. [PMID: 36493762 DOI: 10.1055/a-1961-7137] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/13/2022]
Abstract
Optical coherence tomography angiography (OCTA) and artificial intelligence (AI) are two emerging fields that complement each other. OCTA enables the noninvasive, in vivo, 3D visualization of retinal blood flow with a micrometer resolution, which has been impossible with other imaging modalities. As it does not need dye-based injections, it is also a safer procedure for patients. AI has excited great interest in many fields of daily life, by enabling automatic processing of huge amounts of data with a performance that greatly surpasses previous algorithms. It has been used in many breakthrough studies in recent years, such as the finding that AlphaGo can beat humans in the strategic board game of Go. This paper will give a short introduction into both fields and will then explore the manifold applications of AI in OCTA imaging that have been presented in the recent years. These range from signal generation over signal enhancement to interpretation tasks like segmentation and classification. In all these areas, AI-based algorithms have achieved state-of-the-art performance that has the potential to improve standard care in ophthalmology when integrated into the daily clinical routine.
Collapse
Affiliation(s)
- Julia Schottenhamml
- Augenklinik, Friedrich-Alexander-Universität Erlangen-Nürnberg, Erlangen, Germany
| | - Bettina Hohberger
- Augenklinik, Friedrich-Alexander-Universität Erlangen-Nürnberg, Erlangen, Germany
| | | |
Collapse
|
18
|
Image enhancement of wide-field retinal optical coherence tomography angiography by super-resolution angiogram reconstruction generative adversarial network. Biomed Signal Process Control 2022. [DOI: 10.1016/j.bspc.2022.103957] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/19/2022]
|
19
|
Hao H, Xu C, Zhang D, Yan Q, Zhang J, Liu Y, Zhao Y. Sparse-based Domain Adaptation Network for OCTA Image Super-Resolution Reconstruction. IEEE J Biomed Health Inform 2022; 26:4402-4413. [PMID: 35895639 DOI: 10.1109/jbhi.2022.3194025] [Citation(s) in RCA: 2] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/06/2022]
Abstract
Retinal Optical Coherence Tomography Angiography (OCTA) with high-resolution is important for the quantification and analysis of retinal vasculature. However, the resolution of OCTA images is inversely proportional to the field of view at the same sampling frequency, which is not conducive to clinicians for analyzing larger vascular areas. In this paper, we propose a novel Sparse-based domain Adaptation Super-Resolution network (SASR) for the reconstruction of realistic [Formula: see text]/low-resolution (LR) OCTA images to high-resolution (HR) representations. To be more specific, we first perform a simple degradation of the [Formula: see text]/high-resolution (HR) image to obtain the synthetic LR image. An efficient registration method is then employed to register the synthetic LR with its corresponding [Formula: see text] image region within the [Formula: see text] image to obtain the cropped realistic LR image. We then propose a multi-level super-resolution model for the fully-supervised reconstruction of the synthetic data, guiding the reconstruction of the realistic LR images through a generative-adversarial strategy that allows the synthetic and realistic LR images to be unified in the feature domain. Finally, a novel sparse edge-aware loss is designed to dynamically optimize the vessel edge structure. Extensive experiments on two OCTA sets have shown that our method performs better than state-of-the-art super-resolution reconstruction methods. In addition, we have investigated the performance of the reconstruction results on retina structure segmentations, which further validate the effectiveness of our approach.
Collapse
|
20
|
Gao M, Guo Y, Hormel TT, Tsuboi K, Pacheco G, Poole D, Bailey ST, Flaxel CJ, Huang D, Hwang TS, Jia Y. A Deep Learning Network for Classifying Arteries and Veins in Montaged Widefield OCT Angiograms. OPHTHALMOLOGY SCIENCE 2022; 2:100149. [PMID: 36278031 PMCID: PMC9562370 DOI: 10.1016/j.xops.2022.100149] [Citation(s) in RCA: 11] [Impact Index Per Article: 5.5] [Reference Citation Analysis] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 10/11/2021] [Revised: 03/16/2022] [Accepted: 03/28/2022] [Indexed: 01/18/2023]
|
21
|
Kim G, Kim J, Choi WJ, Kim C, Lee S. Integrated deep learning framework for accelerated optical coherence tomography angiography. Sci Rep 2022; 12:1289. [PMID: 35079046 PMCID: PMC8789830 DOI: 10.1038/s41598-022-05281-0] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/07/2021] [Accepted: 01/10/2022] [Indexed: 11/26/2022] Open
Abstract
Label-free optical coherence tomography angiography (OCTA) has become a premium imaging tool in clinics to obtain structural and functional information of microvasculatures. One primary technical drawback for OCTA, however, is its imaging speed. The current protocols require high sampling density and multiple acquisitions of cross-sectional B-scans to form one image frame, resulting in low acquisition speed. Recently, deep learning (DL)-based methods have gained attention in accelerating the OCTA acquisition process. They achieve faster acquisition using two independent reconstructing approaches: high-quality angiograms from a few repeated B-scans and high-resolution angiograms from undersampled data. While these approaches have shown promising results, they provide limited solutions that only partially account for the OCTA scanning mechanism. Herein, we propose an integrated DL method to simultaneously tackle both factors and further enhance the reconstruction performance in speed and quality. We designed an end-to-end deep neural network (DNN) framework with a two-staged adversarial training scheme to reconstruct fully-sampled, high-quality (8 repeated B-scans) angiograms from their corresponding undersampled, low-quality (2 repeated B-scans) counterparts by successively enhancing the pixel resolution and the image quality. Using an in-vivo mouse brain vasculature dataset, we evaluate our proposed framework through quantitative and qualitative assessments and demonstrate that our method can achieve superior reconstruction performance compared to the conventional means. Our DL-based framework can accelerate the OCTA imaging speed from 16 to 256[Formula: see text] while preserving the image quality, thus enabling a convenient software-only solution to enhance preclinical and clinical studies.
Collapse
Affiliation(s)
- Gyuwon Kim
- Department of Mechanical Engineering, Pohang University of Science and Technology (POSTECH), Pohang, 37673, Republic of Korea
| | - Jongbeom Kim
- Department of Mechanical Engineering, Pohang University of Science and Technology (POSTECH), Pohang, 37673, Republic of Korea
- Departments of Electrical Engineering and Convergence I.T. Engineering, Medical Device Innovation Center, Pohang University of Science and Technology (POSTECH), Pohang, 37673, Republic of Korea
| | - Woo June Choi
- School of Electrical and Electronics Engineering, College of ICT Engineering, Chung-Ang University, Seoul, 06974, Republic of Korea.
| | - Chulhong Kim
- Department of Mechanical Engineering, Pohang University of Science and Technology (POSTECH), Pohang, 37673, Republic of Korea.
- Departments of Electrical Engineering and Convergence I.T. Engineering, Medical Device Innovation Center, Pohang University of Science and Technology (POSTECH), Pohang, 37673, Republic of Korea.
- Graduate School of Artificial Intelligence, Pohang University of Science and Technology (POSTECH), Pohang, 37673, Republic of Korea.
| | - Seungchul Lee
- Department of Mechanical Engineering, Pohang University of Science and Technology (POSTECH), Pohang, 37673, Republic of Korea.
- Graduate School of Artificial Intelligence, Pohang University of Science and Technology (POSTECH), Pohang, 37673, Republic of Korea.
| |
Collapse
|
22
|
Bouma B, de Boer J, Huang D, Jang I, Yonetsu T, Leggett C, Leitgeb R, Sampson D, Suter M, Vakoc B, Villiger M, Wojtkowski M. Optical coherence tomography. NATURE REVIEWS. METHODS PRIMERS 2022; 2:79. [PMID: 36751306 PMCID: PMC9901537 DOI: 10.1038/s43586-022-00162-2] [Citation(s) in RCA: 12] [Impact Index Per Article: 6.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 02/09/2023]
Abstract
Optical coherence tomography (OCT) is a non-contact method for imaging the topological and internal microstructure of samples in three dimensions. OCT can be configured as a conventional microscope, as an ophthalmic scanner, or using endoscopes and small diameter catheters for accessing internal biological organs. In this Primer, we describe the principles underpinning the different instrument configurations that are tailored to distinct imaging applications and explain the origin of signal, based on light scattering and propagation. Although OCT has been used for imaging inanimate objects, we focus our discussion on biological and medical imaging. We examine the signal processing methods and algorithms that make OCT exquisitely sensitive to reflections as weak as just a few photons and that reveal functional information in addition to structure. Image processing, display and interpretation, which are all critical for effective biomedical imaging, are discussed in the context of specific applications. Finally, we consider image artifacts and limitations that commonly arise and reflect on future advances and opportunities.
Collapse
Affiliation(s)
- B.E. Bouma
- Wellman Center for Photomedicine, Massachusetts General Hospital, Boston, MA, USA,Institute for Medical Engineering and Physics, Massachusetts Institute of Technology, Cambridge, MA, USA,Harvard Medical School, Boston, MA, USA,Corresponding author:
| | - J.F. de Boer
- Department of Physics and Astronomy, Vrije Universiteit Amsterdam, Amsterdam, The Netherlands
| | - D. Huang
- Casey Eye Institute, Oregon Health and Science University, Portland, OR, USA
| | - I.K. Jang
- Harvard Medical School, Boston, MA, USA,Cardiology Division, Massachusetts General Hospital, Boston, MA, USA
| | - T. Yonetsu
- Department of Cardiovascular Medicine, Tokyo Medical and Dental University
| | - C.L. Leggett
- Division of Gastroenterology and Hepatology, Mayo Clinic, Rochester, MN, USA
| | - R. Leitgeb
- Institute of Medical Physics, University of Vienna, Wien, Austria
| | - D.D. Sampson
- School of Physics and School of Biosciences and Medicine, University of Surrey, Guildford, United Kingdom
| | - M. Suter
- Harvard Medical School, Boston, MA, USA,Division of Pulmonary and Critical Care Medicine, Massachusetts General Hospital, Boston, MA, USA
| | - B. Vakoc
- Wellman Center for Photomedicine, Massachusetts General Hospital, Boston, MA, USA,Harvard Medical School, Boston, MA, USA
| | - M. Villiger
- Wellman Center for Photomedicine, Massachusetts General Hospital, Boston, MA, USA,Harvard Medical School, Boston, MA, USA
| | - M. Wojtkowski
- Institute of Physical Chemistry and International Center for Translational Eye Research, Institute of Physical Chemistry, Polish Academy of Sciences, Warsaw, Poland,Faculty of Physics, Astronomy and Informatics, Nicolaus Copernicus University, Torun, Poland
| |
Collapse
|
23
|
Kalra G, Zarranz-Ventura J, Chahal R, Bernal-Morales C, Lupidi M, Chhablani J. Optical coherence tomography (OCT) angiolytics: a review of OCT angiography quantitative biomarkers. Surv Ophthalmol 2021; 67:1118-1134. [PMID: 34748794 DOI: 10.1016/j.survophthal.2021.11.002] [Citation(s) in RCA: 14] [Impact Index Per Article: 4.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/28/2021] [Revised: 10/29/2021] [Accepted: 11/01/2021] [Indexed: 02/08/2023]
Abstract
Optical coherence tomography angiography (OCTA) provides a non-invasive method to obtain angiography of the chorioretinal vasculature leading to its recent widespread adoption. With a growing number of studies exploring the use of OCTA, various biomarkers quantifying the vascular characteristics have come to light. In the current report, we summarize the biomarkers currently described for retinal and choroidal vasculature using OCTA systems and the methods used to obtain them. Further, we present a critical review of these methods and key findings in common retinal diseases and appraise future directions, including applications of artificial intelligence in OCTA .
Collapse
Affiliation(s)
- Gagan Kalra
- Cole Eye Institute, Cleveland Clinic, Cleveland, OH, USA; Government Medical College and Hospital, Chandigarh, India
| | - Javier Zarranz-Ventura
- Institut Clinic d'Oftalmologia (ICOF) Hospital Clinic, Barcelona, Spain; Institut de Investigacions Biomediques August Pi i Sunyer (IDIBAPS), Barcelona, Spain
| | - Rutvi Chahal
- Government Medical College and Hospital, Chandigarh, India
| | - Carolina Bernal-Morales
- Institut Clinic d'Oftalmologia (ICOF) Hospital Clinic, Barcelona, Spain; Institut de Investigacions Biomediques August Pi i Sunyer (IDIBAPS), Barcelona, Spain
| | - Marco Lupidi
- Department of Surgical and Biomedical Sciences, University of Perugia, S.Maria della Misericordia Hospital, Perugia, Italy
| | - Jay Chhablani
- University of Pittsburgh Medical Center Eye Center, University of Pittsburgh, Pittsburgh, PA, USA.
| |
Collapse
|
24
|
Gao M, Hormel TT, Wang J, Guo Y, Bailey ST, Hwang TS, Jia Y. An Open-Source Deep Learning Network for Reconstruction of High-Resolution OCT Angiograms of Retinal Intermediate and Deep Capillary Plexuses. Transl Vis Sci Technol 2021; 10:13. [PMID: 34757393 PMCID: PMC8590160 DOI: 10.1167/tvst.10.13.13] [Citation(s) in RCA: 10] [Impact Index Per Article: 3.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/03/2021] [Accepted: 10/06/2021] [Indexed: 01/27/2023] Open
Abstract
Purpose We propose a deep learning-based image reconstruction algorithm to produce high-resolution optical coherence tomographic angiograms (OCTA) of the intermediate capillary plexus (ICP) and deep capillary plexus (DCP). Methods In this study, 6-mm × 6-mm macular scans with a 400 × 400 A-line sampling density and 3-mm × 3-mm scans with a 304 × 304 A-line sampling density were acquired on one or both eyes of 180 participants (including 230 eyes with diabetic retinopathy and 44 healthy controls) using a 70-kHz commercial OCT system (RTVue-XR; Optovue, Inc., Fremont, California, USA). Projection-resolved OCTA algorithm removed projection artifacts in voxel. ICP and DCP angiograms were generated by maximum projection of the OCTA signal within the respective plexus. We proposed a deep learning-based method, which receives inputs from registered 3-mm × 3-mm ICP and DCP angiograms with proper sampling density as the ground truth reference to reconstruct 6-mm × 6-mm high-resolution ICP and DCP en face OCTA. We applied the same network on 3-mm × 3-mm angiograms to enhance these images further. We evaluated the reconstructed 3-mm × 3-mm and 6-mm × 6-mm angiograms based on vascular connectivity, Weber contrast, false flow signal (flow signal erroneously generated from background), and the noise intensity in the foveal avascular zone. Results Compared to the originals, the Deep Capillary Angiogram Reconstruction Network (DCARnet)-enhanced 6-mm × 6-mm angiograms had significantly reduced noise intensity (ICP, 7.38 ± 25.22, P < 0.001; DCP, 11.20 ± 22.52, P < 0.001), improved vascular connectivity (ICP, 0.95 ± 0.01, P < 0.001; DCP, 0.96 ± 0.01, P < 0.001), and enhanced Weber contrast (ICP, 4.25 ± 0.10, P < 0.001; DCP, 3.84 ± 0.84, P < 0.001), without generating false flow signal when noise intensity lower than 650. The DCARnet-enhanced 3-mm × 3-mm angiograms also reduced noise, improved connectivity, and enhanced Weber contrast in 3-mm × 3-mm ICP and DCP angiograms from 101 eyes. In addition, DCARnet preserved the appearance of the dilated vessels in the reconstructed angiograms in diabetic eyes. Conclusions DCARnet can enhance 3-mm × 3-mm and 6-mm × 6-mm ICP and DCP angiogram image quality without introducing artifacts. Translational Relevance The enhanced 6-mm × 6-mm angiograms may be easier for clinicians to interpret qualitatively.
Collapse
Affiliation(s)
- Min Gao
- Casey Eye Institute, Oregon Health & Science University, Portland, OR, USA
| | - Tristan T. Hormel
- Casey Eye Institute, Oregon Health & Science University, Portland, OR, USA
| | - Jie Wang
- Casey Eye Institute, Oregon Health & Science University, Portland, OR, USA
- Department of Biomedical Engineering, Oregon Health & Science University, Portland, OR, USA
| | - Yukun Guo
- Casey Eye Institute, Oregon Health & Science University, Portland, OR, USA
| | - Steven T. Bailey
- Casey Eye Institute, Oregon Health & Science University, Portland, OR, USA
| | - Thomas S. Hwang
- Casey Eye Institute, Oregon Health & Science University, Portland, OR, USA
| | - Yali Jia
- Casey Eye Institute, Oregon Health & Science University, Portland, OR, USA
- Department of Biomedical Engineering, Oregon Health & Science University, Portland, OR, USA
| |
Collapse
|
25
|
Lichtenegger A, Salas M, Sing A, Duelk M, Licandro R, Gesperger J, Baumann B, Drexler W, Leitgeb RA. Reconstruction of visible light optical coherence tomography images retrieved from discontinuous spectral data using a conditional generative adversarial network. BIOMEDICAL OPTICS EXPRESS 2021; 12:6780-6795. [PMID: 34858680 PMCID: PMC8606123 DOI: 10.1364/boe.435124] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 06/24/2021] [Revised: 08/16/2021] [Accepted: 08/19/2021] [Indexed: 06/13/2023]
Abstract
Achieving high resolution in optical coherence tomography typically requires the continuous extension of the spectral bandwidth of the light source. This work demonstrates an alternative approach: combining two discrete spectral windows located in the visible spectrum with a trained conditional generative adversarial network (cGAN) to reconstruct a high-resolution image equivalent to that generated using a continuous spectral band. The cGAN was trained using OCT image pairs acquired with the continuous and discontinuous visible range spectra to learn the relation between low- and high-resolution data. The reconstruction performance was tested using 6000 B-scans of a layered phantom, micro-beads and ex-vivo mouse ear tissue. The resultant cGAN-generated images demonstrate an image quality and axial resolution which approaches that of the high-resolution system.
Collapse
Affiliation(s)
- Antonia Lichtenegger
- Center for Medical Physics and Biomedical Engineering, Medical University of Vienna, Austria
- Christian Doppler Laboratory for Innovative Optical Imaging and Its Translation to Medicine, Medical University of Vienna, Austria
- These authors contributed equally
| | - Matthias Salas
- Center for Medical Physics and Biomedical Engineering, Medical University of Vienna, Austria
- Christian Doppler Laboratory for Innovative Optical Imaging and Its Translation to Medicine, Medical University of Vienna, Austria
- These authors contributed equally
| | - Alexander Sing
- Center for Medical Physics and Biomedical Engineering, Medical University of Vienna, Austria
| | | | - Roxane Licandro
- Department of and Biomedical Imaging and Image-guided Therapy, Computational Imaging Research, Medical University of Vienna, Austria
- Institute of Visual Computing and Human-Centered Technology, Computer Vision Lab, TU Wien, Austria
| | - Johanna Gesperger
- Division of Neuropathology and Neurochemistry, Department of Neurology, Medical University of Vienna, Austria
| | - Bernhard Baumann
- Center for Medical Physics and Biomedical Engineering, Medical University of Vienna, Austria
| | - Wolfgang Drexler
- Center for Medical Physics and Biomedical Engineering, Medical University of Vienna, Austria
| | - Rainer A. Leitgeb
- Center for Medical Physics and Biomedical Engineering, Medical University of Vienna, Austria
- Christian Doppler Laboratory for Innovative Optical Imaging and Its Translation to Medicine, Medical University of Vienna, Austria
| |
Collapse
|
26
|
Hormel TT, Hwang TS, Bailey ST, Wilson DJ, Huang D, Jia Y. Artificial intelligence in OCT angiography. Prog Retin Eye Res 2021; 85:100965. [PMID: 33766775 PMCID: PMC8455727 DOI: 10.1016/j.preteyeres.2021.100965] [Citation(s) in RCA: 41] [Impact Index Per Article: 13.7] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/30/2020] [Revised: 03/09/2021] [Accepted: 03/15/2021] [Indexed: 12/21/2022]
Abstract
Optical coherence tomographic angiography (OCTA) is a non-invasive imaging modality that provides three-dimensional, information-rich vascular images. With numerous studies demonstrating unique capabilities in biomarker quantification, diagnosis, and monitoring, OCTA technology has seen rapid adoption in research and clinical settings. The value of OCTA imaging is significantly enhanced by image analysis tools that provide rapid and accurate quantification of vascular features and pathology. Today, the most powerful image analysis methods are based on artificial intelligence (AI). While AI encompasses a large variety of techniques, machine-learning-based, and especially deep-learning-based, image analysis provides accurate measurements in a variety of contexts, including different diseases and regions of the eye. Here, we discuss the principles of both OCTA and AI that make their combination capable of answering new questions. We also review contemporary applications of AI in OCTA, which include accurate detection of pathologies such as choroidal neovascularization, precise quantification of retinal perfusion, and reliable disease diagnosis.
Collapse
Affiliation(s)
- Tristan T Hormel
- Casey Eye Institute, Oregon Health & Science University, Portland, OR, 97239, USA
| | - Thomas S Hwang
- Casey Eye Institute, Oregon Health & Science University, Portland, OR, 97239, USA
| | - Steven T Bailey
- Casey Eye Institute, Oregon Health & Science University, Portland, OR, 97239, USA
| | - David J Wilson
- Casey Eye Institute, Oregon Health & Science University, Portland, OR, 97239, USA
| | - David Huang
- Casey Eye Institute, Oregon Health & Science University, Portland, OR, 97239, USA
| | - Yali Jia
- Casey Eye Institute, Oregon Health & Science University, Portland, OR, 97239, USA; Department of Biomedical Engineering, Oregon Health & Science University, Portland, OR, 97239, USA.
| |
Collapse
|
27
|
Cai S, Han IC, Scott AW. Artificial intelligence for improving sickle cell retinopathy diagnosis and management. Eye (Lond) 2021; 35:2675-2684. [PMID: 33958737 PMCID: PMC8452674 DOI: 10.1038/s41433-021-01556-4] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/01/2021] [Revised: 03/17/2021] [Accepted: 04/13/2021] [Indexed: 02/04/2023] Open
Abstract
Sickle cell retinopathy is often initially asymptomatic even in proliferative stages, but can progress to cause vision loss due to vitreous haemorrhages or tractional retinal detachments. Challenges with access and adherence to screening dilated fundus examinations, particularly in medically underserved areas where the burden of sickle cell disease is highest, highlight the need for novel approaches to screening for patients with vision-threatening sickle cell retinopathy. This article reviews the existing literature on and suggests future research directions for coupling artificial intelligence with multimodal retinal imaging to expand access to automated, accurate, imaging-based screening for sickle cell retinopathy. Given the variability in retinal specialist practice patterns with regards to monitoring and treatment of sickle cell retinopathy, we also discuss recent progress toward development of machine learning models that can quantitatively track disease progression over time. These artificial intelligence-based applications have great potential for informing evidence-based and resource-efficient clinical diagnosis and management of sickle cell retinopathy.
Collapse
Affiliation(s)
- Sophie Cai
- Retina Division, Duke Eye Center, Durham, NC, USA
| | - Ian C Han
- Institute for Vision Research, Department of Ophthalmology and Visual Sciences, University of Iowa Hospitals and Clinics, Iowa City, IA, USA
| | - Adrienne W Scott
- Retina Division, Wilmer Eye Institute, Johns Hopkins University School of Medicine and Hospital, Baltimore, MD, USA.
| |
Collapse
|
28
|
Yu TT, Ma D, Lo J, Ju MJ, Beg MF, Sarunic MV. Effect of optical coherence tomography and angiography sampling rate towards diabetic retinopathy severity classification. BIOMEDICAL OPTICS EXPRESS 2021; 12:6660-6673. [PMID: 34745763 PMCID: PMC8547994 DOI: 10.1364/boe.431992] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Received: 06/08/2021] [Revised: 08/12/2021] [Accepted: 08/16/2021] [Indexed: 06/13/2023]
Abstract
Optical coherence tomography (OCT) and OCT angiography (OCT-A) may benefit the screening of diabetic retinopathy (DR). This study investigated the effect of laterally subsampling OCT/OCT-A en face scans by up to a factor of 8 when using deep neural networks for automated referable DR classification. There was no significant difference in the classification performance across all evaluation metrics when subsampling up to a factor of 3, and only minimal differences up to a factor of 8. Our findings suggest that OCT/OCT-A can reduce the number of samples (and hence the acquisition time) for a volume for a given field of view on the retina that is acquired for rDR classification.
Collapse
Affiliation(s)
- Timothy T. Yu
- Engineering Science, Simon Fraser University, Burnaby BC V5A1S6, Canada
| | - Da Ma
- Engineering Science, Simon Fraser University, Burnaby BC V5A1S6, Canada
| | - Julian Lo
- Engineering Science, Simon Fraser University, Burnaby BC V5A1S6, Canada
| | - Myeong Jin Ju
- Dept. of Ophthalmology and Visual Sciences, University of British Columbia, Vancouver, BC, V5Z 3N9, Canada
- School of Biomedical Engineering, University of British Columbia, Vancouver, BC, V5Z 3N9, Canada
| | - Mirza Faisal Beg
- Engineering Science, Simon Fraser University, Burnaby BC V5A1S6, Canada
| | | |
Collapse
|
29
|
Ren S, Shen X, Xu J, Li L, Qiu H, Jia H, Wu X, Chen D, Zhao S, Yu B, Gu Y, Dong F. Imaging depth adaptive resolution enhancement for optical coherence tomography via deep neural network with external attention. Phys Med Biol 2021; 66. [PMID: 34464947 DOI: 10.1088/1361-6560/ac2267] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/19/2021] [Accepted: 08/31/2021] [Indexed: 11/11/2022]
Abstract
Optical coherence tomography (OCT) is a promising non-invasive imaging technique that owns many biomedical applications. In this paper, a deep neural network is proposed for enhancing the spatial resolution of OCTen faceimages. Different from the previous reports, the proposed can recover high-resolutionen faceimages from low-resolutionen faceimages at arbitrary imaging depth. This kind of imaging depth adaptive resolution enhancement is achieved through an external attention mechanism, which takes advantage of morphological similarity between the arbitrary-depth and full-depthen faceimages. Firstly, the deep feature maps are extracted by a feature extraction network from the arbitrary-depth and full-depthen faceimages. Secondly, the morphological similarity between the deep feature maps is extracted and utilized to emphasize the features strongly correlated to the vessel structures by using the external attention network. Finally, the SR image is recovered from the enhanced feature map through an up-sampling network. The proposed network is tested on a clinical skin OCT data set and an open-access retinal OCT dataset. The results show that the proposed external attention mechanism can suppress invalid features and enhance significant features in our tasks. For all tests, the proposed SR network outperformed the traditional image interpolation method, e.g. bi-cubic method, and the state-of-the-art image super-resolution networks, e.g. enhanced deep super-resolution network, residual channel attention network, and second-order attention network. The proposed method may increase the quantitative clinical assessment of micro-vascular diseases which is limited by OCT imaging device resolution.
Collapse
Affiliation(s)
- Shangjie Ren
- Tianjin Key Laboratory of Process Measurement and Control, School of Electrical and Information Engineering, Tianjin University, Tianjin, 300072, People's Republic of China
| | - Xiongri Shen
- Tianjin Key Laboratory of Process Measurement and Control, School of Electrical and Information Engineering, Tianjin University, Tianjin, 300072, People's Republic of China
| | - Jingjiang Xu
- School of Physics and Optoelectronic Engineering, Foshan University, Foshan, 528000, People's Republic of China
| | - Liang Li
- College of Intelligence and Computing, Tianjin University, Tianjin, 300072, People's Republic of China
| | - Haixia Qiu
- Department of Laser Medicine, the First Medical Centre, Chinese PLA General Hospital, Beijing, 100853, People's Republic of China
| | - Haibo Jia
- Department of Cardiology, The 2nd Affiliated Hospital of Harbin Medical University, Harbin, 150081, People's Republic of China.,The Key Laboratory of Myocardial Ischemia, Chinese Ministry of Education, Harbin, 150081, People's Republic of China
| | - Xining Wu
- Tianjin Horimed Technology Co., Ltd., Tianjin, 300308, People's Republic of China
| | - Defu Chen
- Institute of Engineering Medicine, Beijing Institute of Technology, Beijing, 100081, People's Republic of China
| | - Shiyong Zhao
- Tianjin Horimed Technology Co., Ltd., Tianjin, 300308, People's Republic of China
| | - Bo Yu
- Department of Cardiology, The 2nd Affiliated Hospital of Harbin Medical University, Harbin, 150081, People's Republic of China.,The Key Laboratory of Myocardial Ischemia, Chinese Ministry of Education, Harbin, 150081, People's Republic of China
| | - Ying Gu
- Department of Laser Medicine, the First Medical Centre, Chinese PLA General Hospital, Beijing, 100853, People's Republic of China.,Precision Laser Medical Diagnosis and Treatment Innovation Unit, Chinese Academy of Medical Sciences, Beijing, 100000, People's Republic of China
| | - Feng Dong
- Tianjin Key Laboratory of Process Measurement and Control, School of Electrical and Information Engineering, Tianjin University, Tianjin, 300072, People's Republic of China
| |
Collapse
|
30
|
Guo Y, Hormel TT, Pi S, Wei X, Gao M, Morrison JC, Jia Y. An end-to-end network for segmenting the vasculature of three retinal capillary plexuses from OCT angiographic volumes. BIOMEDICAL OPTICS EXPRESS 2021; 12:4889-4900. [PMID: 34513231 PMCID: PMC8407822 DOI: 10.1364/boe.431888] [Citation(s) in RCA: 4] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 05/19/2021] [Revised: 06/28/2021] [Accepted: 07/06/2021] [Indexed: 06/13/2023]
Abstract
The segmentation of en face retinal capillary angiograms from volumetric optical coherence tomographic angiography (OCTA) usually relies on retinal layer segmentation, which is time-consuming and error-prone. In this study, we developed a deep-learning-based method to segment vessels in the superficial vascular plexus (SVP), intermediate capillary plexus (ICP), and deep capillary plexus (DCP) directly from volumetric OCTA data. The method contains a three-dimensional convolutional neural network (CNN) for extracting distinct retinal layers, a custom projection module to generate three vascular plexuses from OCTA data, and three parallel CNNs to segment vasculature. Experimental results on OCTA data from rat eyes demonstrated the feasibility of the proposed method. This end-to-end network has the potential to simplify OCTA data processing on retinal vasculature segmentation. The main contribution of this study is that we propose a custom projection module to connect retinal layer segmentation and vasculature segmentation modules and automatically convert data from three to two dimensions, thus establishing an end-to-end method to segment three retinal capillary plexuses from volumetric OCTA without any human intervention.
Collapse
Affiliation(s)
- Yukun Guo
- Casey Eye Institute, Oregon Health & Science University, Portland, OR 97239, USA
| | - Tristan T. Hormel
- Casey Eye Institute, Oregon Health & Science University, Portland, OR 97239, USA
| | - Shaohua Pi
- Casey Eye Institute, Oregon Health & Science University, Portland, OR 97239, USA
| | - Xiang Wei
- Casey Eye Institute, Oregon Health & Science University, Portland, OR 97239, USA
- Department of Biomedical Engineering, Oregon Health & Science University, Portland, OR 97239, USA
| | - Min Gao
- Casey Eye Institute, Oregon Health & Science University, Portland, OR 97239, USA
| | - John C. Morrison
- Casey Eye Institute, Oregon Health & Science University, Portland, OR 97239, USA
| | - Yali Jia
- Casey Eye Institute, Oregon Health & Science University, Portland, OR 97239, USA
- Department of Biomedical Engineering, Oregon Health & Science University, Portland, OR 97239, USA
| |
Collapse
|
31
|
Le D, Son T, Yao X. Machine learning in optical coherence tomography angiography. Exp Biol Med (Maywood) 2021; 246:2170-2183. [PMID: 34279136 DOI: 10.1177/15353702211026581] [Citation(s) in RCA: 7] [Impact Index Per Article: 2.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/16/2022] Open
Abstract
Optical coherence tomography angiography (OCTA) offers a noninvasive label-free solution for imaging retinal vasculatures at the capillary level resolution. In principle, improved resolution implies a better chance to reveal subtle microvascular distortions associated with eye diseases that are asymptomatic in early stages. However, massive screening requires experienced clinicians to manually examine retinal images, which may result in human error and hinder objective screening. Recently, quantitative OCTA features have been developed to standardize and document retinal vascular changes. The feasibility of using quantitative OCTA features for machine learning classification of different retinopathies has been demonstrated. Deep learning-based applications have also been explored for automatic OCTA image analysis and disease classification. In this article, we summarize recent developments of quantitative OCTA features, machine learning image analysis, and classification.
Collapse
Affiliation(s)
- David Le
- Department of Bioengineering, 14681University of Illinois at Chicago, Chicago, IL 60607, USA
| | - Taeyoon Son
- Department of Bioengineering, 14681University of Illinois at Chicago, Chicago, IL 60607, USA
| | - Xincheng Yao
- Department of Bioengineering, 14681University of Illinois at Chicago, Chicago, IL 60607, USA.,Department of Ophthalmology and Visual Sciences, University of Illinois at Chicago, Chicago, IL 60612, USA
| |
Collapse
|
32
|
Ran A, Cheung CY. Deep Learning-Based Optical Coherence Tomography and Optical Coherence Tomography Angiography Image Analysis: An Updated Summary. Asia Pac J Ophthalmol (Phila) 2021; 10:253-260. [PMID: 34383717 DOI: 10.1097/apo.0000000000000405] [Citation(s) in RCA: 8] [Impact Index Per Article: 2.7] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/14/2022] Open
Abstract
ABSTRACT Deep learning (DL) is a subset of artificial intelligence based on deep neural networks. It has made remarkable breakthroughs in medical imaging, particularly for image classification and pattern recognition. In ophthalmology, there are rising interests in applying DL methods to analyze optical coherence tomography (OCT) and optical coherence tomography angiography (OCTA) images. Studies showed that OCT and OCTA image evaluation by DL algorithms achieved good performance for disease detection, prognosis prediction, and image quality control, suggesting that the incorporation of DL technology could potentially enhance the accuracy of disease evaluation and the efficiency of clinical workflow. However, substantial issues, such as small training sample size, data preprocessing standardization, model robustness, results explanation, and performance cross-validation, are yet to be tackled before deploying these DL models in real-time clinics. This review summarized recent studies on DL-based image analysis models for OCT and OCTA images and discussed the potential challenges of clinical deployment and future research directions.
Collapse
Affiliation(s)
- Anran Ran
- Department of Ophthalmology and Visual Sciences, the Chinese University of Hong Kong, Hong Kong SAR
| | | |
Collapse
|
33
|
Guo Y, Hormel TT, Xiong H, Wang J, Hwang TS, Jia Y. Automated Segmentation of Retinal Fluid Volumes From Structural and Angiographic Optical Coherence Tomography Using Deep Learning. Transl Vis Sci Technol 2020; 9:54. [PMID: 33110708 PMCID: PMC7552937 DOI: 10.1167/tvst.9.2.54] [Citation(s) in RCA: 30] [Impact Index Per Article: 7.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/25/2020] [Accepted: 09/07/2020] [Indexed: 01/08/2023] Open
Abstract
Purpose We proposed a deep convolutional neural network (CNN), named Retinal Fluid Segmentation Network (ReF-Net), to segment retinal fluid in diabetic macular edema (DME) in optical coherence tomography (OCT) volumes. Methods The 3- × 3-mm OCT scans were acquired on one eye by a 70-kHz OCT commercial AngioVue system (RTVue-XR; Optovue, Inc., Fremont, CA, USA) from 51 participants in a clinical diabetic retinopathy (DR) study (45 with retinal edema and six healthy controls, age 61.3 ± 10.1 (mean ± SD), 33% female, and all DR cases were diagnosed as severe NPDR or PDR). A CNN with U-Net-like architecture was constructed to detect and segment the retinal fluid. Cross-sectional OCT and angiography (OCTA) scans were used for training and testing ReF-Net. The effect of including OCTA data for retinal fluid segmentation was investigated in this study. Volumetric retinal fluid can be constructed using the output of ReF-Net. Area-under-receiver-operating-characteristic-curve, intersection-over-union (IoU), and F1-score were calculated to evaluate the performance of ReF-Net. Results ReF-Net shows high accuracy (F1 = 0.864 ± 0.084) in retinal fluid segmentation. The performance can be further improved (F1 = 0.892 ± 0.038) by including information from both OCTA and structural OCT. ReF-Net also shows strong robustness to shadow artifacts. Volumetric retinal fluid can provide more comprehensive information than the two-dimensional (2D) area, whether cross-sectional or en face projections. Conclusions A deep-learning-based method can accurately segment retinal fluid volumetrically on OCT/OCTA scans with strong robustness to shadow artifacts. OCTA data can improve retinal fluid segmentation. Volumetric representations of retinal fluid are superior to 2D projections. Translational Relevance Using a deep learning method to segment retinal fluid volumetrically has the potential to improve the diagnostic accuracy of diabetic macular edema by OCT systems.
Collapse
Affiliation(s)
- Yukun Guo
- Casey Eye Institute, Oregon Health & Science University, Portland, OR, USA
| | - Tristan T Hormel
- Casey Eye Institute, Oregon Health & Science University, Portland, OR, USA
| | - Honglian Xiong
- Casey Eye Institute, Oregon Health & Science University, Portland, OR, USA.,School of Physics and Optoelectronic Engineering, Foshan University, Foshan, Guangdong, China
| | - Jie Wang
- Casey Eye Institute, Oregon Health & Science University, Portland, OR, USA.,Department of Biomedical Engineering, Oregon Health & Science University, Portland, OR, USA
| | - Thomas S Hwang
- Casey Eye Institute, Oregon Health & Science University, Portland, OR, USA
| | - Yali Jia
- Casey Eye Institute, Oregon Health & Science University, Portland, OR, USA.,Department of Biomedical Engineering, Oregon Health & Science University, Portland, OR, USA
| |
Collapse
|