1
|
Yu X, Li M, Ge C, Yuan M, Liu L, Mo J, Shum PP, Chen J. Loss-balanced parallel decoding network for retinal fluid segmentation in OCT. Comput Biol Med 2023; 165:107319. [PMID: 37611427 DOI: 10.1016/j.compbiomed.2023.107319] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/08/2023] [Revised: 07/12/2023] [Accepted: 08/07/2023] [Indexed: 08/25/2023]
Abstract
As a leading cause of blindness worldwide, macular edema (ME) is mainly determined by sub-retinal fluid (SRF), intraretinal fluid (IRF), and pigment epithelial detachment (PED) accumulation, and therefore, the characterization of SRF, IRF, and PED, which is also known as ME segmentation, has become a crucial issue in ophthalmology. Due to the subjective and time-consuming nature of ME segmentation in retinal optical coherence tomography (OCT) images, automatic computer-aided systems are highly desired in clinical practice. This paper proposes a novel loss-balanced parallel decoding network, namely PadNet, for ME segmentation. Specifically, PadNet mainly consists of an encoder and three parallel decoder modules, which serve as segmentation, contour, and diffusion branches, and they are employed to extract the ME's characteristics, the contour area features, and to expand the ME area from the center to edge, respectively. A new loss-balanced joint-loss function with three components corresponding to each of the three parallel decoding branches is also devised for training. Experiments are conducted with three public datasets to verify the effectiveness of PadNet, and the performances of PadNet are compared with those of five state-of-the-art methods. Results show that PadNet improves ME segmentation accuracy by 8.1%, 11.1%, 0.6%, 1.4% and 8.3%, as compared with UNet, sASPP, MsTGANet, YNet, RetiFluidNet, respectively, which convincingly demonstrates that the proposed PadNet is robust and effective in ME segmentation in different cases.
Collapse
Affiliation(s)
- Xiaojun Yu
- School of Automation, Northwestern Polytechnical University, Xi'an, 710072, Shaanxi, China; Shenzhen Research Institute of Northwestern Polytechnical University, Shenzhen, 518057, Guangdong, China.
| | - Mingshuai Li
- School of Automation, Northwestern Polytechnical University, Xi'an, 710072, Shaanxi, China.
| | - Chenkun Ge
- School of Automation, Northwestern Polytechnical University, Xi'an, 710072, Shaanxi, China.
| | - Miao Yuan
- School of Automation, Northwestern Polytechnical University, Xi'an, 710072, Shaanxi, China.
| | - Linbo Liu
- School of Electrical and Electronic Engineering, Nanyang Technological University, 639798, Singapore.
| | - Jianhua Mo
- School of Electronics and Information Engineering, Soochow University, Suzhou 215006, China.
| | - Perry Ping Shum
- Department of Electrical and Electronic Engineering, Southern University of Science and Technology, Shenzhen, 518055, China.
| | - Jinna Chen
- Department of Electrical and Electronic Engineering, Southern University of Science and Technology, Shenzhen, 518055, China.
| |
Collapse
|
2
|
Vousten V, Moradi H, Wu Z, Boctor EM, Salcudean SE. Laser diode photoacoustic point source detection: machine learning-based denoising and reconstruction. OPTICS EXPRESS 2023; 31:13895-13910. [PMID: 37157265 DOI: 10.1364/oe.483892] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 05/10/2023]
Abstract
A new development in photoacoustic (PA) imaging has been the use of compact, portable and low-cost laser diodes (LDs), but LD-based PA imaging suffers from low signal intensity recorded by the conventional transducers. A common method to improve signal strength is temporal averaging, which reduces frame rate and increases laser exposure to patients. To tackle this problem, we propose a deep learning method that will denoise point source PA radio-frequency (RF) data before beamforming with a very few frames, even one. We also present a deep learning method to automatically reconstruct point sources from noisy pre-beamformed data. Finally, we employ a strategy of combined denoising and reconstruction, which can supplement the reconstruction algorithm for very low signal-to-noise ratio inputs.
Collapse
|
3
|
Shahid AR, Yan H. SqueezExpNet: Dual-stage convolutional neural network for accurate facial expression recognition with attention mechanism. Knowl Based Syst 2023. [DOI: 10.1016/j.knosys.2023.110451] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 03/28/2023]
|
4
|
Loo J, Teo KY, Vyas CH, Jordan-Yu JMN, Juhari AB, Jaffe GJ, Cheung CMG, Farsiu S. Joint multimodal deep learning-based automatic segmentation of ICGA and OCT images for assessment of PCV biomarkers. OPHTHALMOLOGY SCIENCE 2023; 3:100292. [PMID: 37025946 PMCID: PMC10070921 DOI: 10.1016/j.xops.2023.100292] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Received: 10/01/2022] [Revised: 01/08/2023] [Accepted: 02/17/2023] [Indexed: 02/26/2023]
Abstract
Purpose To develop a fully-automatic hybrid algorithm to jointly segment and quantify biomarkers of polypoidal choroidal vasculopathy (PCV) on indocyanine green angiography (ICGA) and spectral domain-OCT (SD-OCT) images. Design Evaluation of diagnostic test or technology. Participants Seventy-two participants with PCV enrolled in clinical studies at Singapore National Eye Center. Methods The dataset consisted of 2-dimensional (2-D) ICGA and 3-dimensional (3-D) SD-OCT images which were spatially registered and manually segmented by clinicians. A deep learning-based hybrid algorithm called PCV-Net was developed for automatic joint segmentation of biomarkers. The PCV-Net consisted of a 2-D segmentation branch for ICGA and 3-D segmentation branch for SD-OCT. We developed fusion attention modules to connect the 2-D and 3-D branches for effective use of the spatial correspondence between the imaging modalities by sharing learned features. We also used self-supervised pretraining and ensembling to further enhance the performance of the algorithm without the need for additional datasets. We compared the proposed PCV-Net to several alternative model variants. Main Outcome Measures The PCV-Net was evaluated based on the Dice similarity coefficient (DSC) of the segmentations and the Pearson's correlation and absolute difference of the clinical measurements obtained from the segmentations. Manual grading was used as the gold standard. Results The PCV-Net showed good performance compared to manual grading and alternative model variants based on both quantitative and qualitative analyses. Compared to the baseline variant, PCV-Net improved the DSC by 0.04 to 0.43 across the different biomarkers, increased the correlations, and decreased the absolute differences of clinical measurements of interest. Specifically, the largest average (mean ± standard error) DSC improvement was for intraretinal fluid, from 0.02 ± 0.00 (baseline variant) to 0.45 ± 0.06 (PCV-Net). In general, improving trends were observed across the model variants as more technical specifications were added, demonstrating the importance of each aspect of the proposed method. Conclusion The PCV-Net has the potential to aid clinicians in disease assessment and research to improve clinical understanding and management of PCV. Financial Disclosures Proprietary or commercial disclosure may be found after the references.
Collapse
|
5
|
He X, Zhong Z, Fang L, He M, Sebe N. Structure-Guided Cross-Attention Network for Cross-Domain OCT Fluid Segmentation. IEEE TRANSACTIONS ON IMAGE PROCESSING : A PUBLICATION OF THE IEEE SIGNAL PROCESSING SOCIETY 2022; PP:309-320. [PMID: 37015552 DOI: 10.1109/tip.2022.3228163] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/19/2023]
Abstract
Accurate retinal fluid segmentation on Optical Coherence Tomography (OCT) images plays an important role in diagnosing and treating various eye diseases. The art deep models have shown promising performance on OCT image segmentation given pixel-wise annotated training data. However, the learned model will achieve poor performance on OCT images that are obtained from different devices (domains) due to the domain shift issue. This problem largely limits the real-world application of OCT image segmentation since the types of devices usually are different in each hospital. In this paper, we study the task of cross-domain OCT fluid segmentation, where we are given a labeled dataset of the source device (domain) and an unlabeled dataset of the target device (domain). The goal is to learn a model that can perform well on the target domain. To solve this problem, in this paper, we propose a novel Structure-guided Cross-Attention Network (SCAN), which leverages the retinal layer structure to facilitate domain alignment. Our SCAN is inspired by the fact that the retinal layer structure is robust to domains and can reflect regions that are important to fluid segmentation. In light of this, we build our SCAN in a multi-task manner by jointly learning the retinal structure prediction and fluid segmentation. To exploit the mutual benefit between layer structure and fluid segmentation, we further introduce a cross-attention module to measure the correlation between the layer-specific feature and the fluid-specific feature encouraging the model to concentrate on highly relative regions during domain alignment. Moreover, an adaptation difficulty map is evaluated based on the retinal structure predictions from different domains, which enforces the model focus on hard regions during structure-aware adversarial learning. Extensive experiments on the three domains of the RETOUCH dataset demonstrate the effectiveness of the proposed method and show that our approach produces state-of-the-art performance on cross-domain OCT fluid segmentation.
Collapse
|
6
|
Wongchaisuwat P, Thamphithak R, Jitpukdee P, Wongchaisuwat N. Application of Deep Learning for Automated Detection of Polypoidal Choroidal Vasculopathy in Spectral Domain Optical Coherence Tomography. Transl Vis Sci Technol 2022; 11:16. [PMID: 36219163 PMCID: PMC9580222 DOI: 10.1167/tvst.11.10.16] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/25/2022] Open
Abstract
Objective To develop an automated polypoidal choroidal vasculopathy (PCV) screening model to distinguish PCV from wet age-related macular degeneration (wet AMD). Methods A retrospective review of spectral domain optical coherence tomography (SD-OCT) images was undertaken. The included SD-OCT images were classified into two distinct categories (PCV or wet AMD) prior to the development of the PCV screening model. The automated detection of PCV using the developed model was compared with the results of gold-standard fundus fluorescein angiography and indocyanine green (FFA + ICG) angiography. A framework of SHapley Additive exPlanations was used to interpret the results from the model. Results A total of 2334 SD-OCT images were enrolled for training purposes, and an additional 1171 SD-OCT images were used for external validation. The ResNet attention model yielded superior performance with average area under the curve values of 0.8 and 0.81 for the training and external validation data sets, respectively. The sensitivity/specificity calculated at a patient level was 100%/60% and 85%/71% for the training and external validation data sets, respectively. Conclusions A conventional FFA + ICG investigation to differentiate PCV from wet AMD requires intense health care resources and adversely affects patients. A deep learning algorithm is proposed to automatically distinguish PCV from wet AMD. The developed algorithm exhibited promising performance for further development into an alternative PCV screening tool. Enhancement of the model's performance with additional data is needed prior to implementation of this diagnostic tool in real-world clinical practice. The invisibility of disease signs within SD-OCT images is the main limitation of the proposed model. Translational Relevance Basic research of deep learning algorithms was applied to differentiate PCV from wet AMD based on OCT images, benefiting a diagnosis process and minimizing a risk of ICG angiogram.
Collapse
Affiliation(s)
- Papis Wongchaisuwat
- Department of Industrial Engineering, Faculty of Engineering, Kasetsart University, Bangkok, Thailand
| | - Ranida Thamphithak
- Department of Ophthalmology, Faculty of Medicine Siriraj Hospital, Mahidol University, Bangkok, Thailand
| | - Peerakarn Jitpukdee
- Department of Industrial Engineering, Faculty of Engineering, Kasetsart University, Bangkok, Thailand
| | - Nida Wongchaisuwat
- Department of Ophthalmology, Faculty of Medicine Siriraj Hospital, Mahidol University, Bangkok, Thailand
| |
Collapse
|
7
|
|
8
|
Song X, Xu Q, Li H, Fan Q, Zheng Y, Zhang Q, Chu C, Zhang Z, Yuan C, Ning M, Bian C, Ma K, Qu Y. Automatic quantification of retinal photoreceptor integrity to predict persistent disease activity in neovascular age-related macular degeneration using deep learning. Front Neurosci 2022; 16:952735. [PMID: 36061600 PMCID: PMC9434346 DOI: 10.3389/fnins.2022.952735] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/09/2022] [Accepted: 07/26/2022] [Indexed: 11/13/2022] Open
Abstract
Purpose Using deep learning (DL)-based technique, we identify risk factors and create a prediction model for refractory neovascular age-related macular degeneration (nAMD) characterized by persistent disease activity (PDA) in spectral domain optical coherence tomography (SD-OCT) images. Materials and methods A total of 671 typical B-scans were collected from 186 eyes of 186 patients with nAMD. Spectral domain optical coherence tomography images were analyzed using a classification convolutional neural network (CNN) and a fully convolutional network (FCN) algorithm to extract six features involved in nAMD, including ellipsoid zone (EZ), external limiting membrane (ELM), intraretinal fluid (IRF), subretinal fluid (SRF), pigment epithelium detachment (PED), and subretinal hyperreflective material (SHRM). Random forest models were probed to predict 1-year disease activity (stable, PDA, and cured) based on the quantitative features computed from automated segmentation and evaluated with cross-validation. Results The algorithm to segment six SD-OCT features achieved the mean accuracy of 0.930 (95% CI: 0.916–0.943), dice coefficients of 0.873 (95% CI: 0.847–0.899), a sensitivity of 0.873 (95% CI: 0.844–0.910), and a specificity of 0.922 (95% CI: 0.905–0.940). The six-metric model including EZ and ELM achieved the optimal performance to predict 1-year disease activity, with an area under the receiver operating characteristic (ROC) curve (AUC) of 0.980, the accuracy of 0.930, the sensitivity of 0.920, and the specificity of 0.962. The integrity of EZ and ELM significantly improved the performance of the six-metric model than that of the four-metric model. Conclusion The prediction model reveals the potential to predict PDA in nAMD eyes. The integrity of EZ and ELM constituted the strongest predictive factor for PDA in nAMD eyes in real-world clinical practice. The results of this study are a significant step toward image-guided prediction of long-term disease activity in the management of nAMD and highlight the importance of the automatic identification of photoreceptor layers.
Collapse
Affiliation(s)
- Xian Song
- Department of Geriatrics, Qilu Hospital of Shandong University, Jinan, China
| | - Qian Xu
- Department of Geriatrics, Qilu Hospital of Shandong University, Jinan, China
| | - Haiming Li
- Department of Geriatrics, Qilu Hospital of Shandong University, Jinan, China
| | - Qian Fan
- Department of Geriatrics, Qilu Hospital of Shandong University, Jinan, China
| | | | | | | | | | | | | | - Cheng Bian
- Xiaohe Healthcare, ByteDance, Guangzhou, China
| | - Kai Ma
- Tencent Healthcare, Shenzhen, China
| | - Yi Qu
- Department of Geriatrics, Qilu Hospital of Shandong University, Jinan, China
- *Correspondence: Yi Qu,
| |
Collapse
|
9
|
Qi X, Hu J, Zhang L, Bai S, Yi Z. Automated Segmentation of the Clinical Target Volume in the Planning CT for Breast Cancer Using Deep Neural Networks. IEEE TRANSACTIONS ON CYBERNETICS 2022; 52:3446-3456. [PMID: 32833659 DOI: 10.1109/tcyb.2020.3012186] [Citation(s) in RCA: 4] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/11/2023]
Abstract
3-D radiotherapy is an effective treatment modality for breast cancer. In 3-D radiotherapy, delineation of the clinical target volume (CTV) is an essential step in the establishment of treatment plans. However, manual delineation is subjective and time consuming. In this study, we propose an automated segmentation model based on deep neural networks for the breast cancer CTV in planning computed tomography (CT). Our model is composed of three stages that work in a cascade manner, making it applicable to real-world scenarios. The first stage determines which slices contain CTVs, as not all CT slices include breast lesions. The second stage detects the region of the human body in an entire CT slice, eliminating boundary areas, which may have side effects for the segmentation of the CTV. The third stage delineates the CTV. To permit the network to focus on the breast mass in the slice, a novel dynamically strided convolution operation, which shows better performance than standard convolution, is proposed. To train and evaluate the model, a large dataset containing 455 cases and 50 425 CT slices is constructed. The proposed model achieves an average dice similarity coefficient (DSC) of 0.802 and 0.801 for right-0 and left-sided breast, respectively. Our method shows superior performance to that of previous state-of-the-art approaches.
Collapse
|
10
|
He X, Fang L, Tan M, Chen X. Intra- and Inter-Slice Contrastive Learning for Point Supervised OCT Fluid Segmentation. IEEE TRANSACTIONS ON IMAGE PROCESSING : A PUBLICATION OF THE IEEE SIGNAL PROCESSING SOCIETY 2022; 31:1870-1881. [PMID: 35139015 DOI: 10.1109/tip.2022.3148814] [Citation(s) in RCA: 13] [Impact Index Per Article: 6.5] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/14/2023]
Abstract
OCT fluid segmentation is a crucial task for diagnosis and therapy in ophthalmology. The current convolutional neural networks (CNNs) supervised by pixel-wise annotated masks achieve great success in OCT fluid segmentation. However, requiring pixel-wise masks from OCT images is time-consuming, expensive and expertise needed. This paper proposes an Intra- and inter-Slice Contrastive Learning Network (ISCLNet) for OCT fluid segmentation with only point supervision. Our ISCLNet learns visual representation by designing contrastive tasks that exploit the inherent similarity or dissimilarity from unlabeled OCT data. Specifically, we propose an intra-slice contrastive learning strategy to leverage the fluid-background similarity and the retinal layer-background dissimilarity. Moreover, we construct an inter-slice contrastive learning architecture to learn the similarity of adjacent OCT slices from one OCT volume. Finally, an end-to-end model combining intra- and inter-slice contrastive learning processes learns to segment fluid under the point supervision. The experimental results on two public OCT fluid segmentation datasets (i.e., AI Challenger and RETOUCH) demonstrate that the ISCLNet bridges the gap between fully-supervised and weakly-supervised OCT fluid segmentation and outperforms other well-known point-supervised segmentation methods.
Collapse
|
11
|
Shigueoka LS, Jammal AA, Medeiros FA, Costa VP. Artificial Intelligence in Ophthalmology. Artif Intell Med 2022. [DOI: 10.1007/978-3-030-64573-1_201] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/01/2022]
|
12
|
Shigueoka LS, Jammal AA, Medeiros FA, Costa VP. Artificial Intelligence in Ophthalmology. Artif Intell Med 2021. [DOI: 10.1007/978-3-030-58080-3_201-1] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/28/2022]
|
13
|
Kugelman J, Alonso-Caneiro D, Chen Y, Arunachalam S, Huang D, Vallis N, Collins MJ, Chen FK. Retinal Boundary Segmentation in Stargardt Disease Optical Coherence Tomography Images Using Automated Deep Learning. Transl Vis Sci Technol 2020; 9:12. [PMID: 33133774 PMCID: PMC7581491 DOI: 10.1167/tvst.9.11.12] [Citation(s) in RCA: 16] [Impact Index Per Article: 4.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/26/2020] [Accepted: 09/11/2020] [Indexed: 12/13/2022] Open
Abstract
Purpose To use a deep learning model to develop a fully automated method (fully semantic network and graph search [FS-GS]) of retinal segmentation for optical coherence tomography (OCT) images from patients with Stargardt disease. Methods Eighty-seven manually segmented (ground truth) OCT volume scan sets (5171 B-scans) from 22 patients with Stargardt disease were used for training, validation and testing of a novel retinal boundary detection approach (FS-GS) that combines a fully semantic deep learning segmentation method, which generates a per-pixel class prediction map with a graph-search method to extract retinal boundary positions. The performance was evaluated using the mean absolute boundary error and the differences in two clinical metrics (retinal thickness and volume) compared with the ground truth. The performance of a separate deep learning method and two publicly available software algorithms were also evaluated against the ground truth. Results FS-GS showed an excellent agreement with the ground truth, with a boundary mean absolute error of 0.23 and 1.12 pixels for the internal limiting membrane and the base of retinal pigment epithelium or Bruch's membrane, respectively. The mean difference in thickness and volume across the central 6 mm zone were 2.10 µm and 0.059 mm3. The performance of the proposed method was more accurate and consistent than the publicly available OCTExplorer and AURA tools. Conclusions The FS-GS method delivers good performance in segmentation of OCT images of pathologic retina in Stargardt disease. Translational Relevance Deep learning models can provide a robust method for retinal segmentation and support a high-throughput analysis pipeline for measuring retinal thickness and volume in Stargardt disease.
Collapse
Affiliation(s)
- Jason Kugelman
- Queensland University of Technology (QUT), Contact Lens and Visual Optics Laboratory, Centre for Vision and Eye Research, School of Optometry and Vision Science, Queensland, Australia
| | - David Alonso-Caneiro
- Queensland University of Technology (QUT), Contact Lens and Visual Optics Laboratory, Centre for Vision and Eye Research, School of Optometry and Vision Science, Queensland, Australia.,Centre for Ophthalmology and Visual Science (incorporating Lions Eye Institute), The University of Western Australia, Perth, Western Australia, Australia
| | - Yi Chen
- Centre for Ophthalmology and Visual Science (incorporating Lions Eye Institute), The University of Western Australia, Perth, Western Australia, Australia
| | - Sukanya Arunachalam
- Centre for Ophthalmology and Visual Science (incorporating Lions Eye Institute), The University of Western Australia, Perth, Western Australia, Australia
| | - Di Huang
- Centre for Ophthalmology and Visual Science (incorporating Lions Eye Institute), The University of Western Australia, Perth, Western Australia, Australia.,Centre for Molecular Medicine and Innovative Therapeutics, Murdoch University, Murdoch, Western Australia, Australia.,Centre for Neuromuscular and Neurological Disorders, The University of Western Australia and Perron Institute for Neurological and Translational Science, Nedlands, Western Australia, Australia
| | - Natasha Vallis
- Centre for Ophthalmology and Visual Science (incorporating Lions Eye Institute), The University of Western Australia, Perth, Western Australia, Australia
| | - Michael J Collins
- Queensland University of Technology (QUT), Contact Lens and Visual Optics Laboratory, Centre for Vision and Eye Research, School of Optometry and Vision Science, Queensland, Australia
| | - Fred K Chen
- Centre for Ophthalmology and Visual Science (incorporating Lions Eye Institute), The University of Western Australia, Perth, Western Australia, Australia.,Department of Ophthalmology, Royal Perth Hospital, Perth, Western Australia, Australia.,Department of Ophthalmology, Perth Children's Hospital, Nedlands, Western Australia, Australia
| |
Collapse
|
14
|
Moraru AD, Costin D, Moraru RL, Branisteanu DC. Artificial intelligence and deep learning in ophthalmology - present and future (Review). Exp Ther Med 2020; 20:3469-3473. [PMID: 32905155 PMCID: PMC7465350 DOI: 10.3892/etm.2020.9118] [Citation(s) in RCA: 16] [Impact Index Per Article: 4.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/29/2020] [Accepted: 06/30/2020] [Indexed: 02/06/2023] Open
Abstract
Since its introduction in 1959, artificial intelligence technology has evolved rapidly and helped benefit research, industries and medicine. Deep learning, as a process of artificial intelligence (AI) is used in ophthalmology for data analysis, segmentation, automated diagnosis and possible outcome predictions. The association of deep learning and optical coherence tomography (OCT) technologies has proven reliable for the detection of retinal diseases and improving the diagnostic performance of the eye's posterior segment diseases. This review explored the possibility of implementing and using AI in establishing the diagnosis of retinal disorders. The benefits and limitations of AI in the field of retinal disease medical management were investigated by analyzing the most recent literature data. Furthermore, the future trends of AI involvement in ophthalmology were analyzed, as AI will be part of the decision-making regarding the scientific investigation, diagnosis and therapeutic management.
Collapse
Affiliation(s)
- Andreea Dana Moraru
- Department of Ophthalmology, 'Grigore T. Popa' University of Medicine and Pharmacy, 700115 Iaşi, Romania.,Department of Ophthalmology, 'N. Oblu' Clinical Hospital, 700309 Iași, Romania
| | - Danut Costin
- Department of Ophthalmology, 'Grigore T. Popa' University of Medicine and Pharmacy, 700115 Iaşi, Romania.,Department of Ophthalmology, 'N. Oblu' Clinical Hospital, 700309 Iași, Romania
| | - Radu Lucian Moraru
- Department of Otorhinolaryngology, Transmed Expert, 700011 Iaşi; 4'Retina Center' Eye Clinic, 700126 Iaşi, Romania
| | - Daniel Constantin Branisteanu
- Department of Ophthalmology, 'Grigore T. Popa' University of Medicine and Pharmacy, 700115 Iaşi, Romania.,'Retina Center' Eye Clinic, 700126 Iași, Romania
| |
Collapse
|
15
|
Diving Deep into Deep Learning: An Update on Artificial Intelligence in Retina. CURRENT OPHTHALMOLOGY REPORTS 2020; 8:121-128. [PMID: 33224635 DOI: 10.1007/s40135-020-00240-2] [Citation(s) in RCA: 3] [Impact Index Per Article: 0.8] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/18/2022]
Abstract
Purpose of Review In the present article, we will provide an understanding and review of artificial intelligence in the subspecialty of retina and its potential applications within the specialty. Recent Findings Given the significant use of diagnostic imaging within retina, this subspecialty is a fitting area for the incorporation of artificial intelligence. Researchers have aimed at creating models to assist in the diagnosis and management of retinal disease as well as in the prediction of disease course and treatment response. Most of this work thus far has focused on diabetic retinopathy, age-related macular degeneration, and retinopathy of prematurity, although other retinal diseases have started to be explored as well. Summary Artificial intelligence is well-suited to transform the practice of ophthalmology. A basic understanding of the technology is important for its effective implementation and growth.
Collapse
|
16
|
Tong Y, Lu W, Yu Y, Shen Y. Application of machine learning in ophthalmic imaging modalities. EYE AND VISION 2020; 7:22. [PMID: 32322599 PMCID: PMC7160952 DOI: 10.1186/s40662-020-00183-6] [Citation(s) in RCA: 36] [Impact Index Per Article: 9.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 10/16/2019] [Accepted: 03/10/2020] [Indexed: 12/27/2022]
Abstract
In clinical ophthalmology, a variety of image-related diagnostic techniques have begun to offer unprecedented insights into eye diseases based on morphological datasets with millions of data points. Artificial intelligence (AI), inspired by the human multilayered neuronal system, has shown astonishing success within some visual and auditory recognition tasks. In these tasks, AI can analyze digital data in a comprehensive, rapid and non-invasive manner. Bioinformatics has become a focus particularly in the field of medical imaging, where it is driven by enhanced computing power and cloud storage, as well as utilization of novel algorithms and generation of data in massive quantities. Machine learning (ML) is an important branch in the field of AI. The overall potential of ML to automatically pinpoint, identify and grade pathological features in ocular diseases will empower ophthalmologists to provide high-quality diagnosis and facilitate personalized health care in the near future. This review offers perspectives on the origin, development, and applications of ML technology, particularly regarding its applications in ophthalmic imaging modalities.
Collapse
Affiliation(s)
- Yan Tong
- 1Eye Center, Renmin Hospital of Wuhan University, Wuhan, 430060 Hubei China
| | - Wei Lu
- 1Eye Center, Renmin Hospital of Wuhan University, Wuhan, 430060 Hubei China
| | - Yue Yu
- 1Eye Center, Renmin Hospital of Wuhan University, Wuhan, 430060 Hubei China
| | - Yin Shen
- 1Eye Center, Renmin Hospital of Wuhan University, Wuhan, 430060 Hubei China.,2Medical Research Institute, Wuhan University, Wuhan, Hubei China
| |
Collapse
|
17
|
Kugelman J, Alonso-Caneiro D, Read SA, Hamwood J, Vincent SJ, Chen FK, Collins MJ. Automatic choroidal segmentation in OCT images using supervised deep learning methods. Sci Rep 2019; 9:13298. [PMID: 31527630 PMCID: PMC6746702 DOI: 10.1038/s41598-019-49816-4] [Citation(s) in RCA: 52] [Impact Index Per Article: 10.4] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/31/2018] [Accepted: 09/02/2019] [Indexed: 11/08/2022] Open
Abstract
The analysis of the choroid in the eye is crucial for our understanding of a range of ocular diseases and physiological processes. Optical coherence tomography (OCT) imaging provides the ability to capture highly detailed cross-sectional images of the choroid yet only a very limited number of commercial OCT instruments provide methods for automatic segmentation of choroidal tissue. Manual annotation of the choroidal boundaries is often performed but this is impractical due to the lengthy time taken to analyse large volumes of images. Therefore, there is a pressing need for reliable and accurate methods to automatically segment choroidal tissue boundaries in OCT images. In this work, a variety of patch-based and fully-convolutional deep learning methods are proposed to accurately determine the location of the choroidal boundaries of interest. The effect of network architecture, patch-size and contrast enhancement methods was tested to better understand the optimal architecture and approach to maximize performance. The results are compared with manual boundary segmentation used as a ground-truth, as well as with a standard image analysis technique. Results of total retinal layer segmentation are also presented for comparison purposes. The findings presented here demonstrate the benefit of deep learning methods for segmentation of the chorio-retinal boundary analysis in OCT images.
Collapse
Affiliation(s)
- Jason Kugelman
- Contact Lens and Visual Optics Laboratory, School of Optometry and Vision Science, Queensland University of Technology, Brisbane, Queensland, Australia
| | - David Alonso-Caneiro
- Contact Lens and Visual Optics Laboratory, School of Optometry and Vision Science, Queensland University of Technology, Brisbane, Queensland, Australia.
- Centre for Ophthalmology and Visual Science, The University of Western Australia, Perth, Australia.
| | - Scott A Read
- Contact Lens and Visual Optics Laboratory, School of Optometry and Vision Science, Queensland University of Technology, Brisbane, Queensland, Australia
| | - Jared Hamwood
- Contact Lens and Visual Optics Laboratory, School of Optometry and Vision Science, Queensland University of Technology, Brisbane, Queensland, Australia
| | - Stephen J Vincent
- Contact Lens and Visual Optics Laboratory, School of Optometry and Vision Science, Queensland University of Technology, Brisbane, Queensland, Australia
| | - Fred K Chen
- Centre for Ophthalmology and Visual Science, The University of Western Australia, Perth, Australia
- Ocular Tissue Engineering Laboratory, Lions Eye Institute, Perth, Australia
- Department of Ophthalmology, Royal Perth Hospital, Perth, Australia
| | - Michael J Collins
- Contact Lens and Visual Optics Laboratory, School of Optometry and Vision Science, Queensland University of Technology, Brisbane, Queensland, Australia
| |
Collapse
|
18
|
Ting DS, Peng L, Varadarajan AV, Keane PA, Burlina PM, Chiang MF, Schmetterer L, Pasquale LR, Bressler NM, Webster DR, Abramoff M, Wong TY. Deep learning in ophthalmology: The technical and clinical considerations. Prog Retin Eye Res 2019; 72:100759. [DOI: 10.1016/j.preteyeres.2019.04.003] [Citation(s) in RCA: 137] [Impact Index Per Article: 27.4] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/23/2018] [Revised: 04/21/2019] [Accepted: 04/23/2019] [Indexed: 12/22/2022]
|
19
|
Rong Y, Xiang D, Zhu W, Shi F, Gao E, Fan Z, Chen X. Deriving external forces via convolutional neural networks for biomedical image segmentation. BIOMEDICAL OPTICS EXPRESS 2019; 10:3800-3814. [PMID: 31452976 PMCID: PMC6701547 DOI: 10.1364/boe.10.003800] [Citation(s) in RCA: 7] [Impact Index Per Article: 1.4] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Received: 05/15/2019] [Revised: 06/26/2019] [Accepted: 06/27/2019] [Indexed: 05/07/2023]
Abstract
Active contours, or snakes, are widely applied on biomedical image segmentation. They are curves defined within an image domain that can move to object boundaries under the influence of internal forces and external forces, in which the internal forces are generally computed from curves themselves and external forces from image data. Designing external forces properly is a key point with active contour algorithms since the external forces play a leading role in the evolution of active contours. One of most popular external forces for active contour models is gradient vector flow (GVF). However, GVF is sensitive to noise and false edges, which limits its application area. To handle this problem, in this paper, we propose using GVF as reference to train a convolutional neural network to derive an external force. The derived external force is then integrated into the active contour models for curve evolution. Three clinical applications, segmentation of optic disk in fundus images, fluid in retinal optical coherence tomography images and fetal head in ultrasound images, are employed to evaluate the proposed method. The results show that the proposed method is very promising since it achieves competitive performance for different tasks compared to the state-of-the-art algorithms.
Collapse
Affiliation(s)
- Yibiao Rong
- School of Electrical and Information Engineering, Soochow University, 215006, Suzhou, China
- Contributed equally to this work
| | - Dehui Xiang
- School of Electrical and Information Engineering, Soochow University, 215006, Suzhou, China
- Contributed equally to this work
| | - Weifang Zhu
- School of Electrical and Information Engineering, Soochow University, 215006, Suzhou, China
| | - Fei Shi
- School of Electrical and Information Engineering, Soochow University, 215006, Suzhou, China
| | - Enting Gao
- School of Electronic and Information Engineering, Suzhou University of Science and Technology, Suzhou, China
| | - Zhun Fan
- Key Laboratory of Digital Signal and Image Processing of Guangdong Provincial, College of Engineering, Shantou University, 515063, Shantou, China
| | - Xinjian Chen
- School of Electrical and Information Engineering, Soochow University, 215006, Suzhou, China
- State Key Laboratory of Radiation Medicine and Protection, Soochow University, 215123, Suzhou, China
| |
Collapse
|
20
|
[Deep learning and neuronal networks in ophthalmology : Applications in the field of optical coherence tomography]. Ophthalmologe 2019; 115:714-721. [PMID: 29675699 DOI: 10.1007/s00347-018-0706-0] [Citation(s) in RCA: 7] [Impact Index Per Article: 1.4] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/13/2023]
Abstract
Deep learning is increasingly becoming the focus of various imaging methods in medicine. Due to the large number of different imaging modalities, ophthalmology is particularly suitable for this field of application. This article gives a general overview on the topic of deep learning and its current applications in the field of optical coherence tomography. For the benefit of the reader it focuses on the clinical rather than the technical aspects.
Collapse
|
21
|
|
22
|
Hu J, Chen Y, Yi Z. Automated segmentation of macular edema in OCT using deep neural networks. Med Image Anal 2019; 55:216-227. [PMID: 31096135 DOI: 10.1016/j.media.2019.05.002] [Citation(s) in RCA: 35] [Impact Index Per Article: 7.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/25/2018] [Revised: 04/23/2019] [Accepted: 05/09/2019] [Indexed: 11/29/2022]
Abstract
Macular edema is an eye disease that can affect visual acuity. Typical disease symptoms include subretinal fluid (SRF) and pigment epithelium detachment (PED). Optical coherence tomography (OCT) has been widely used for diagnosing macular edema because of its non-invasive and high resolution properties. Segmentation for macular edema lesions from OCT images plays an important role in clinical diagnosis. Many computer-aided systems have been proposed for the segmentation. Most traditional segmentation methods used in these systems are based on low-level hand-crafted features, which require significant domain knowledge and are sensitive to the variations of lesions. To overcome these shortcomings, this paper proposes to use deep neural networks (DNNs) together with atrous spatial pyramid pooling (ASPP) to automatically segment the SRF and PED lesions. Lesions-related features are first extracted by DNNs, then processed by ASPP which is composed of multiple atrous convolutions with different fields of view to accommodate the various scales of the lesions. Based on ASPP, a novel module called stochastic ASPP (sASPP) is proposed to combat the co-adaptation of multiple atrous convolutions. A large OCT dataset provided by a competition platform called "AI Challenger" are used to train and evaluate the proposed model. Experimental results demonstrate that the DNNs together with ASPP achieve higher segmentation accuracy compared with the state-of-the-art method. The stochastic operation added in sASPP is empirically verified as an effective regularization method that can alleviate the overfitting problem and significantly reduce the validation error.
Collapse
Affiliation(s)
- Junjie Hu
- Machine Intelligence Laboratory, College of Computer Science, Sichuan University, Chengdu 610065, PR China
| | - Yuanyuan Chen
- Machine Intelligence Laboratory, College of Computer Science, Sichuan University, Chengdu 610065, PR China
| | - Zhang Yi
- Machine Intelligence Laboratory, College of Computer Science, Sichuan University, Chengdu 610065, PR China.
| |
Collapse
|
23
|
Liu X, Bi L, Xu Y, Feng D, Kim J, Xu X. Robust deep learning method for choroidal vessel segmentation on swept source optical coherence tomography images. BIOMEDICAL OPTICS EXPRESS 2019; 10:1601-1612. [PMID: 31061759 PMCID: PMC6485000 DOI: 10.1364/boe.10.001601] [Citation(s) in RCA: 22] [Impact Index Per Article: 4.4] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Received: 12/21/2018] [Revised: 02/13/2019] [Accepted: 02/13/2019] [Indexed: 05/06/2023]
Abstract
Accurate choroidal vessel segmentation with swept-source optical coherence tomography (SS-OCT) images provide unprecedented quantitative analysis towards the understanding of choroid-related diseases. Motivated by the leading segmentation performance in medical images from the use of deep learning methods, in this study, we proposed the adoption of a deep learning method, RefineNet, to segment the choroidal vessels from SS-OCT images. We quantitatively evaluated the RefineNet on 40 SS-OCT images consisting of ~3,900 manually annotated choroidal vessels regions. We achieved a segmentation agreement (SA) of 0.840 ± 0.035 with clinician 1 (C1) and 0.823 ± 0.027 with clinician 2 (C2). These results were higher than inter-observer variability measure in SA between C1 and C2 of 0.821 ± 0.037. Our results demonstrated that the choroidal vessels from SS-OCT can be automatically segmented using a deep learning method and thus provided a new approach towards an objective and reproducible quantitative analysis of vessel regions.
Collapse
Affiliation(s)
- Xiaoxiao Liu
- Department of Ophthalmology, Shanghai General Hospital, Shanghai Jiao Tong University School of Medicine, Shanghai, 200080, China
- Shanghai Key Laboratory of Fundus Disease, Shanghai Engineering Center for Visual Science and Photomedicine, Shanghai, 200080, China
| | - Lei Bi
- Biomedical and Multimedia Information Technology (BMIT) Research Group, The University of Sydney, Sydney, NSW, 2006, Australia
| | - Yupeng Xu
- Department of Ophthalmology, Shanghai General Hospital, Shanghai Jiao Tong University School of Medicine, Shanghai, 200080, China
- Shanghai Key Laboratory of Fundus Disease, Shanghai Engineering Center for Visual Science and Photomedicine, Shanghai, 200080, China
| | - Dagan Feng
- Shanghai Key Laboratory of Fundus Disease, Shanghai Engineering Center for Visual Science and Photomedicine, Shanghai, 200080, China
| | - Jinman Kim
- Biomedical and Multimedia Information Technology (BMIT) Research Group, The University of Sydney, Sydney, NSW, 2006, Australia
| | - Xun Xu
- Department of Ophthalmology, Shanghai General Hospital, Shanghai Jiao Tong University School of Medicine, Shanghai, 200080, China
| |
Collapse
|
24
|
Liaskos M, Asvestas PA, Matsopoulos GK, Charonis A, Anastassopoulos V. Detection of retinal pigment epithelium detachment from OCT images using multiscale Gaussian filtering. Technol Health Care 2019; 27:301-316. [PMID: 30829626 DOI: 10.3233/thc-181501] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.2] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/15/2022]
Abstract
Macular diseases, including neovascular age-related macular degeneration (nvAMD), are leading causes of irreversible blindness and visual impairment. One prominent feature of nvAMD is the detachment of the retinal pigment epithelium. The aim of this study is to implement an automated method for the segmentation of the pigment epithelial detachment (PED) using optical coherence tomography (OCT). OCT datasets from 8 patients with nvAMD were acquired during multiple sessions. At each session, 17 images with a resolution of 1020 × 640 pixels were obtained. The images were segmented using Gaussian filtering and template matching for the detection of the upper and lower border of the PED, respectively. The results of the method were compared with the ones obtained from the manual segmentation of the images by an expert. Four well-known metrics were used to evaluate the performance of the method with respect to the manual segmentation, resulting in high scores of consistency. Furthermore, the proposed method was also compared with four other well-known methods providing similar or superior performance.
Collapse
Affiliation(s)
- Meletios Liaskos
- Physics Department, University of Patras, Patras, Greece.,School of Electrical and Computer Engineering, National Technical University of Athens, Athens, Greece
| | - Pantelis A Asvestas
- Department of Biomedical Engineering, University of West Attica, Athens, Greece
| | - George K Matsopoulos
- School of Electrical and Computer Engineering, National Technical University of Athens, Athens, Greece
| | | | | |
Collapse
|
25
|
Pfister M, Schützenberger K, Pfeiffenberger U, Messner A, Chen Z, dos Santos VA, Puchner S, Garhöfer G, Schmetterer L, Gröschl M, Werkmeister RM. Automated segmentation of dermal fillers in OCT images of mice using convolutional neural networks. BIOMEDICAL OPTICS EXPRESS 2019; 10:1315-1328. [PMID: 30891348 PMCID: PMC6420291 DOI: 10.1364/boe.10.001315] [Citation(s) in RCA: 9] [Impact Index Per Article: 1.8] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Received: 10/24/2018] [Revised: 01/16/2019] [Accepted: 01/18/2019] [Indexed: 05/30/2023]
Abstract
We present a system for automatic determination of the intradermal volume of hydrogels based on optical coherence tomography (OCT) and deep learning. Volumetric image data was acquired using a custom-built OCT prototype that employs an akinetic swept laser at ~1310 nm with a bandwidth of 87 nm, providing an axial resolution of ~6.5 μm in tissue. Three-dimensional data sets of a 10 mm × 10 mm skin patch comprising the intradermal filler and the surrounding tissue were acquired. A convolutional neural network using a u-net-like architecture was trained from slices of 100 OCT volume data sets where the dermal filler volume was manually annotated. Using six-fold cross-validation, a mean accuracy of 0.9938 and a Jaccard similarity coefficient of 0.879 were achieved.
Collapse
Affiliation(s)
- Martin Pfister
- Center for Medical Physics and Biomedical Engineering, Medical University of Vienna, Waehringer Guertel 18-20, 1090 Vienna,
Austria
- Christian Doppler Laboratory for Ocular and Dermal Effects of Thiomers, Medical University of Vienna, Waehringer Guertel 18-20, 1090 Vienna,
Austria
- Institute of Applied Physics, Vienna University of Technology, Wiedner Hauptstr. 8-10, 1040 Vienna,
Austria
| | - Kornelia Schützenberger
- Center for Medical Physics and Biomedical Engineering, Medical University of Vienna, Waehringer Guertel 18-20, 1090 Vienna,
Austria
- Christian Doppler Laboratory for Ocular and Dermal Effects of Thiomers, Medical University of Vienna, Waehringer Guertel 18-20, 1090 Vienna,
Austria
| | - Ulrike Pfeiffenberger
- Center for Medical Physics and Biomedical Engineering, Medical University of Vienna, Waehringer Guertel 18-20, 1090 Vienna,
Austria
- Christian Doppler Laboratory for Ocular and Dermal Effects of Thiomers, Medical University of Vienna, Waehringer Guertel 18-20, 1090 Vienna,
Austria
| | - Alina Messner
- Center for Medical Physics and Biomedical Engineering, Medical University of Vienna, Waehringer Guertel 18-20, 1090 Vienna,
Austria
| | - Zhe Chen
- Center for Medical Physics and Biomedical Engineering, Medical University of Vienna, Waehringer Guertel 18-20, 1090 Vienna,
Austria
| | - Valentin Aranha dos Santos
- Center for Medical Physics and Biomedical Engineering, Medical University of Vienna, Waehringer Guertel 18-20, 1090 Vienna,
Austria
| | - Stefan Puchner
- Center for Medical Physics and Biomedical Engineering, Medical University of Vienna, Waehringer Guertel 18-20, 1090 Vienna,
Austria
- Christian Doppler Laboratory for Ocular and Dermal Effects of Thiomers, Medical University of Vienna, Waehringer Guertel 18-20, 1090 Vienna,
Austria
- Department of Clinical Pharmacology, Medical University of Vienna, Waehringer Guertel 18-20, 1090 Vienna,
Austria
| | - Gerhard Garhöfer
- Christian Doppler Laboratory for Ocular and Dermal Effects of Thiomers, Medical University of Vienna, Waehringer Guertel 18-20, 1090 Vienna,
Austria
- Department of Clinical Pharmacology, Medical University of Vienna, Waehringer Guertel 18-20, 1090 Vienna,
Austria
| | - Leopold Schmetterer
- Center for Medical Physics and Biomedical Engineering, Medical University of Vienna, Waehringer Guertel 18-20, 1090 Vienna,
Austria
- Christian Doppler Laboratory for Ocular and Dermal Effects of Thiomers, Medical University of Vienna, Waehringer Guertel 18-20, 1090 Vienna,
Austria
- Department of Clinical Pharmacology, Medical University of Vienna, Waehringer Guertel 18-20, 1090 Vienna,
Austria
- Singapore Eye Research Institute, The Academia, 20 College Road, Discovery Tower Level 6, 169856,
Singapore
- Lee Kong Chian School of Medicine, Nanyang Technological University, Novena Campus, 11 Mandalay Road, 308232,
Singapore
- Ophthalmology and Visual Sciences Academic Clinical Program, Duke-NUS Medical School, 8 College Road, 169857,
Singapore
| | - Martin Gröschl
- Institute of Applied Physics, Vienna University of Technology, Wiedner Hauptstr. 8-10, 1040 Vienna,
Austria
| | - René M. Werkmeister
- Center for Medical Physics and Biomedical Engineering, Medical University of Vienna, Waehringer Guertel 18-20, 1090 Vienna,
Austria
- Christian Doppler Laboratory for Ocular and Dermal Effects of Thiomers, Medical University of Vienna, Waehringer Guertel 18-20, 1090 Vienna,
Austria
| |
Collapse
|
26
|
Akkara J, Kuriakose A. Role of artificial intelligence and machine learning in ophthalmology. KERALA JOURNAL OF OPHTHALMOLOGY 2019. [DOI: 10.4103/kjo.kjo_54_19] [Citation(s) in RCA: 24] [Impact Index Per Article: 4.8] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/04/2022] Open
|
27
|
Applications of Artificial Intelligence in Ophthalmology: General Overview. J Ophthalmol 2018; 2018:5278196. [PMID: 30581604 PMCID: PMC6276430 DOI: 10.1155/2018/5278196] [Citation(s) in RCA: 50] [Impact Index Per Article: 8.3] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/07/2018] [Revised: 10/06/2018] [Accepted: 10/17/2018] [Indexed: 12/26/2022] Open
Abstract
With the emergence of unmanned plane, autonomous vehicles, face recognition, and language processing, the artificial intelligence (AI) has remarkably revolutionized our lifestyle. Recent studies indicate that AI has astounding potential to perform much better than human beings in some tasks, especially in the image recognition field. As the amount of image data in imaging center of ophthalmology is increasing dramatically, analyzing and processing these data is in urgent need. AI has been tried to apply to decipher medical data and has made extraordinary progress in intelligent diagnosis. In this paper, we presented the basic workflow for building an AI model and systematically reviewed applications of AI in the diagnosis of eye diseases. Future work should focus on setting up systematic AI platforms to diagnose general eye diseases based on multimodal data in the real world.
Collapse
|
28
|
Kugelman J, Alonso-Caneiro D, Read SA, Vincent SJ, Collins MJ. Automatic segmentation of OCT retinal boundaries using recurrent neural networks and graph search. BIOMEDICAL OPTICS EXPRESS 2018; 9:5759-5777. [PMID: 30460160 PMCID: PMC6238930 DOI: 10.1364/boe.9.005759] [Citation(s) in RCA: 64] [Impact Index Per Article: 10.7] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Received: 09/05/2018] [Revised: 10/13/2018] [Accepted: 10/15/2018] [Indexed: 05/06/2023]
Abstract
The manual segmentation of individual retinal layers within optical coherence tomography (OCT) images is a time-consuming task and is prone to errors. The investigation into automatic segmentation methods that are both efficient and accurate has seen a variety of methods proposed. In particular, recent machine learning approaches have focused on the use of convolutional neural networks (CNNs). Traditionally applied to sequential data, recurrent neural networks (RNNs) have recently demonstrated success in the area of image analysis, primarily due to their usefulness to extract temporal features from sequences of images or volumetric data. However, their potential use in OCT retinal layer segmentation has not previously been reported, and their direct application for extracting spatial features from individual 2D images has been limited. This paper proposes the use of a recurrent neural network trained as a patch-based image classifier (retinal boundary classifier) with a graph search (RNN-GS) to segment seven retinal layer boundaries in OCT images from healthy children and three retinal layer boundaries in OCT images from patients with age-related macular degeneration (AMD). The optimal architecture configuration to maximize classification performance is explored. The results demonstrate that a RNN is a viable alternative to a CNN for image classification tasks in the case where the images exhibit a clear sequential structure. Compared to a CNN, the RNN showed a slightly superior average generalization classification accuracy. Secondly, in terms of segmentation, the RNN-GS performed competitively against a previously proposed CNN based method (CNN-GS) with respect to both accuracy and consistency. These findings apply to both normal and AMD data. Overall, the RNN-GS method yielded superior mean absolute errors in terms of the boundary position with an average error of 0.53 pixels (normal) and 1.17 pixels (AMD). The methodology and results described in this paper may assist the future investigation of techniques within the area of OCT retinal segmentation and highlight the potential of RNN methods for OCT image analysis.
Collapse
|
29
|
Hamwood J, Alonso-Caneiro D, Read SA, Vincent SJ, Collins MJ. Effect of patch size and network architecture on a convolutional neural network approach for automatic segmentation of OCT retinal layers. BIOMEDICAL OPTICS EXPRESS 2018; 9:3049-3066. [PMID: 29984082 PMCID: PMC6033561 DOI: 10.1364/boe.9.003049] [Citation(s) in RCA: 48] [Impact Index Per Article: 8.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Subscribe] [Scholar Register] [Received: 03/27/2018] [Revised: 06/01/2018] [Accepted: 06/01/2018] [Indexed: 05/06/2023]
Abstract
Deep learning strategies, particularly convolutional neural networks (CNNs), are especially suited to finding patterns in images and using those patterns for image classification. The method is normally applied to an image patch and assigns a class weight to the patch; this method has recently been used to detect the probability of retinal boundary locations in OCT images, which is subsequently used to segment the OCT image using a graph-search approach. This paper examines the effects of a number of modifications to the CNN architecture with the aim of optimizing retinal layer segmentation, specifically the effect of patch size as well as the network architecture design on CNN performance and subsequent layer segmentation. The results demonstrate that increasing patch size can improve the performance of the classification and provides a more reliable segmentation in the analysis of retinal layer characteristics in OCT imaging. Similarly, this work shows that changing aspects of the CNN network design can also significantly improve the segmentation results. This work also demonstrates that the performance of the method can change depending on the number of classes (i.e. boundaries) used to train the CNN, with fewer classes showing an inferior performance due to the presence of similar image features between classes that can trigger false positives. Changes in the network (patch size and or architecture) can be applied to provide a superior segmentation performance, which is robust to the class effect. The findings from this work may inform future CNN development in OCT retinal image analysis.
Collapse
Affiliation(s)
- Jared Hamwood
- Contact Lens and Visual Optics Laboratory, School of Optometry and Vision Science, Queensland University of Technology, Brisbane, Queensland, Australia
| | - David Alonso-Caneiro
- Contact Lens and Visual Optics Laboratory, School of Optometry and Vision Science, Queensland University of Technology, Brisbane, Queensland, Australia
| | - Scott A. Read
- Contact Lens and Visual Optics Laboratory, School of Optometry and Vision Science, Queensland University of Technology, Brisbane, Queensland, Australia
| | - Stephen J. Vincent
- Contact Lens and Visual Optics Laboratory, School of Optometry and Vision Science, Queensland University of Technology, Brisbane, Queensland, Australia
| | - Michael J. Collins
- Contact Lens and Visual Optics Laboratory, School of Optometry and Vision Science, Queensland University of Technology, Brisbane, Queensland, Australia
| |
Collapse
|
30
|
Automated Segmentation of Lesions Including Subretinal Hyperreflective Material in Neovascular Age-related Macular Degeneration. Am J Ophthalmol 2018; 191:64-75. [PMID: 29655643 DOI: 10.1016/j.ajo.2018.04.007] [Citation(s) in RCA: 18] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/10/2018] [Revised: 03/31/2018] [Accepted: 04/04/2018] [Indexed: 12/15/2022]
Abstract
PURPOSE To evaluate an automated segmentation algorithm with a convolutional neural network (CNN) to quantify and detect intraretinal fluid (IRF), subretinal fluid (SRF), pigment epithelial detachment (PED), and subretinal hyperreflective material (SHRM) through analyses of spectral-domain optical coherence tomography (SD-OCT) images from patients with neovascular age-related macular degeneration (nAMD). DESIGN Reliability and validity analysis of a diagnostic tool. METHODS We constructed a dataset including 930 B-scans from 93 eyes of 93 patients with nAMD. A CNN-based deep neural network was trained using 11 550 augmented images derived from 550 B-scans. The performance of the trained network was evaluated using a validation set including 140 B-scans and a test set of 240 B-scans. The Dice coefficient, positive predictive value (PPV), sensitivity, relative area difference (RAD), and intraclass correlation coefficient (ICC) were used to evaluate segmentation and detection performance. RESULTS Good agreement was observed for both segmentation and detection of lesions between the trained network and clinicians. The Dice coefficients for segmentation of IRF, SRF, SHRM, and PED were 0.78, 0.82, 0.75, and 0.80, respectively; the PPVs were 0.79, 0.80, 0.75, and 0.80, respectively; and the sensitivities were 0.77, 0.84, 0.73, and 0.81, respectively. The RADs were -4.32%, -10.29%, 4.13%, and 0.34%, respectively, and the ICCs were 0.98, 0.98, 0.97, and 0.98, respectively. All lesions were detected with high PPVs (range 0.94-0.99) and sensitivities (range 0.97-0.99). CONCLUSIONS A CNN-based network provides clinicians with quantitative data regarding nAMD through automatic segmentation and detection of pathologic lesions, including IRF, SRF, PED, and SHRM.
Collapse
|
31
|
Loo J, Fang L, Cunefare D, Jaffe GJ, Farsiu S. Deep longitudinal transfer learning-based automatic segmentation of photoreceptor ellipsoid zone defects on optical coherence tomography images of macular telangiectasia type 2. BIOMEDICAL OPTICS EXPRESS 2018; 9:2681-2698. [PMID: 30258683 PMCID: PMC6154208 DOI: 10.1364/boe.9.002681] [Citation(s) in RCA: 37] [Impact Index Per Article: 6.2] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 03/26/2018] [Revised: 05/10/2018] [Accepted: 05/11/2018] [Indexed: 05/20/2023]
Abstract
Photoreceptor ellipsoid zone (EZ) defects visible on optical coherence tomography (OCT) are important imaging biomarkers for the onset and progression of macular diseases. As such, accurate quantification of EZ defects is paramount to monitor disease progression and treatment efficacy over time. We developed and trained a novel deep learning-based method called Deep OCT Atrophy Detection (DOCTAD) to automatically segment EZ defect areas by classifying 3-dimensional A-scan clusters as normal or defective. Furthermore, we introduce a longitudinal transfer learning paradigm in which the algorithm learns from segmentation errors on images obtained at one time point to segment subsequent images with higher accuracy. We evaluated the performance of this method on 134 eyes of 67 subjects enrolled in a clinical trial of a novel macular telangiectasia type 2 (MacTel2) therapeutic agent. Our method compared favorably to other deep learning-based and non-deep learning-based methods in matching expert manual segmentations. To the best of our knowledge, this is the first automatic segmentation method developed for EZ defects on OCT images of MacTel2.
Collapse
Affiliation(s)
- Jessica Loo
- Department of Biomedical Engineering, Duke University, Durham, NC 27708, USA
| | - Leyuan Fang
- Department of Biomedical Engineering, Duke University, Durham, NC 27708, USA
| | - David Cunefare
- Department of Biomedical Engineering, Duke University, Durham, NC 27708, USA
| | - Glenn J. Jaffe
- Department of Ophthalmology, Duke University, Durham, NC 27708, USA
| | - Sina Farsiu
- Department of Biomedical Engineering, Duke University, Durham, NC 27708, USA
- Department of Ophthalmology, Duke University, Durham, NC 27708, USA
| |
Collapse
|