1
|
Liu X, Li X, Zhang Y, Wang M, Yao J, Tang J. Boundary-Repairing Dual-Path Network for Retinal Layer Segmentation in OCT Image with Pigment Epithelial Detachment. JOURNAL OF IMAGING INFORMATICS IN MEDICINE 2024:10.1007/s10278-024-01093-y. [PMID: 38740662 DOI: 10.1007/s10278-024-01093-y] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 10/05/2023] [Revised: 03/05/2024] [Accepted: 03/06/2024] [Indexed: 05/16/2024]
Abstract
Automatic retinal layer segmentation in optical coherence tomography (OCT) images is crucial for the diagnosis of ocular diseases. Currently, automatic retinal layer segmentation works well with normal OCT images. However, pigment epithelial detachment (PED) dramatically alters the retinal structure, causing blurred boundaries and partial disappearance of the Bruch's Membrane (BM), thus posing challenges to the segmentation. To tackle these problems, we propose a novel dual-path U-shaped network for simultaneous layer segmentation and boundary regression. This network first designs a feature interaction fusion (FIF) module to strengthen the boundary shape constraints in the layer path. To address the challenge posed by partial BM disappearance and boundary-blurring, we propose a layer boundary repair (LBR) module. This module aims to use contrastive loss to enhance the confidence of blurred boundary regions and refine the segmentation of layer boundaries through the re-prediction head. In addition, we introduce a novel bilateral threshold distance map (BTDM) designed for the boundary path. The BTDM serves to emphasize information within boundary regions. This map, combined with the updated probability map, culminates in topology-guaranteed segmentation results achieved through a topology correction (TC) module. We investigated the proposed network on two severely deformed datasets (i.e., OCTA-500 and Aier-PED) and one slightly deformed dataset (i.e., DUKE). The proposed method achieves an average Dice score of 94.26% on the OCTA-500 dataset, which was 1.5% higher than BAU-Net and outperformed other methods. In the DUKE and Aier-PED datasets, the proposed method achieved average Dice scores of 91.65% and 95.75%, respectively.
Collapse
Affiliation(s)
- Xiaoming Liu
- School of Computer Science and Technology, Wuhan University of Science and Technology, Wuhan, 430065, China.
- Hubei Province Key Laboratory of Intelligent Information Processing and Real-Time Industrial System, Wuhan, 430065, China.
| | - Xiao Li
- School of Computer Science and Technology, Wuhan University of Science and Technology, Wuhan, 430065, China
- Hubei Province Key Laboratory of Intelligent Information Processing and Real-Time Industrial System, Wuhan, 430065, China
| | - Ying Zhang
- Wuhan Aier Eye Hospital of Wuhan University, Wuhan, China
| | - Man Wang
- Wuhan Aier Eye Hospital of Wuhan University, Wuhan, China
| | - Junping Yao
- Department of Ophthalmology, Tianyou Hospital Affiliated to Wuhan University of Science and Technology, Wuhan, China
| | - Jinshan Tang
- Department of Health Administration and Policy, College of Health and Human Services, George Mason University, Fairfax, VA, 22030, USA
| |
Collapse
|
2
|
Lu J, Cheng Y, Hiya FE, Shen M, Herrera G, Zhang Q, Gregori G, Rosenfeld PJ, Wang RK. Deep-learning-based automated measurement of outer retinal layer thickness for use in the assessment of age-related macular degeneration, applicable to both swept-source and spectral-domain OCT imaging. BIOMEDICAL OPTICS EXPRESS 2024; 15:413-427. [PMID: 38223170 PMCID: PMC10783897 DOI: 10.1364/boe.512359] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 11/16/2023] [Revised: 12/17/2023] [Accepted: 12/17/2023] [Indexed: 01/16/2024]
Abstract
Effective biomarkers are required for assessing the progression of age-related macular degeneration (AMD), a prevalent and progressive eye disease. This paper presents a deep learning-based automated algorithm, applicable to both swept-source OCT (SS-OCT) and spectral-domain OCT (SD-OCT) scans, for measuring outer retinal layer (ORL) thickness as a surrogate biomarker for outer retinal degeneration, e.g., photoreceptor disruption, to assess AMD progression. The algorithm was developed based on a modified TransUNet model with clinically annotated retinal features manifested in the progression of AMD. The algorithm demonstrates a high accuracy with an intersection of union (IoU) of 0.9698 in the testing dataset for segmenting ORL using both SS-OCT and SD-OCT datasets. The robustness and applicability of the algorithm are indicated by strong correlation (r = 0.9551, P < 0.0001 in the central-fovea 3 mm-circle, and r = 0.9442, P < 0.0001 in the 5 mm-circle) and agreement (the mean bias = 0.5440 um in the 3-mm circle, and 1.392 um in the 5-mm circle) of the ORL thickness measurements between SS-OCT and SD-OCT scans. Comparative analysis reveals significant differences (P < 0.0001) in ORL thickness among 80 normal eyes, 30 intermediate AMD eyes with reticular pseudodrusen, 49 intermediate AMD eyes with drusen, and 40 late AMD eyes with geographic atrophy, highlighting its potential as an independent biomarker for predicting AMD progression. The findings provide valuable insights into the ORL alterations associated with different stages of AMD and emphasize the potential of ORL thickness as a sensitive indicator of AMD severity and progression.
Collapse
Affiliation(s)
- Jie Lu
- Department of Bioengineering, University of Washington, Seattle, Washington, USA
| | - Yuxuan Cheng
- Department of Bioengineering, University of Washington, Seattle, Washington, USA
| | - Farhan E. Hiya
- Department of Ophthalmology, Bascom Palmer Eye Institute, University of Miami Miller School of Medicine, Miami, Florida, USA
| | - Mengxi Shen
- Department of Ophthalmology, Bascom Palmer Eye Institute, University of Miami Miller School of Medicine, Miami, Florida, USA
| | - Gissel Herrera
- Department of Ophthalmology, Bascom Palmer Eye Institute, University of Miami Miller School of Medicine, Miami, Florida, USA
| | - Qinqin Zhang
- Research and Development, Carl Zeiss Meditec, Inc., Dublin, CA, USA
| | - Giovanni Gregori
- Department of Ophthalmology, Bascom Palmer Eye Institute, University of Miami Miller School of Medicine, Miami, Florida, USA
| | - Philip J. Rosenfeld
- Department of Ophthalmology, Bascom Palmer Eye Institute, University of Miami Miller School of Medicine, Miami, Florida, USA
| | - Ruikang K. Wang
- Department of Bioengineering, University of Washington, Seattle, Washington, USA
- Department of Ophthalmology, University of Washington, Seattle, Washington, USA
| |
Collapse
|
3
|
Liu H, Wei D, Lu D, Tang X, Wang L, Zheng Y. Simultaneous alignment and surface regression using hybrid 2D-3D networks for 3D coherent layer segmentation of retinal OCT images with full and sparse annotations. Med Image Anal 2024; 91:103019. [PMID: 37944431 DOI: 10.1016/j.media.2023.103019] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/11/2022] [Revised: 08/28/2023] [Accepted: 10/31/2023] [Indexed: 11/12/2023]
Abstract
Layer segmentation is important to quantitative analysis of retinal optical coherence tomography (OCT). Recently, deep learning based methods have been developed to automate this task and yield remarkable performance. However, due to the large spatial gap and potential mismatch between the B-scans of an OCT volume, all of them were based on 2D segmentation of individual B-scans, which may lose the continuity and diagnostic information of the retinal layers in 3D space. Besides, most of these methods required dense annotation of the OCT volumes, which is labor-intensive and expertise-demanding. This work presents a novel framework based on hybrid 2D-3D convolutional neural networks (CNNs) to obtain continuous 3D retinal layer surfaces from OCT volumes, which works well with both full and sparse annotations. The 2D features of individual B-scans are extracted by an encoder consisting of 2D convolutions. These 2D features are then used to produce the alignment displacement vectors and layer segmentation by two 3D decoders coupled via a spatial transformer module. Two losses are proposed to utilize the retinal layers' natural property of being smooth for B-scan alignment and layer segmentation, respectively, and are the key to the semi-supervised learning with sparse annotation. The entire framework is trained end-to-end. To the best of our knowledge, this is the first work that attempts 3D retinal layer segmentation in volumetric OCT images based on CNNs. Experiments on a synthetic dataset and three public clinical datasets show that our framework can effectively align the B-scans for potential motion correction, and achieves superior performance to state-of-the-art 2D deep learning methods in terms of both layer segmentation accuracy and cross-B-scan 3D continuity in both fully and semi-supervised settings, thus offering more clinical values than previous works.
Collapse
Affiliation(s)
- Hong Liu
- School of Informatics, Xiamen University, Xiamen 361005, China; National Institute for Data Science in Health and Medicine, Xiamen University, Xiamen 361005, China; Jarvis Research Center, Tencent YouTu Lab, Shenzhen 518075, China
| | - Dong Wei
- Jarvis Research Center, Tencent YouTu Lab, Shenzhen 518075, China
| | - Donghuan Lu
- Jarvis Research Center, Tencent YouTu Lab, Shenzhen 518075, China
| | - Xiaoying Tang
- Department of Electronic and Electrical Engineering, Southern University of Science and Technology, Shenzhen 518055, China
| | - Liansheng Wang
- School of Informatics, Xiamen University, Xiamen 361005, China.
| | - Yefeng Zheng
- Jarvis Research Center, Tencent YouTu Lab, Shenzhen 518075, China
| |
Collapse
|
4
|
Xie H, Pan Z, Xue CC, Chen D, Jonas JB, Wu X, Wang YX. Arterial hypertension and retinal layer thickness: the Beijing Eye Study. Br J Ophthalmol 2023; 108:105-111. [PMID: 36428008 DOI: 10.1136/bjo-2022-322229] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/14/2022] [Accepted: 11/11/2022] [Indexed: 11/26/2022]
Abstract
PURPOSE To investigate relationships between blood pressure and the thickness of single retinal layers in the macula. METHODS Participants of the population-based Beijing Eye Study, free of retinal or optic nerve disease, underwent medical and ophthalmological examinations including optical coherence tomographic examination of the macula. Applying a multiple-surface segmentation solution, we automatically segmented the retina into its various layers. RESULTS The study included 2237 participants (mean age 61.8±8.4 years, range 50-93 years). Mean thicknesses of the retinal nerve fibre layer (RNFL), ganglion cell layer (GCL), inner plexiform layer, inner nuclear layer (INL), outer plexiform layer, outer nuclear layer/external limiting membrane, ellipsoid zone, photoreceptor outer segments (POS) and retinal pigment epithelium-Bruch membrane were 31.1±2.3 µm, 39.7±3.5 µm, 38.4±3.3 µm, 34.8±2.0 µm, 28.1±3.0 µm, 79.2±7.3 µm, 22.9±0.6 µm, 19.2±3.3 µm and 20.7±1.4 µm, respectively. In multivariable analysis, higher systolic blood pressure (SBP) and diastolic blood pressure (DBP) were associated with thinner GCL and thicker INL, after adjusting for age, sex and axial length (all p<0.0056). Higher SBP was additionally associated with thinner POS and higher DBP with thinner RNFL. For an elevation of SBP/DBP by 10 mm Hg, the RNFL, GCL, INL and POS changed by 2.0, 3.0, 1.5 and 2.0 µm, respectively. CONCLUSIONS Thickness of RNFL, GCL and POS was inversely and INL thickness was positively associated with higher blood pressure, while the thickness of the other retinal layers was not significantly correlated with blood pressure. The findings may be helpful for refinement of the morphometric detection of retinal diseases.
Collapse
Affiliation(s)
- Hui Xie
- Department of Electrical and Computer Engineering, University of Iowa, Iowa City, Iowa, USA
| | - Zhe Pan
- Beijing Institute of Ophthalmology, Beijing Tongren Hospital, Capital University of Medical Science, Beijing Ophthalmology and Visual Sciences Key Laboratory, Beijing, China
- Department of Ophthalmology, Peking University Third Hospital, Beijing, China
| | - Can Can Xue
- Beijing Institute of Ophthalmology, Beijing Tongren Hospital, Capital University of Medical Science, Beijing Ophthalmology and Visual Sciences Key Laboratory, Beijing, China
- Department of Ophthalmology, Peking University Third Hospital, Beijing, China
| | - Danny Chen
- Department of Computer Science and Engineering, University of Notre Dame, Notre Dame, Indiana, USA
| | - Jost B Jonas
- Beijing Institute of Ophthalmology, Beijing Tongren Hospital, Capital University of Medical Science, Beijing Ophthalmology and Visual Sciences Key Laboratory, Beijing, China
- Ruprecht-Karls-University Heidelberg, Seegartenklinik Heidelberg, Heidelberg University, Heidelberg, Baden-Württemberg, Germany
- Institute of Clinical and Scientific Ophthalmology and Acupuncture Jonas & Panda, Heidelberg, Germany
- Institute of Molecular and Clinical Ophthalmology, Basel, Switzerland
| | - Xiaodong Wu
- Department of Electrical and Computer Engineering, University of Iowa, Iowa City, Iowa, USA
- Department of Radiation Oncology, University of Iowa, Iowa City, Iowa, USA
| | - Ya Xing Wang
- Beijing Institute of Ophthalmology, Beijing Tongren Hospital, Capital University of Medical Science, Beijing Ophthalmology and Visual Sciences Key Laboratory, Beijing, China
| |
Collapse
|
5
|
Wang YZ, Juroch K, Birch DG. Deep Learning-Assisted Measurements of Photoreceptor Ellipsoid Zone Area and Outer Segment Volume as Biomarkers for Retinitis Pigmentosa. Bioengineering (Basel) 2023; 10:1394. [PMID: 38135984 PMCID: PMC10740805 DOI: 10.3390/bioengineering10121394] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/30/2023] [Revised: 11/13/2023] [Accepted: 11/29/2023] [Indexed: 12/24/2023] Open
Abstract
The manual segmentation of retinal layers from OCT scan images is time-consuming and costly. The deep learning approach has potential for the automatic delineation of retinal layers to significantly reduce the burden of human graders. In this study, we compared deep learning model (DLM) segmentation with manual correction (DLM-MC) to conventional manual grading (MG) for the measurements of the photoreceptor ellipsoid zone (EZ) area and outer segment (OS) volume in retinitis pigmentosa (RP) to assess whether DLM-MC can be a new gold standard for retinal layer segmentation and for the measurement of retinal layer metrics. Ninety-six high-speed 9 mm 31-line volume scans obtained from 48 patients with RPGR-associated XLRP were selected based on the following criteria: the presence of an EZ band within the scan limit and a detectable EZ in at least three B-scans in a volume scan. All the B-scan images in each volume scan were manually segmented for the EZ and proximal retinal pigment epithelium (pRPE) by two experienced human graders to serve as the ground truth for comparison. The test volume scans were also segmented by a DLM and then manually corrected for EZ and pRPE by the same two graders to obtain DLM-MC segmentation. The EZ area and OS volume were determined by interpolating the discrete two-dimensional B-scan EZ-pRPE layer over the scan area. Dice similarity, Bland-Altman analysis, correlation, and linear regression analyses were conducted to assess the agreement between DLM-MC and MG for the EZ area and OS volume measurements. For the EZ area, the overall mean dice score (SD) between DLM-MC and MG was 0.8524 (0.0821), which was comparable to 0.8417 (0.1111) between two MGs. For the EZ area > 1 mm2, the average dice score increased to 0.8799 (0.0614). When comparing DLM-MC to MG, the Bland-Altman plots revealed a mean difference (SE) of 0.0132 (0.0953) mm2 and a coefficient of repeatability (CoR) of 1.8303 mm2 for the EZ area and a mean difference (SE) of 0.0080 (0.0020) mm3 and a CoR of 0.0381 mm3 for the OS volume. The correlation coefficients (95% CI) were 0.9928 (0.9892-0.9952) and 0.9938 (0.9906-0.9958) for the EZ area and OS volume, respectively. The linear regression slopes (95% CI) were 0.9598 (0.9399-0.9797) and 1.0104 (0.9909-1.0298), respectively. The results from this study suggest that the manual correction of deep learning model segmentation can generate EZ area and OS volume measurements in excellent agreement with those of conventional manual grading in RP. Because DLM-MC is more efficient for retinal layer segmentation from OCT scan images, it has the potential to reduce the burden of human graders in obtaining quantitative measurements of biomarkers for assessing disease progression and treatment outcomes in RP.
Collapse
Affiliation(s)
- Yi-Zhong Wang
- Retina Foundation of the Southwest, 9600 North Central Expressway, Suite 200, Dallas, TX 75231, USA; (K.J.); (D.G.B.)
- Department of Ophthalmology, University of Texas Southwestern Medical Center, 5323 Harry Hines Blvd, Dallas, TX 75390, USA
| | - Katherine Juroch
- Retina Foundation of the Southwest, 9600 North Central Expressway, Suite 200, Dallas, TX 75231, USA; (K.J.); (D.G.B.)
| | - David Geoffrey Birch
- Retina Foundation of the Southwest, 9600 North Central Expressway, Suite 200, Dallas, TX 75231, USA; (K.J.); (D.G.B.)
- Department of Ophthalmology, University of Texas Southwestern Medical Center, 5323 Harry Hines Blvd, Dallas, TX 75390, USA
| |
Collapse
|
6
|
Shen Y, Li J, Zhu W, Yu K, Wang M, Peng Y, Zhou Y, Guan L, Chen X. Graph Attention U-Net for Retinal Layer Surface Detection and Choroid Neovascularization Segmentation in OCT Images. IEEE TRANSACTIONS ON MEDICAL IMAGING 2023; 42:3140-3154. [PMID: 37022267 DOI: 10.1109/tmi.2023.3240757] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/19/2023]
Abstract
Choroidal neovascularization (CNV) is a typical symptom of age-related macular degeneration (AMD) and is one of the leading causes for blindness. Accurate segmentation of CNV and detection of retinal layers are critical for eye disease diagnosis and monitoring. In this paper, we propose a novel graph attention U-Net (GA-UNet) for retinal layer surface detection and CNV segmentation in optical coherence tomography (OCT) images. Due to retinal layer deformation caused by CNV, it is challenging for existing models to segment CNV and detect retinal layer surfaces with the correct topological order. We propose two novel modules to address the challenge. The first module is a graph attention encoder (GAE) in a U-Net model that automatically integrates topological and pathological knowledge of retinal layers into the U-Net structure to achieve effective feature embedding. The second module is a graph decorrelation module (GDM) that takes reconstructed features by the decoder of the U-Net as inputs, it then decorrelates and removes information unrelated to retinal layer for improved retinal layer surface detection. In addition, we propose a new loss function to maintain the correct topological order of retinal layers and the continuity of their boundaries. The proposed model learns graph attention maps automatically during training and performs retinal layer surface detection and CNV segmentation simultaneously with the attention maps during inference. We evaluated the proposed model on our private AMD dataset and another public dataset. Experiment results show that the proposed model outperformed the competing methods for retinal layer surface detection and CNV segmentation and achieved new state of the arts on the datasets.
Collapse
|
7
|
Gende M, Mallen V, de Moura J, Cordon B, Garcia-Martin E, Sanchez CI, Novo J, Ortega M. Automatic Segmentation of Retinal Layers in Multiple Neurodegenerative Disorder Scenarios. IEEE J Biomed Health Inform 2023; 27:5483-5494. [PMID: 37682646 DOI: 10.1109/jbhi.2023.3313392] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 09/10/2023]
Abstract
Retinal Optical Coherence Tomography (OCT) allows the non-invasive direct observation of the central nervous system, enabling the measurement and extraction of biomarkers from neural tissue that can be helpful in the assessment of ocular, systemic and Neurological Disorders (ND). Deep learning models can be trained to segment the retinal layers for biomarker extraction. However, the onset of ND can have an impact on the neural tissue, which can lead to the degraded performance of models not exposed to images displaying signs of disease during training. We present a fully automatic approach for the retinal layer segmentation in multiple neurodegenerative disorder scenarios, using an annotated dataset of patients of the most prevalent NDs: Alzheimer's disease, Parkinson's disease, multiple sclerosis and essential tremor, along with healthy control patients. Furthermore, we present a two-part, comprehensive study on the effects of ND on the performance of these models. The results show that images of healthy patients may not be sufficient for the robust training of automated segmentation models intended for the analysis of ND patients, and that using images representative of different NDs can increase the model performance. These results indicate that the presence or absence of patients of ND in datasets should be taken into account when training deep learning models for retinal layer segmentation, and that the proposed approach can provide a valuable tool for the robust and reliable diagnosis in multiple scenarios of ND.
Collapse
|
8
|
Xie H, Xu W, Wang YX, Wu X. Deep learning network with differentiable dynamic programming for retina OCT surface segmentation. BIOMEDICAL OPTICS EXPRESS 2023; 14:3190-3202. [PMID: 37497505 PMCID: PMC10368040 DOI: 10.1364/boe.492670] [Citation(s) in RCA: 3] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Received: 04/10/2023] [Revised: 05/19/2023] [Accepted: 05/23/2023] [Indexed: 07/28/2023]
Abstract
Multiple-surface segmentation in optical coherence tomography (OCT) images is a challenging problem, further complicated by the frequent presence of weak image boundaries. Recently, many deep learning-based methods have been developed for this task and yield remarkable performance. Unfortunately, due to the scarcity of training data in medical imaging, it is challenging for deep learning networks to learn the global structure of the target surfaces, including surface smoothness. To bridge this gap, this study proposes to seamlessly unify a U-Net for feature learning with a constrained differentiable dynamic programming module to achieve end-to-end learning for retina OCT surface segmentation to explicitly enforce surface smoothness. It effectively utilizes the feedback from the downstream model optimization module to guide feature learning, yielding better enforcement of global structures of the target surfaces. Experiments on Duke AMD (age-related macular degeneration) and JHU MS (multiple sclerosis) OCT data sets for retinal layer segmentation demonstrated that the proposed method was able to achieve subvoxel accuracy on both datasets, with the mean absolute surface distance (MASD) errors of 1.88 ± 1.96μm and 2.75 ± 0.94μm, respectively, over all the segmented surfaces.
Collapse
Affiliation(s)
- Hui Xie
- Department of Electrical and Computer Engineering, The University of Iowa, Iowa City, IA, USA
| | - Weiyu Xu
- Department of Electrical and Computer Engineering, The University of Iowa, Iowa City, IA, USA
| | - Ya Xing Wang
- Beijing Institute of Ophthalmology, Beijing Tongren Hospital, Capital University of Medical Science, Beijing Ophthalmology and Visual Sciences Key Laboratory, Beijing, China
| | - Xiaodong Wu
- Department of Electrical and Computer Engineering, The University of Iowa, Iowa City, IA, USA
| |
Collapse
|
9
|
Morelle O, Wintergerst MWM, Finger RP, Schultz T. Accurate drusen segmentation in optical coherence tomography via order-constrained regression of retinal layer heights. Sci Rep 2023; 13:8162. [PMID: 37208407 DOI: 10.1038/s41598-023-35230-4] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/26/2022] [Accepted: 05/15/2023] [Indexed: 05/21/2023] Open
Abstract
Drusen are an important biomarker for age-related macular degeneration (AMD). Their accurate segmentation based on optical coherence tomography (OCT) is therefore relevant to the detection, staging, and treatment of disease. Since manual OCT segmentation is resource-consuming and has low reproducibility, automatic techniques are required. In this work, we introduce a novel deep learning based architecture that directly predicts the position of layers in OCT and guarantees their correct order, achieving state-of-the-art results for retinal layer segmentation. In particular, the average absolute distance between our model's prediction and the ground truth layer segmentation in an AMD dataset is 0.63, 0.85, and 0.44 pixel for Bruch's membrane (BM), retinal pigment epithelium (RPE) and ellipsoid zone (EZ), respectively. Based on layer positions, we further quantify drusen load with excellent accuracy, achieving 0.994 and 0.988 Pearson correlation between drusen volumes estimated by our method and two human readers, and increasing the Dice score to 0.71 ± 0.16 (from 0.60 ± 0.23) and 0.62 ± 0.23 (from 0.53 ± 0.25), respectively, compared to a previous state-of-the-art method. Given its reproducible, accurate, and scalable results, our method can be used for the large-scale analysis of OCT data.
Collapse
Affiliation(s)
- Olivier Morelle
- B-IT and Department of Computer Science, University of Bonn, 53115, Bonn, Germany
- Department of Ophthalmology, University Hospital Bonn, 53127, Bonn, Germany
| | | | - Robert P Finger
- Department of Ophthalmology, University Hospital Bonn, 53127, Bonn, Germany
| | - Thomas Schultz
- B-IT and Department of Computer Science, University of Bonn, 53115, Bonn, Germany.
- Lamarr Institute for Machine Learning and Artificial Intelligence, .
| |
Collapse
|
10
|
Exploring healthy retinal aging with deep learning. OPHTHALMOLOGY SCIENCE 2023; 3:100294. [PMID: 37113474 PMCID: PMC10127123 DOI: 10.1016/j.xops.2023.100294] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Received: 10/30/2022] [Revised: 01/24/2023] [Accepted: 02/17/2023] [Indexed: 03/04/2023]
Abstract
Purpose To study the individual course of retinal changes caused by healthy aging using deep learning. Design Retrospective analysis of a large data set of retinal OCT images. Participants A total of 85 709 adults between the age of 40 and 75 years of whom OCT images were acquired in the scope of the UK Biobank population study. Methods We created a counterfactual generative adversarial network (GAN), a type of neural network that learns from cross-sectional, retrospective data. It then synthesizes high-resolution counterfactual OCT images and longitudinal time series. These counterfactuals allow visualization and analysis of hypothetical scenarios in which certain characteristics of the imaged subject, such as age or sex, are altered, whereas other attributes, crucially the subject's identity and image acquisition settings, remain fixed. Main Outcome Measures Using our counterfactual GAN, we investigated subject-specific changes in the retinal layer structure as a function of age and sex. In particular, we measured changes in the retinal nerve fiber layer (RNFL), combined ganglion cell layer plus inner plexiform layer (GCIPL), inner nuclear layer to the inner boundary of the retinal pigment epithelium (INL-RPE), and retinal pigment epithelium (RPE). Results Our counterfactual GAN is able to smoothly visualize the individual course of retinal aging. Across all counterfactual images, the RNFL, GCIPL, INL-RPE, and RPE changed by -0.1 μm ± 0.1 μm, -0.5 μm ± 0.2 μm, -0.2 μm ± 0.1 μm, and 0.1 μm ± 0.1 μm, respectively, per decade of age. These results agree well with previous studies based on the same cohort from the UK Biobank population study. Beyond population-wide average measures, our counterfactual GAN allows us to explore whether the retinal layers of a given eye will increase in thickness, decrease in thickness, or stagnate as a subject ages. Conclusion This study demonstrates how counterfactual GANs can aid research into retinal aging by generating high-resolution, high-fidelity OCT images, and longitudinal time series. Ultimately, we envision that they will enable clinical experts to derive and explore hypotheses for potential imaging biomarkers for healthy and pathologic aging that can be refined and tested in prospective clinical trials. Financial Disclosures Proprietary or commercial disclosure may be found after the references.
Collapse
|
11
|
Liu H, Li X, Bamba AL, Song X, Brott BC, Litovsky SH, Gan Y. Toward reliable calcification detection: calibration of uncertainty in object detection from coronary optical coherence tomography images. JOURNAL OF BIOMEDICAL OPTICS 2023; 28:036008. [PMID: 36992694 PMCID: PMC10042069 DOI: 10.1117/1.jbo.28.3.036008] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 11/21/2022] [Accepted: 03/06/2023] [Indexed: 06/19/2023]
Abstract
SIGNIFICANCE Optical coherence tomography (OCT) has become increasingly essential in assisting the treatment of coronary artery disease (CAD). However, unidentified calcified regions within a narrowed artery could impair the outcome of the treatment. Fast and objective identification is paramount to automatically procuring accurate readings on calcifications within the artery. AIM We aim to rapidly identify calcification in coronary OCT images using a bounding box and reduce the prediction bias in automated prediction models. APPROACH We first adopt a deep learning-based object detection model to rapidly draw the calcified region from coronary OCT images using a bounding box. We measure the uncertainty of predictions based on the expected calibration errors, thus assessing the certainty level of detection results. To calibrate confidence scores of predictions, we implement dependent logistic calibration using each detection result's confidence and center coordinates. RESULTS We implemented an object detection module to draw the boundary of the calcified region at a rate of 140 frames per second. With the calibrated confidence score of each prediction, we lower the uncertainty of predictions in calcification detection and eliminate the estimation bias from various object detection methods. The calibrated confidence of prediction results in a confidence error of ∼ 0.13 , suggesting that the confidence calibration on calcification detection could provide a more trustworthy result. CONCLUSIONS Given the rapid detection and effective calibration of the proposed work, we expect that it can assist in clinical evaluation of treating the CAD during the imaging-guided procedure.
Collapse
Affiliation(s)
- Hongshan Liu
- Stevens Institute of Technology, Biomedical Engineering Department, Hoboken, New Jersey, United States
| | - Xueshen Li
- Stevens Institute of Technology, Biomedical Engineering Department, Hoboken, New Jersey, United States
| | - Abdul Latif Bamba
- Columbia University, Department of Electrical Engineering, New York, United States
| | - Xiaoyu Song
- Icahn School of Medicine at Mount Sinai, New York, United States
| | - Brigitta C. Brott
- University of Alabama at Birmingham, School of Medicine, Birmingham, Alabama, United States
| | - Silvio H. Litovsky
- University of Alabama at Birmingham, School of Medicine, Birmingham, Alabama, United States
| | - Yu Gan
- Stevens Institute of Technology, Biomedical Engineering Department, Hoboken, New Jersey, United States
| |
Collapse
|
12
|
Lou S, Chen X, Wang Y, Cai H, Chen S, Liu L. Multiscale joint segmentation method for retinal optical coherence tomography images using a bidirectional wave algorithm and improved graph theory. OPTICS EXPRESS 2023; 31:6862-6876. [PMID: 36823933 DOI: 10.1364/oe.472154] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 08/01/2022] [Accepted: 12/16/2022] [Indexed: 06/18/2023]
Abstract
Morphology and functional metrics of retinal layers are important biomarkers for many human ophthalmic diseases. Automatic and accurate segmentation of retinal layers is crucial for disease diagnosis and research. To improve the performance of retinal layer segmentation, a multiscale joint segmentation framework for retinal optical coherence tomography (OCT) images based on bidirectional wave algorithm and improved graph theory is proposed. In this framework, the bidirectional wave algorithm was used to segment edge information in multiscale images, and the improved graph theory was used to modify edge information globally, to realize automatic and accurate segmentation of eight retinal layer boundaries. This framework was tested on two public datasets and two OCT imaging systems. The test results show that, compared with other state-of-the-art methods, this framework does not need data pre-training and parameter pre-adjustment on different datasets, and can achieve sub-pixel retinal layer segmentation on a low-configuration computer.
Collapse
|
13
|
Bao D, Wang L, Zhou X, Yang S, He K, Xu M. Automated detection and growth tracking of 3D bio-printed organoid clusters using optical coherence tomography with deep convolutional neural networks. Front Bioeng Biotechnol 2023; 11:1133090. [PMID: 37122853 PMCID: PMC10130530 DOI: 10.3389/fbioe.2023.1133090] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/28/2022] [Accepted: 03/31/2023] [Indexed: 05/02/2023] Open
Abstract
Organoids are advancing the development of accurate prediction of drug efficacy and toxicity in vitro. These advancements are attributed to the ability of organoids to recapitulate key structural and functional features of organs and parent tumor. Specifically, organoids are self-organized assembly with a multi-scale structure of 30-800 μm, which exacerbates the difficulty of non-destructive three-dimensional (3D) imaging, tracking and classification analysis for organoid clusters by traditional microscopy techniques. Here, we devise a 3D imaging, segmentation and analysis method based on Optical coherence tomography (OCT) technology and deep convolutional neural networks (CNNs) for printed organoid clusters (Organoid Printing and optical coherence tomography-based analysis, OPO). The results demonstrate that the organoid scale influences the segmentation effect of the neural network. The multi-scale information-guided optimized EGO-Net we designed achieves the best results, especially showing better recognition workout for the biologically significant organoid with diameter ≥50 μm than other neural networks. Moreover, OPO achieves to reconstruct the multiscale structure of organoid clusters within printed microbeads and calibrate the printing errors by segmenting the printed microbeads edges. Overall, the classification, tracking and quantitative analysis based on image reveal that the growth process of organoid undergoes morphological changes such as volume growth, cavity creation and fusion, and quantitative calculation of the volume demonstrates that the growth rate of organoid is associated with the initial scale. The new method we proposed enable the study of growth, structural evolution and heterogeneity for the organoid cluster, which is valuable for drug screening and tumor drug sensitivity detection based on organoids.
Collapse
Affiliation(s)
- Di Bao
- School of Automation, Hangzhou Dianzi University, Hangzhou, China
| | - Ling Wang
- School of Automation, Hangzhou Dianzi University, Hangzhou, China
- Key Laboratory of Medical Information and 3D Bioprinting of Zhejiang Province, Hangzhou, China
- *Correspondence: Ling Wang, ; Mingen Xu,
| | - Xiaofei Zhou
- School of Automation, Hangzhou Dianzi University, Hangzhou, China
- Key Laboratory of Medical Information and 3D Bioprinting of Zhejiang Province, Hangzhou, China
| | - Shanshan Yang
- School of Automation, Hangzhou Dianzi University, Hangzhou, China
- Key Laboratory of Medical Information and 3D Bioprinting of Zhejiang Province, Hangzhou, China
| | - Kangxin He
- Key Laboratory of Medical Information and 3D Bioprinting of Zhejiang Province, Hangzhou, China
| | - Mingen Xu
- School of Automation, Hangzhou Dianzi University, Hangzhou, China
- Key Laboratory of Medical Information and 3D Bioprinting of Zhejiang Province, Hangzhou, China
- *Correspondence: Ling Wang, ; Mingen Xu,
| |
Collapse
|
14
|
Multi-layer segmentation of retina OCT images via advanced U-net architecture. Neurocomputing 2023. [DOI: 10.1016/j.neucom.2022.10.001] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/06/2022]
|
15
|
Schottenhamml J, Hohberger B, Mardin CY. Applications of Artificial Intelligence in Optical Coherence Tomography Angiography Imaging. Klin Monbl Augenheilkd 2022; 239:1412-1426. [PMID: 36493762 DOI: 10.1055/a-1961-7137] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/13/2022]
Abstract
Optical coherence tomography angiography (OCTA) and artificial intelligence (AI) are two emerging fields that complement each other. OCTA enables the noninvasive, in vivo, 3D visualization of retinal blood flow with a micrometer resolution, which has been impossible with other imaging modalities. As it does not need dye-based injections, it is also a safer procedure for patients. AI has excited great interest in many fields of daily life, by enabling automatic processing of huge amounts of data with a performance that greatly surpasses previous algorithms. It has been used in many breakthrough studies in recent years, such as the finding that AlphaGo can beat humans in the strategic board game of Go. This paper will give a short introduction into both fields and will then explore the manifold applications of AI in OCTA imaging that have been presented in the recent years. These range from signal generation over signal enhancement to interpretation tasks like segmentation and classification. In all these areas, AI-based algorithms have achieved state-of-the-art performance that has the potential to improve standard care in ophthalmology when integrated into the daily clinical routine.
Collapse
Affiliation(s)
- Julia Schottenhamml
- Augenklinik, Friedrich-Alexander-Universität Erlangen-Nürnberg, Erlangen, Germany
| | - Bettina Hohberger
- Augenklinik, Friedrich-Alexander-Universität Erlangen-Nürnberg, Erlangen, Germany
| | | |
Collapse
|
16
|
|
17
|
Mukherjee S, De Silva T, Grisso P, Wiley H, Tiarnan DLK, Thavikulwat AT, Chew E, Cukras C. Retinal layer segmentation in optical coherence tomography (OCT) using a 3D deep-convolutional regression network for patients with age-related macular degeneration. BIOMEDICAL OPTICS EXPRESS 2022; 13:3195-3210. [PMID: 35781941 PMCID: PMC9208604 DOI: 10.1364/boe.450193] [Citation(s) in RCA: 13] [Impact Index Per Article: 6.5] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Received: 01/03/2022] [Revised: 03/11/2022] [Accepted: 03/11/2022] [Indexed: 05/25/2023]
Abstract
Introduction - Retinal layer segmentation in optical coherence tomography (OCT) images is an important approach for detecting and prognosing disease. Automating segmentation using robust machine learning techniques lead to computationally efficient solutions and significantly reduces the cost of labor-intensive labeling, which is traditionally performed by trained graders at a reading center, sometimes aided by semi-automated algorithms. Although several algorithms have been proposed since the revival of deep learning, eyes with severe pathological conditions continue to challenge fully automated segmentation approaches. There remains an opportunity to leverage the underlying spatial correlations between the retinal surfaces in the segmentation approach. Methods - Some of these proposed traditional methods can be expanded to utilize the three-dimensional spatial context governing the retinal image volumes by replacing the use of 2D filters with 3D filters. Towards this purpose, we propose a spatial-context, continuity and anatomical relationship preserving semantic segmentation algorithm, which utilizes the 3D spatial context from the image volumes with the use of 3D filters. We propose a 3D deep neural network capable of learning the surface positions of the layers in the retinal volumes. Results - We utilize a dataset of OCT images from patients with Age-related Macular Degeneration (AMD) to assess performance of our model and provide both qualitative (including segmentation maps and thickness maps) and quantitative (including error metric comparisons and volumetric comparisons) results, which demonstrate that our proposed method performs favorably even for eyes with pathological changes caused by severe retinal diseases. The Mean Absolute Error (MAE) and Root Mean Squared Error (RMSE) for patients with a wide range of AMD severity scores (0-11) were within 0.84±0.41 and 1.33±0.73 pixels, respectively, which are significantly better than some of the other state-of-the-art algorithms. Conclusion - The results demonstrate the utility of extracting features from the entire OCT volume by treating the volume as a correlated entity and show the benefit of utilizing 3D autoencoder based regression networks for smoothing the approximated retinal layers by inducing shape based regularization constraints.
Collapse
Affiliation(s)
- Souvick Mukherjee
- Unit on Clinical Investigation
of Retinal Disease, 10 Center Drive, Building 10-CRC
Room 3-2531, MD 20892-1204, USA
| | - Tharindu De Silva
- Unit on Clinical Investigation
of Retinal Disease, 10 Center Drive, Building 10-CRC
Room 3-2531, MD 20892-1204, USA
| | - Peyton Grisso
- Unit on Clinical Investigation
of Retinal Disease, 10 Center Drive, Building 10-CRC
Room 3-2531, MD 20892-1204, USA
| | - Henry Wiley
- Division of Epidemiology and Clinical
Applications in National Eye Institute, National
Institutes of Health, Bethesda, MD 20892-4874, USA
| | - D. L. Keenan Tiarnan
- Division of Epidemiology and Clinical
Applications in National Eye Institute, National
Institutes of Health, Bethesda, MD 20892-4874, USA
| | - Alisa T Thavikulwat
- Division of Epidemiology and Clinical
Applications in National Eye Institute, National
Institutes of Health, Bethesda, MD 20892-4874, USA
| | - Emily Chew
- Division of Epidemiology and Clinical
Applications in National Eye Institute, National
Institutes of Health, Bethesda, MD 20892-4874, USA
| | - Catherine Cukras
- Unit on Clinical Investigation
of Retinal Disease, 10 Center Drive, Building 10-CRC
Room 3-2531, MD 20892-1204, USA
| |
Collapse
|
18
|
Yadav SK, Kafieh R, Zimmermann HG, Kauer-Bonin J, Nouri-Mahdavi K, Mohammadzadeh V, Shi L, Kadas EM, Paul F, Motamedi S, Brandt AU. Intraretinal Layer Segmentation Using Cascaded Compressed U-Nets. J Imaging 2022; 8:139. [PMID: 35621903 PMCID: PMC9146486 DOI: 10.3390/jimaging8050139] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/31/2022] [Revised: 04/23/2022] [Accepted: 05/03/2022] [Indexed: 12/24/2022] Open
Abstract
Reliable biomarkers quantifying neurodegeneration and neuroinflammation in central nervous system disorders such as Multiple Sclerosis, Alzheimer's dementia or Parkinson's disease are an unmet clinical need. Intraretinal layer thicknesses on macular optical coherence tomography (OCT) images are promising noninvasive biomarkers querying neuroretinal structures with near cellular resolution. However, changes are typically subtle, while tissue gradients can be weak, making intraretinal segmentation a challenging task. A robust and efficient method that requires no or minimal manual correction is an unmet need to foster reliable and reproducible research as well as clinical application. Here, we propose and validate a cascaded two-stage network for intraretinal layer segmentation, with both networks being compressed versions of U-Net (CCU-INSEG). The first network is responsible for retinal tissue segmentation from OCT B-scans. The second network segments eight intraretinal layers with high fidelity. At the post-processing stage, we introduce Laplacian-based outlier detection with layer surface hole filling by adaptive non-linear interpolation. Additionally, we propose a weighted version of focal loss to minimize the foreground-background pixel imbalance in the training data. We train our method using 17,458 B-scans from patients with autoimmune optic neuropathies, i.e., multiple sclerosis, and healthy controls. Voxel-wise comparison against manual segmentation produces a mean absolute error of 2.3 μm, outperforming current state-of-the-art methods on the same data set. Voxel-wise comparison against external glaucoma data leads to a mean absolute error of 2.6 μm when using the same gold standard segmentation approach, and 3.7 μm mean absolute error in an externally segmented data set. In scans from patients with severe optic atrophy, 3.5% of B-scan segmentation results were rejected by an experienced grader, whereas this was the case in 41.4% of B-scans segmented with a graph-based reference method. The validation results suggest that the proposed method can robustly segment macular scans from eyes with even severe neuroretinal changes.
Collapse
Affiliation(s)
- Sunil Kumar Yadav
- Experimental and Clinical Research Center, Max Delbrück Center for Molecular Medicine and Charité-Universitätsmedizin Berlin, Corporate Member of Freie Universität Berlin and Humboldt-Universität zu Berlin, 13125 Berlin, Germany; (S.K.Y.); (R.K.); (H.G.Z.); (J.K.-B.); (F.P.); (S.M.)
- Nocturne GmbH, 10119 Berlin, Germany;
| | - Rahele Kafieh
- Experimental and Clinical Research Center, Max Delbrück Center for Molecular Medicine and Charité-Universitätsmedizin Berlin, Corporate Member of Freie Universität Berlin and Humboldt-Universität zu Berlin, 13125 Berlin, Germany; (S.K.Y.); (R.K.); (H.G.Z.); (J.K.-B.); (F.P.); (S.M.)
| | - Hanna Gwendolyn Zimmermann
- Experimental and Clinical Research Center, Max Delbrück Center for Molecular Medicine and Charité-Universitätsmedizin Berlin, Corporate Member of Freie Universität Berlin and Humboldt-Universität zu Berlin, 13125 Berlin, Germany; (S.K.Y.); (R.K.); (H.G.Z.); (J.K.-B.); (F.P.); (S.M.)
| | - Josef Kauer-Bonin
- Experimental and Clinical Research Center, Max Delbrück Center for Molecular Medicine and Charité-Universitätsmedizin Berlin, Corporate Member of Freie Universität Berlin and Humboldt-Universität zu Berlin, 13125 Berlin, Germany; (S.K.Y.); (R.K.); (H.G.Z.); (J.K.-B.); (F.P.); (S.M.)
- Nocturne GmbH, 10119 Berlin, Germany;
| | - Kouros Nouri-Mahdavi
- Glaucoma Division, Stein Eye Institute, David Geffen School of Medicine, University of California Los Angeles, Los Angeles, CA 90095, USA; (K.N.-M.); (V.M.); (L.S.)
| | - Vahid Mohammadzadeh
- Glaucoma Division, Stein Eye Institute, David Geffen School of Medicine, University of California Los Angeles, Los Angeles, CA 90095, USA; (K.N.-M.); (V.M.); (L.S.)
| | - Lynn Shi
- Glaucoma Division, Stein Eye Institute, David Geffen School of Medicine, University of California Los Angeles, Los Angeles, CA 90095, USA; (K.N.-M.); (V.M.); (L.S.)
| | | | - Friedemann Paul
- Experimental and Clinical Research Center, Max Delbrück Center for Molecular Medicine and Charité-Universitätsmedizin Berlin, Corporate Member of Freie Universität Berlin and Humboldt-Universität zu Berlin, 13125 Berlin, Germany; (S.K.Y.); (R.K.); (H.G.Z.); (J.K.-B.); (F.P.); (S.M.)
- Department of Neurology, Charité-Universitätsmedizin Berlin, Corporate Member of Freie Universität Berlin and Humboldt-Universität zu Berlin, 10098 Berlin, Germany
| | - Seyedamirhosein Motamedi
- Experimental and Clinical Research Center, Max Delbrück Center for Molecular Medicine and Charité-Universitätsmedizin Berlin, Corporate Member of Freie Universität Berlin and Humboldt-Universität zu Berlin, 13125 Berlin, Germany; (S.K.Y.); (R.K.); (H.G.Z.); (J.K.-B.); (F.P.); (S.M.)
| | - Alexander Ulrich Brandt
- Experimental and Clinical Research Center, Max Delbrück Center for Molecular Medicine and Charité-Universitätsmedizin Berlin, Corporate Member of Freie Universität Berlin and Humboldt-Universität zu Berlin, 13125 Berlin, Germany; (S.K.Y.); (R.K.); (H.G.Z.); (J.K.-B.); (F.P.); (S.M.)
- Department of Neurology, University of California Irvine, Irvine, CA 92697, USA
| |
Collapse
|
19
|
Yang J, Tao Y, Xu Q, Zhang Y, Ma X, Yuan S, Chen Q. Self-Supervised Sequence Recovery for Semi-Supervised Retinal Layer Segmentation. IEEE J Biomed Health Inform 2022; 26:3872-3883. [PMID: 35412994 DOI: 10.1109/jbhi.2022.3166778] [Citation(s) in RCA: 6] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/06/2022]
Abstract
Automated layer segmentation plays an important role for retinal disease diagnosis in optical coherence tomography (OCT) images. However, the severe retinal diseases result in the performance degeneration of automated layer segmentation approaches. In this paper, we present a robust semi-supervised retinal layer segmentation network to relieve the model failures on abnormal retinas, in which we obtain the lesion features from the labeled images with disease-balanced distribution, and utilize the unlabeled images to supplement the layer structure information. Specifically, in our proposed method, the cross-consistency training is utilized over the predictions of the different decoders, and we enforce a consistency between different decoder predictions to improve the encoders representation. Then, we proposed a sequence prediction branch based on self-supervised manner, which is designed to predict the position of each jigsaw puzzle to obtain sensory perception of the retinal layer structure. To this task, a layer spatial pyramid pooling (LSPP) module is designed to extract multi-scale layer spatial features. Furthermore, we use the optical coherence tomography angiography (OCTA) to supplement the information damaged by diseases. The experimental results validate that our method achieves more robust results compared with current supervised segmentation methods. Meanwhile, advanced segmentation performance can be obtained compared with state-of-the-art semi-supervised segmentation methods.
Collapse
|
20
|
Sommersperger M, Martin-Gomez A, Mach K, Gehlbach PL, Ali Nasseri M, Iordachita I, Navab N. Surgical scene generation and adversarial networks for physics-based iOCT synthesis. BIOMEDICAL OPTICS EXPRESS 2022; 13:2414-2430. [PMID: 35519277 PMCID: PMC9045909 DOI: 10.1364/boe.454286] [Citation(s) in RCA: 2] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Received: 01/19/2022] [Revised: 02/17/2022] [Accepted: 03/11/2022] [Indexed: 06/14/2023]
Abstract
The development and integration of intraoperative optical coherence tomography (iOCT) into modern operating rooms has motivated novel procedures directed at improving the outcome of ophthalmic surgeries. Although computer-assisted algorithms could further advance such interventions, the limited availability and accessibility of iOCT systems constrains the generation of dedicated data sets. This paper introduces a novel framework combining a virtual setup and deep learning algorithms to generate synthetic iOCT data in a simulated environment. The virtual setup reproduces the geometry of retinal layers extracted from real data and allows the integration of virtual microsurgical instrument models. Our scene rendering approach extracts information from the environment and considers iOCT typical imaging artifacts to generate cross-sectional label maps, which in turn are used to synthesize iOCT B-scans via a generative adversarial network. In our experiments we investigate the similarity between real and synthetic images, show the relevance of using the generated data for image-guided interventions and demonstrate the potential of 3D iOCT data synthesis.
Collapse
Affiliation(s)
- Michael Sommersperger
- Chair for Computer Aided Medical Procedures and Augmented Reality, Informatics Department, Technical University of Munich, Munich, Bayern, Germany
- These authors contributed equally to this work
| | - Alejandro Martin-Gomez
- Chair for Computer Aided Medical Procedures and Augmented Reality, Informatics Department, Technical University of Munich, Munich, Bayern, Germany
- Laboratory for Computational Sensing and Robotics, Whiting School of Engineering, Johns Hopkins University, Baltimore, Maryland, USA
- These authors contributed equally to this work
| | - Kristina Mach
- Chair for Computer Aided Medical Procedures and Augmented Reality, Informatics Department, Technical University of Munich, Munich, Bayern, Germany
| | - Peter Louis Gehlbach
- Wilmer Eye Institute Research, School of Medicine, Johns Hopkins University, Baltimore, Maryland, USA
| | - M Ali Nasseri
- Klinikum Rechts der Isar, Augenklinik, Munich, Bayern, Germany
| | - Iulian Iordachita
- Laboratory for Computational Sensing and Robotics, Whiting School of Engineering, Johns Hopkins University, Baltimore, Maryland, USA
| | - Nassir Navab
- Chair for Computer Aided Medical Procedures and Augmented Reality, Informatics Department, Technical University of Munich, Munich, Bayern, Germany
- Laboratory for Computational Sensing and Robotics, Whiting School of Engineering, Johns Hopkins University, Baltimore, Maryland, USA
| |
Collapse
|
21
|
A Deep Learning Framework for Image Authentication: An Automatic Source Camera Identification Deep-Net. ARABIAN JOURNAL FOR SCIENCE AND ENGINEERING 2022. [DOI: 10.1007/s13369-022-06743-3] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 11/02/2022]
|
22
|
Xie H, Pan Z, Zhou L, Zaman FA, Chen DZ, Jonas JB, Xu W, Wang YX, Wu X. Globally optimal OCT surface segmentation using a constrained IPM optimization. OPTICS EXPRESS 2022; 30:2453-2471. [PMID: 35209385 DOI: 10.1364/oe.444369] [Citation(s) in RCA: 5] [Impact Index Per Article: 2.5] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Received: 09/27/2021] [Accepted: 01/01/2022] [Indexed: 06/14/2023]
Abstract
Segmentation of multiple surfaces in optical coherence tomography (OCT) images is a challenging problem, further complicated by the frequent presence of weak boundaries, varying layer thicknesses, and mutual influence between adjacent surfaces. The traditional graph-based optimal surface segmentation method has proven its effectiveness with its ability to capture various surface priors in a uniform graph model. However, its efficacy heavily relies on handcrafted features that are used to define the surface cost for the "goodness" of a surface. Recently, deep learning (DL) is emerging as a powerful tool for medical image segmentation thanks to its superior feature learning capability. Unfortunately, due to the scarcity of training data in medical imaging, it is nontrivial for DL networks to implicitly learn the global structure of the target surfaces, including surface interactions. This study proposes to parameterize the surface cost functions in the graph model and leverage DL to learn those parameters. The multiple optimal surfaces are then simultaneously detected by minimizing the total surface cost while explicitly enforcing the mutual surface interaction constraints. The optimization problem is solved by the primal-dual interior-point method (IPM), which can be implemented by a layer of neural networks, enabling efficient end-to-end training of the whole network. Experiments on spectral-domain optical coherence tomography (SD-OCT) retinal layer segmentation demonstrated promising segmentation results with sub-pixel accuracy.
Collapse
|
23
|
Zhang ZB, Zou YN, Huang YL, Li Q. CT image crack segmentation method based on linear feature enhancement. JOURNAL OF X-RAY SCIENCE AND TECHNOLOGY 2022; 30:903-917. [PMID: 35723166 DOI: 10.3233/xst-221171] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/15/2023]
Abstract
Industrial computed tomography (CT) crack segmentation is a key technology in industrial CT image processing. Unfortunately, the interference of artifact and noise in CT image often bring great trouble to the crack segmentation. In order to improve the segmentation accuracy of cracks in CT images, we propose to develop and test a new crack segmentation algorithm based on linear feature enhancement by analyzing the features of cracks in CT images. Firstly, the total variational model is used to denoise the input image. Next, a Frangi multiscale filter is used to extract linear structures in the image, and then the extracted linear structures are used to enhance the contrast of the image. Finally, the cracks in the image are detected and segmented by Otsu algorithm. By comparing with the manual segmentation results, the average intersection-over-union (IOU) reaches 86.10% and the average F1 score reaches 92.44%, which verifies the effectiveness and correctness of the algorithm developed in this study. Overall, experiments demonstrate that the new algorithm improves the accuracy of crack segmentation and it is effective applying to industry CT images.
Collapse
Affiliation(s)
- Zhi-Bin Zhang
- Engineering Research Center of Industrial Computed Tomography Nondestructive Testing, Ministry of Education, Chongqing University, Chongqing, China
- College of Optoelectronic Engineering, Chongqing University, Chongqing, China
| | - Yong-Ning Zou
- Engineering Research Center of Industrial Computed Tomography Nondestructive Testing, Ministry of Education, Chongqing University, Chongqing, China
- College of Optoelectronic Engineering, Chongqing University, Chongqing, China
| | - Ye-Ling Huang
- Engineering Research Center of Industrial Computed Tomography Nondestructive Testing, Ministry of Education, Chongqing University, Chongqing, China
- College of Optoelectronic Engineering, Chongqing University, Chongqing, China
| | - Qi Li
- Engineering Research Center of Industrial Computed Tomography Nondestructive Testing, Ministry of Education, Chongqing University, Chongqing, China
- College of Optoelectronic Engineering, Chongqing University, Chongqing, China
| |
Collapse
|
24
|
Gong D, Aronow ME, Eliott D. Rapid, Spontaneous Resolution of Prominent Subretinal Infiltrate in Vitreoretinal Lymphoma. JOURNAL OF VITREORETINAL DISEASES 2022; 6:80-85. [PMID: 37007723 PMCID: PMC9976220 DOI: 10.1177/24741264211009804] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 11/17/2022]
Abstract
Purpose: This case report describes a patient with vitreoretinal lymphoma who subacutely developed a large, peripapillary subretinal infiltrate that rapidly and spontaneously resolved. Methods: A case report is presented. Results: A 65-year-old Asian-American woman was referred for evaluation of a dense, peripapillary subretinal infiltrate in the left eye. A diagnostic vitrectomy revealed large, atypical lymphocytes with irregularly shaped nuclei, and mutational testing was positive for myeloid differentiation primary response 88 ( MYD88). Prior to surgery, the patient’s subretinal infiltrate had begun to resolve spontaneously, a process that continued after surgery without initiation of systemic or local ocular therapy. Conclusions: Patients with vitreoretinal lymphoma may present with transient, subretinal infiltrates that can resolve without treatment.
Collapse
Affiliation(s)
- Dan Gong
- Retina Service, Massachusetts Eye and Ear, Department of Ophthalmology, Harvard Medical School, Boston, MA, USA
| | - Mary E. Aronow
- Retina Service, Massachusetts Eye and Ear, Department of Ophthalmology, Harvard Medical School, Boston, MA, USA
| | - Dean Eliott
- Retina Service, Massachusetts Eye and Ear, Department of Ophthalmology, Harvard Medical School, Boston, MA, USA
| |
Collapse
|
25
|
Stankiewicz A, Marciniak T, Dabrowski A, Stopa M, Marciniak E, Obara B. Segmentation of Preretinal Space in Optical Coherence Tomography Images Using Deep Neural Networks. SENSORS 2021; 21:s21227521. [PMID: 34833597 PMCID: PMC8623441 DOI: 10.3390/s21227521] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 09/28/2021] [Revised: 11/08/2021] [Accepted: 11/09/2021] [Indexed: 02/01/2023]
Abstract
This paper proposes an efficient segmentation of the preretinal area between the inner limiting membrane (ILM) and posterior cortical vitreous (PCV) of the human eye in an image obtained with the use of optical coherence tomography (OCT). The research was carried out using a database of three-dimensional OCT imaging scans obtained with the Optovue RTVue XR Avanti device. Various types of neural networks (UNet, Attention UNet, ReLayNet, LFUNet) were tested for semantic segmentation, their effectiveness was assessed using the Dice coefficient and compared to the graph theory techniques. Improvement in segmentation efficiency was achieved through the use of relative distance maps. We also show that selecting a larger kernel size for convolutional layers can improve segmentation quality depending on the neural network model. In the case of PVC, we obtain the effectiveness reaching up to 96.35%. The proposed solution can be widely used to diagnose vitreomacular traction changes, which is not yet available in scientific or commercial OCT imaging solutions.
Collapse
Affiliation(s)
- Agnieszka Stankiewicz
- Division of Electronic Systems and Signal Processing, Institute of Automatic Control and Robotics, Poznan University of Technology, 60-965 Poznan, Poland; (A.S.); (A.D.)
| | - Tomasz Marciniak
- Division of Electronic Systems and Signal Processing, Institute of Automatic Control and Robotics, Poznan University of Technology, 60-965 Poznan, Poland; (A.S.); (A.D.)
- Correspondence:
| | - Adam Dabrowski
- Division of Electronic Systems and Signal Processing, Institute of Automatic Control and Robotics, Poznan University of Technology, 60-965 Poznan, Poland; (A.S.); (A.D.)
| | - Marcin Stopa
- Department of Ophthalmology, Chair of Ophthalmology and Optometry, Heliodor Swiecicki University Hospital, Poznan University of Medical Sciences, 60-780 Poznan, Poland; (M.S.); (E.M.)
| | - Elzbieta Marciniak
- Department of Ophthalmology, Chair of Ophthalmology and Optometry, Heliodor Swiecicki University Hospital, Poznan University of Medical Sciences, 60-780 Poznan, Poland; (M.S.); (E.M.)
| | - Boguslaw Obara
- School of Computing, Newcastle University, Newcastle upon Tyne NE4 5TG, UK;
- Biosciences Institute, Newcastle University, Newcastle upon Tyne NE2 4HH, UK
| |
Collapse
|
26
|
Wang YZ, Wu W, Birch DG. A Hybrid Model Composed of Two Convolutional Neural Networks (CNNs) for Automatic Retinal Layer Segmentation of OCT Images in Retinitis Pigmentosa (RP). Transl Vis Sci Technol 2021; 10:9. [PMID: 34751740 PMCID: PMC8590180 DOI: 10.1167/tvst.10.13.9] [Citation(s) in RCA: 5] [Impact Index Per Article: 1.7] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/23/2022] Open
Abstract
Purpose We propose and evaluate a hybrid model composed of two convolutional neural networks (CNNs) with different architectures for automatic segmentation of retina layers in spectral domain optical coherence tomography (SD-OCT) B-scans of retinitis pigmentosa (RP). Methods The hybrid model consisted of a U-Net for initial semantic segmentation and a sliding-window (SW) CNN for refinement by correcting the segmentation errors of U-Net. The U-Net construction followed Ronneberger et al. (2015) with an input image size of 256 × 32. The SW model was similar to our previously reported approach. Training image patches were generated from 480 horizontal midline B-scans obtained from 220 patients with RP and 20 normal participants. Testing images were 160 midline B-scans from a separate group of 80 patients with RP. The Spectralis segmentation of B-scans was manually corrected for the boundaries of the inner limiting membrane, inner nuclear layer, ellipsoid zone (EZ), retinal pigment epithelium, and Bruch's membrane by one grader for the training set and two for the testing set. The trained U-Net and SW, as well as the hybrid model, were used to classify all pixels in the testing B-scans. Bland–Altman and correlation analyses were conducted to compare layer boundary lines, EZ width, and photoreceptor outer segment (OS) length and area determined by the models to those by human graders. Results The mean times to classify a B-scan image were 0.3, 65.7, and 2.4 seconds for U-Net, SW, and the hybrid model, respectively. The mean ± SD accuracies to segment retinal layers were 90.8% ± 4.8% and 90.7% ± 4.0% for U-Net and SW, respectively. The hybrid model improved mean ± SD accuracy to 91.5% ± 4.8% (P < 0.039 vs. U-Net), resulting in an improvement in layer boundary segmentation as revealed by Bland–Altman analyses. EZ width, OS length, and OS area measured by the models were highly correlated with those measured by the human graders (r > 0.95 for EZ width; r > 0.83 for OS length; r > 0.97 for OS area; P < 0.05). The hybrid model further improved the performance of measuring retinal layer thickness by correcting misclassification of retinal layers from U-Net. Conclusions While the performances of U-Net and the SW model were comparable in delineating various retinal layers, U-Net was much faster than the SW model to segment B-scan images. The hybrid model that combines the two improves automatic retinal layer segmentation from OCT images in RP. Translational Relevance A hybrid deep machine learning model composed of CNNs with different architectures can be more effective than either model separately for automatic analysis of SD-OCT scan images, which is becoming increasingly necessary with current high-resolution, high-density volume scans.
Collapse
Affiliation(s)
- Yi-Zhong Wang
- Retina Foundation of the Southwest, Dallas, TX, USA.,Department of Ophthalmology, University of Texas Southwestern Medical Center at Dallas, Dallas, TX, USA
| | - Wenxuan Wu
- Retina Foundation of the Southwest, Dallas, TX, USA
| | - David G Birch
- Retina Foundation of the Southwest, Dallas, TX, USA.,Department of Ophthalmology, University of Texas Southwestern Medical Center at Dallas, Dallas, TX, USA
| |
Collapse
|
27
|
Zhang Y, Li M, Yuan S, Liu Q, Chen Q. Robust region encoding and layer attribute protection for the segmentation of retina with multifarious abnormalities. Med Phys 2021; 48:7773-7789. [PMID: 34716932 DOI: 10.1002/mp.15315] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/26/2021] [Revised: 09/30/2021] [Accepted: 10/19/2021] [Indexed: 11/07/2022] Open
Abstract
PURPOSE To robustly segment retinal layers that are affected by complex variety of retinal diseases for optical coherence tomography angiography (OCTA) en face projection generation. METHODS In this paper, we propose a robust retinal layer segmentation model to reduce the impact of multifarious abnormalities on model performance. OCTA vascular distribution that is regarded as the supplements of spectral domain optical coherence tomography (SD-OCT) structural information is introduced to improve the robustness of layer region encoding. To further reduce the sensitivity of region encoding to retinal abnormalities, we propose a multitask layer-wise refinement (MLR) module that can refine the initial layer region segmentation results layer-by-layer. Finally, we design a region-to-surface transformation (RtST) module without additional training parameters to convert the encoding layer regions to their corresponding layer surfaces. This transformation from layer regions to layer surfaces can remove the inaccurate segmentation regions, and the layer surfaces are easier to be used to protect the retinal layer natures than layer regions. RESULTS Experimental data includes 273 eyes, where 95 eyes are normal and 178 eyes contain complex retinal diseases, including age-related macular degeneration (AMD), diabetic retinopathy (DR), central serous chorioretinopathy (CSC), choroidal neovascularization (CNV), and so forth. The dice similarity coefficient (DSC: %) of superficial, deep and outer retina achieves 98.92, 97.48, and 98.87 on normal eyes and 98.35, 95.33, and 98.17 on abnormal eyes. Compared with other commonly used layer segmentation models, our model achieves the state-of-the-art layer segmentation performance. CONCLUSIONS The final results prove that our proposed model obtains outstanding performance and has enough ability to resist retinal abnormalities. Besides, OCTA modality is helpful for retinal layer segmentation.
Collapse
Affiliation(s)
- Yuhan Zhang
- School of Computer Science and Engineering, Nanjing University of Science and Technology, Nanjing, China
| | - Mingchao Li
- School of Computer Science and Engineering, Nanjing University of Science and Technology, Nanjing, China
| | - Songtao Yuan
- Department of Ophthalmology, The First Affiliated Hospital with Nanjing Medical University, Nanjing, China
| | - Qinghuai Liu
- Department of Ophthalmology, The First Affiliated Hospital with Nanjing Medical University, Nanjing, China
| | - Qiang Chen
- School of Computer Science and Engineering, Nanjing University of Science and Technology, Nanjing, China
| |
Collapse
|
28
|
Ma R, Liu Y, Tao Y, Alawa KA, Shyu ML, Lee RK. Deep Learning-Based Retinal Nerve Fiber Layer Thickness Measurement of Murine Eyes. Transl Vis Sci Technol 2021; 10:21. [PMID: 34297789 PMCID: PMC8300062 DOI: 10.1167/tvst.10.8.21] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/24/2022] Open
Abstract
Purpose To design a robust and automated estimation method for measuring the retinal nerve fiber layer (RNFL) thickness using spectral domain optical coherence tomography (SD-OCT). Methods We developed a deep learning-based image segmentation network for automated segmentation of the RNFL in SD-OCT B-scans of mouse eyes. In total, 5500 SD-OCT B-scans (5200 B-scans were used as training data with the remaining 300 B-scans used as testing data) were used to develop this segmentation network. Postprocessing operations were then applied on the segmentation results to fill any discontinuities or remove any speckles in the RNFL. Subsequently, a three-dimensional retina thickness map was generated by z-stacking 100 segmentation processed thickness B-scan images together. Finally, the average absolute difference between algorithm predicted RNFL thickness compared to the ground truth manual human segmentation was calculated. Results The proposed method achieves an average dice similarity coefficient of 0.929 in the SD-OCT segmentation task and an average absolute difference of 0.0009 mm in thickness estimation task on the basis of the testing dataset. We also evaluated our segmentation algorithm on another biological dataset with SD-OCT volumes for RNFL thickness after the optic nerve crush injury. Results were shown to be comparable between the predicted and manually measured retina thickness values. Conclusions Experimental results demonstrate that our automated segmentation algorithm reliably predicts the RNFL thickness in SD-OCT volumes of mouse eyes compared to laborious and more subjective manual SD-OCT RNFL segmentation. Translational Relevance Automated segmentation using a deep learning-based algorithm for murine eye OCT effectively and rapidly produced nerve fiber layer thicknesses comparable to manual segmentation.
Collapse
Affiliation(s)
- Rui Ma
- Department of Electrical and Computer Engineering, University of Miami, Coral Gables, FL, USA
| | - Yuan Liu
- Bascom Palmer Eye Institute, University of Miami Miller School of Medicine, Miami, FL, USA
| | - Yudong Tao
- Department of Electrical and Computer Engineering, University of Miami, Coral Gables, FL, USA
| | - Karam A Alawa
- Bascom Palmer Eye Institute, University of Miami Miller School of Medicine, Miami, FL, USA
| | - Mei-Ling Shyu
- Department of Electrical and Computer Engineering, University of Miami, Coral Gables, FL, USA
| | - Richard K Lee
- Department of Electrical and Computer Engineering, University of Miami, Coral Gables, FL, USA.,Bascom Palmer Eye Institute, University of Miami Miller School of Medicine, Miami, FL, USA
| |
Collapse
|
29
|
Tian L, Hunt B, Bell MAL, Yi J, Smith JT, Ochoa M, Intes X, Durr NJ. Deep Learning in Biomedical Optics. Lasers Surg Med 2021; 53:748-775. [PMID: 34015146 PMCID: PMC8273152 DOI: 10.1002/lsm.23414] [Citation(s) in RCA: 25] [Impact Index Per Article: 8.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/11/2021] [Revised: 04/02/2021] [Accepted: 04/15/2021] [Indexed: 01/02/2023]
Abstract
This article reviews deep learning applications in biomedical optics with a particular emphasis on image formation. The review is organized by imaging domains within biomedical optics and includes microscopy, fluorescence lifetime imaging, in vivo microscopy, widefield endoscopy, optical coherence tomography, photoacoustic imaging, diffuse tomography, and functional optical brain imaging. For each of these domains, we summarize how deep learning has been applied and highlight methods by which deep learning can enable new capabilities for optics in medicine. Challenges and opportunities to improve translation and adoption of deep learning in biomedical optics are also summarized. Lasers Surg. Med. © 2021 Wiley Periodicals LLC.
Collapse
Affiliation(s)
- L. Tian
- Department of Electrical and Computer Engineering, Boston University, Boston, MA, USA
| | - B. Hunt
- Thayer School of Engineering, Dartmouth College, Hanover, NH, USA
| | - M. A. L. Bell
- Department of Electrical and Computer Engineering, Johns Hopkins University, Baltimore, MD, USA
- Department of Biomedical Engineering, Johns Hopkins University, Baltimore, MD, USA
- Department of Computer Science, Johns Hopkins University, Baltimore, MD, USA
| | - J. Yi
- Department of Biomedical Engineering, Johns Hopkins University, Baltimore, MD, USA
- Department of Ophthalmology, Johns Hopkins University, Baltimore, MD, USA
| | - J. T. Smith
- Center for Modeling, Simulation, and Imaging in Medicine, Rensselaer Polytechnic Institute, Troy, New York NY 12180
| | - M. Ochoa
- Center for Modeling, Simulation, and Imaging in Medicine, Rensselaer Polytechnic Institute, Troy, New York NY 12180
| | - X. Intes
- Center for Modeling, Simulation, and Imaging in Medicine, Rensselaer Polytechnic Institute, Troy, New York NY 12180
| | - N. J. Durr
- Department of Electrical and Computer Engineering, Johns Hopkins University, Baltimore, MD, USA
- Department of Biomedical Engineering, Johns Hopkins University, Baltimore, MD, USA
| |
Collapse
|
30
|
Szeskin A, Yehuda R, Shmueli O, Levy J, Joskowicz L. A column-based deep learning method for the detection and quantification of atrophy associated with AMD in OCT scans. Med Image Anal 2021; 72:102130. [PMID: 34198041 DOI: 10.1016/j.media.2021.102130] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/18/2020] [Revised: 05/27/2021] [Accepted: 06/03/2021] [Indexed: 10/21/2022]
Abstract
The objective quantification of retinal atrophy associated with age-related macular degeneration (AMD) is required for clinical diagnosis, follow-up, treatment efficacy evaluation, and clinical research. Spectral Domain Optical Coherence Tomography (OCT) has become an essential imaging technology to evaluate the macula. This paper describes a novel automatic method for the identification and quantification of atrophy associated with AMD in OCT scans and its visualization in the corresponding infrared imaging (IR) image. The method is based on the classification of light scattering patterns in vertical pixel-wide columns (A-scans) in OCT slices (B-scans) in which atrophy appears with a custom column-based convolutional neural network (CNN). The network classifies individual columns with 3D column patches formed by adjacent neighboring columns from the volumetric OCT scan. Subsequent atrophy columns form atrophy segments which are then projected onto the IR image and are used to identify and segment atrophy lesions in the IR image and to measure their areas and distances from the fovea. Experimental results on 106 clinical OCT scans (5,207 slices) in which cRORA atrophy (the end point of advanced dry AMD) was identified in 2,952 atrophy segments and 1,046 atrophy lesions yield a mean F1 score of 0.78 (std 0.06) and an AUC of 0.937, both close to the observer variability. Automated computer-based detection and quantification of atrophy associated with AMD using a column-based CNN classification in OCT scans can be performed at expert level and may be a useful clinical decision support and research tool for the diagnosis, follow-up and treatment of retinal degenerations and dystrophies.
Collapse
Affiliation(s)
- Adi Szeskin
- School of Computer Science and Engineering, The Hebrew University of Jerusalem, Israel
| | - Roei Yehuda
- School of Computer Science and Engineering, The Hebrew University of Jerusalem, Israel
| | - Or Shmueli
- Department of Ophthalmology, Hadassah Medical Center, Jerusalem, Israel
| | - Jaime Levy
- Department of Ophthalmology, Hadassah Medical Center, Jerusalem, Israel
| | - Leo Joskowicz
- School of Computer Science and Engineering, The Hebrew University of Jerusalem, Israel.
| |
Collapse
|
31
|
Lee S, Kang JU. CNN-based CP-OCT sensor integrated with a subretinal injector for retinal boundary tracking and injection guidance. JOURNAL OF BIOMEDICAL OPTICS 2021; 26:JBO-210109R. [PMID: 34196137 PMCID: PMC8242537 DOI: 10.1117/1.jbo.26.6.068001] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 04/05/2021] [Accepted: 06/15/2021] [Indexed: 05/08/2023]
Abstract
SIGNIFICANCE Subretinal injection is an effective way of delivering transplant genes and cells to treat many degenerative retinal diseases. However, the technique requires high-dexterity and microscale precision of experienced surgeons, who have to overcome the physiological hand tremor and limited visualization of the subretinal space. AIM To automatically guide the axial motion of microsurgical tools (i.e., a subretinal injector) with microscale precision in real time using a fiber-optic common-path swept-source optical coherence tomography distal sensor. APPROACH We propose, implement, and study real-time retinal boundary tracking of A-scan optical coherence tomography (OCT) images using a convolutional neural network (CNN) for automatic depth targeting of a selected retinal boundary for accurate subretinal injection guidance. A simplified 1D U-net is used for the retinal layer segmentation on A-scan OCT images. A Kalman filter, combining retinal boundary position measurement by CNN and velocity measurement by cross correlation between consecutive A-scan images, is applied to optimally estimate the retinal boundary position. Unwanted axial motions of the surgical tools are compensated by a piezoelectric linear motor based on the retinal boundary tracking. RESULTS CNN-based segmentation on A-scan OCT images achieves the mean unsigned error (MUE) of ∼3 pixels (8.1 μm) using an ex vivo bovine retina model. GPU parallel computing allows real-time inference (∼2 ms) and thus real-time retinal boundary tracking. Involuntary tremors, which include low-frequency draft in hundreds of micrometers and physiological tremors in tens of micrometers, are compensated effectively. The standard deviations of photoreceptor (PR) and choroid (CH) boundary positions get as low as 10.8 μm when the depth targeting is activated. CONCLUSIONS A CNN-based common-path OCT distal sensor successfully tracks retinal boundaries, especially the PR/CH boundary for subretinal injection, and automatically guides the tooltip's axial position in real time. The microscale depth targeting accuracy of our system shows its promising possibility for clinical application.
Collapse
Affiliation(s)
- Soohyun Lee
- Johns Hopkins University, Department of Electrical and Computer Engineering, Baltimore, Maryland, United States
- Address all correspondence to Soohyun Lee,
| | - Jin U. Kang
- Johns Hopkins University, Department of Electrical and Computer Engineering, Baltimore, Maryland, United States
| |
Collapse
|
32
|
Hu B. Deep Learning Image Feature Recognition Algorithm for Judgment on the Rationality of Landscape Planning and Design. COMPLEXITY 2021; 2021:1-15. [DOI: 10.1155/2021/9921095] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 09/01/2023]
Abstract
This paper uses an improved deep learning algorithm to judge the rationality of the design of landscape image feature recognition. The preprocessing of the image is proposed to enhance the data. The deficiencies in landscape feature extraction are further addressed based on the new model. Then, the two-stage training method of the model is used to solve the problems of long training time and convergence difficulties in deep learning. Innovative methods for zoning and segmentation training of landscape pattern features are proposed, which makes model training faster and generates more creative landscape patterns. Because of the impact of too many types of landscape elements in landscape images, traditional convolutional neural networks can no longer effectively solve this problem. On this basis, a fully convolutional neural network model is designed to perform semantic segmentation of landscape elements in landscape images. Through the method of deconvolution, the pixel-level semantic segmentation is realized. Compared with the 65% accuracy rate of the convolutional neural network, the fully convolutional neural network has an accuracy rate of 90.3% for the recognition of landscape elements. The method is effective, accurate, and intelligent for the classification of landscape element design, which better improves the accuracy of classification, greatly reduces the cost of landscape element design classification, and ensures that the technical method is feasible. This paper classifies landscape behavior based on this model for full convolutional neural network landscape images and demonstrates the effectiveness of using the model. In terms of landscape image processing, the image evaluation provides a certain basis.
Collapse
Affiliation(s)
- Bin Hu
- Xinyang Vocational and Technical College, Xinyang 464000, Henan, China
| |
Collapse
|
33
|
Huang Y, Zhang N, Hao Q. Real-time noise reduction based on ground truth free deep learning for optical coherence tomography. BIOMEDICAL OPTICS EXPRESS 2021; 12:2027-2040. [PMID: 33996214 PMCID: PMC8086449 DOI: 10.1364/boe.419584] [Citation(s) in RCA: 7] [Impact Index Per Article: 2.3] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Received: 01/11/2021] [Revised: 02/27/2021] [Accepted: 03/08/2021] [Indexed: 06/07/2023]
Abstract
Optical coherence tomography (OCT) is a high-resolution non-invasive 3D imaging modality, which has been widely used for biomedical research and clinical studies. The presence of noise on OCT images is inevitable which will cause problems for post-image processing and diagnosis. The frame-averaging technique that acquires multiple OCT images at the same or adjacent locations can enhance the image quality significantly. Both conventional frame averaging methods and deep learning-based methods using averaged frames as ground truth have been reported. However, conventional averaging methods suffer from the limitation of long image acquisition time, while deep learning-based methods require complicated and tedious ground truth label preparation. In this work, we report a deep learning-based noise reduction method that does not require clean images as ground truth for model training. Three network structures, including Unet, super-resolution residual network (SRResNet), and our modified asymmetric convolution-SRResNet (AC-SRResNet), were trained and evaluated using signal-to-noise ratio (SNR), contrast-to-noise ratio (CNR), edge preservation index (EPI) and computation time (CT). The effectiveness of these three trained models on OCT images of different samples and different systems was also investigated and confirmed. The SNR improvement for different sample images for L2-loss-trained Unet, SRResNet, and AC-SRResNet are 20.83 dB, 24.88 dB, and 22.19 dB, respectively. The SNR improvement for public images from different system for L1-loss-trained Unet, SRResNet, and AC-SRResNet are 19.36 dB, 20.11 dB, and 22.15 dB, respectively. AC-SRResNet and SRResNet demonstrate better denoising effect than Unet with longer computation time. AC-SRResNet demonstrates better edge preservation capability than SRResNet while Unet is close to AC-SRResNet. Eventually, we incorporated Unet, SRResNet, and AC-SRResNet into our graphic processing unit accelerated OCT imaging system for online noise reduction evaluation. Real-time noise reduction for OCT images with size of 512×512 pixels for Unet, SRResNet, and AC-SRResNet at 64 fps, 19 fps, and 17 fps were achieved respectively.
Collapse
|
34
|
Gong D, Kras A, Miller JB. Application of Deep Learning for Diagnosing, Classifying, and Treating Age-Related Macular Degeneration. Semin Ophthalmol 2021; 36:198-204. [PMID: 33617390 DOI: 10.1080/08820538.2021.1889617] [Citation(s) in RCA: 4] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 01/20/2023]
Abstract
Age-related macular degeneration (AMD) affects nearly 200 million people and is the third leading cause of irreversible vision loss worldwide. Deep learning, a branch of artificial intelligence that can learn image recognition based on pre-existing datasets, creates an opportunity for more accurate and efficient diagnosis, classification, and treatment of AMD on both individual and population levels. Current algorithms based on fundus photography and optical coherence tomography imaging have already achieved diagnostic accuracy levels comparable to human graders. This accuracy can be further increased when deep learning algorithms are simultaneously applied to multiple diagnostic imaging modalities. Combined with advances in telemedicine and imaging technology, deep learning can enable large populations of patients to be screened than would otherwise be possible and allow ophthalmologists to focus on seeing those patients who are in need of treatment, thus reducing the number of patients with significant visual impairment from AMD.
Collapse
Affiliation(s)
- Dan Gong
- Department of Ophthalmology, Retina Service, Massachusetts Eye and Ear Infirmary, Harvard Medical School, Boston, MA,USA
| | - Ashley Kras
- Harvard Retinal Imaging Lab, Massachusetts Eye and Ear Infirmary, Boston, MA
| | - John B Miller
- Department of Ophthalmology, Retina Service, Massachusetts Eye and Ear Infirmary, Harvard Medical School, Boston, MA,USA.,Harvard Retinal Imaging Lab, Massachusetts Eye and Ear Infirmary, Boston, MA
| |
Collapse
|
35
|
Sommersperger M, Weiss J, Ali Nasseri M, Gehlbach P, Iordachita I, Navab N. Real-time tool to layer distance estimation for robotic subretinal injection using intraoperative 4D OCT. BIOMEDICAL OPTICS EXPRESS 2021; 12:1085-1104. [PMID: 33680560 PMCID: PMC7901333 DOI: 10.1364/boe.415477] [Citation(s) in RCA: 12] [Impact Index Per Article: 4.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Received: 12/03/2020] [Revised: 01/15/2021] [Accepted: 01/19/2021] [Indexed: 05/24/2023]
Abstract
The emergence of robotics could enable ophthalmic microsurgical procedures that were previously not feasible due to the precision limits of manual delivery, for example, targeted subretinal injection. Determining the distance between the needle tip, the internal limiting membrane (ILM), and the retinal pigment epithelium (RPE) both precisely and reproducibly is required for safe and successful robotic retinal interventions. Recent advances in intraoperative optical coherence tomography (iOCT) have opened the path for 4D image-guided surgery by providing near video-rate imaging with micron-level resolution to visualize retinal structures, surgical instruments, and tool-tissue interactions. In this work, we present a novel pipeline to precisely estimate the distance between the injection needle and the surface boundaries of two retinal layers, the ILM and the RPE, from iOCT volumes. To achieve high computational efficiency, we reduce the analysis to the relevant area around the needle tip. We employ a convolutional neural network (CNN) to segment the tool surface, as well as the retinal layer boundaries from selected iOCT B-scans within this tip area. This results in the generation and processing of 3D surface point clouds for the tool, ILM and RPE from the B-scan segmentation maps, which in turn allows the estimation of the minimum distance between the resulting tool and layer point clouds. The proposed method is evaluated on iOCT volumes from ex-vivo porcine eyes and achieves an average error of 9.24 µm and 8.61 µm measuring the distance from the needle tip to the ILM and the RPE, respectively. The results demonstrate that this approach is robust to the high levels of noise present in iOCT B-scans and is suitable for the interventional use case by providing distance feedback at an average update rate of 15.66 Hz.
Collapse
Affiliation(s)
- Michael Sommersperger
- Johns Hopkins University, Baltimore, MD 21218, USA
- Technical University of Munich, Germany
| | | | - M. Ali Nasseri
- Technical University of Munich, Germany
- Klinikum Rechts der Isar, Augenklinik, Munich, Germany
| | | | | | - Nassir Navab
- Johns Hopkins University, Baltimore, MD 21218, USA
- Technical University of Munich, Germany
| |
Collapse
|
36
|
Li Q, Li S, He Z, Guan H, Chen R, Xu Y, Wang T, Qi S, Mei J, Wang W. DeepRetina: Layer Segmentation of Retina in OCT Images Using Deep Learning. Transl Vis Sci Technol 2020; 9:61. [PMID: 33329940 PMCID: PMC7726589 DOI: 10.1167/tvst.9.2.61] [Citation(s) in RCA: 16] [Impact Index Per Article: 4.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/14/2019] [Accepted: 10/19/2020] [Indexed: 11/24/2022] Open
Abstract
Purpose To automate the segmentation of retinal layers, we propose DeepRetina, a method based on deep neural networks. Methods DeepRetina uses the improved Xception65 to extract and learn the characteristics of retinal layers. The Xception65-extracted feature maps are inputted to an atrous spatial pyramid pooling module to obtain multiscale feature information. This information is then recovered to capture clearer retinal layer boundaries in the encoder-decoder module, thus completing retinal layer auto-segmentation of the retinal optical coherence tomography (OCT) images. Results We validated this method using a retinal OCT image database containing 280 volumes (40 B-scans per volume) to demonstrate its effectiveness. The results showed that the method exhibits excellent performance in terms of the mean intersection over union and sensitivity (Se), which are as high as 90.41 and 92.15%, respectively. The intersection over union and Se values of the nerve fiber layer, ganglion cell layer, inner plexiform layer, inner nuclear layer, outer plexiform layer, outer nuclear layer, outer limiting membrane, photoreceptor inner segment, photoreceptor outer segment, and pigment epithelium layer were found to be above 88%. Conclusions DeepRetina can automate the segmentation of retinal layers and has great potential for the early diagnosis of fundus retinal diseases. In addition, our approach will provide a segmentation model framework for other types of tissues and cells in clinical practice. Translational Relevance Automating the segmentation of retinal layers can help effectively diagnose and monitor clinical retinal diseases. In addition, it requires only a small amount of manual segmentation, significantly improving work efficiency.
Collapse
Affiliation(s)
- Qiaoliang Li
- National-Regional Key Technology Engineering Laboratory for Medical Ultrasound, Guangdong Key Laboratory for Biomedical Measurements and Ultrasound Imaging, Department of Biomedical Engineering, School of Medicine, Shenzhen University, Xueyuan Avenue, Nanshan District, Shenzhen, Guangdong Province, China
| | - Shiyu Li
- National-Regional Key Technology Engineering Laboratory for Medical Ultrasound, Guangdong Key Laboratory for Biomedical Measurements and Ultrasound Imaging, Department of Biomedical Engineering, School of Medicine, Shenzhen University, Xueyuan Avenue, Nanshan District, Shenzhen, Guangdong Province, China
| | - Zhuoying He
- National-Regional Key Technology Engineering Laboratory for Medical Ultrasound, Guangdong Key Laboratory for Biomedical Measurements and Ultrasound Imaging, Department of Biomedical Engineering, School of Medicine, Shenzhen University, Xueyuan Avenue, Nanshan District, Shenzhen, Guangdong Province, China
| | - Huimin Guan
- National-Regional Key Technology Engineering Laboratory for Medical Ultrasound, Guangdong Key Laboratory for Biomedical Measurements and Ultrasound Imaging, Department of Biomedical Engineering, School of Medicine, Shenzhen University, Xueyuan Avenue, Nanshan District, Shenzhen, Guangdong Province, China
| | - Runmin Chen
- National-Regional Key Technology Engineering Laboratory for Medical Ultrasound, Guangdong Key Laboratory for Biomedical Measurements and Ultrasound Imaging, Department of Biomedical Engineering, School of Medicine, Shenzhen University, Xueyuan Avenue, Nanshan District, Shenzhen, Guangdong Province, China
| | - Ying Xu
- National-Regional Key Technology Engineering Laboratory for Medical Ultrasound, Guangdong Key Laboratory for Biomedical Measurements and Ultrasound Imaging, Department of Biomedical Engineering, School of Medicine, Shenzhen University, Xueyuan Avenue, Nanshan District, Shenzhen, Guangdong Province, China
| | - Tao Wang
- National-Regional Key Technology Engineering Laboratory for Medical Ultrasound, Guangdong Key Laboratory for Biomedical Measurements and Ultrasound Imaging, Department of Biomedical Engineering, School of Medicine, Shenzhen University, Xueyuan Avenue, Nanshan District, Shenzhen, Guangdong Province, China
| | - Suwen Qi
- National-Regional Key Technology Engineering Laboratory for Medical Ultrasound, Guangdong Key Laboratory for Biomedical Measurements and Ultrasound Imaging, Department of Biomedical Engineering, School of Medicine, Shenzhen University, Xueyuan Avenue, Nanshan District, Shenzhen, Guangdong Province, China
| | - Jun Mei
- Medical Imaging Department of Shenzhen Eye Hospital Affiliated to Jinan University, Shenzhen, Guangdong Province, China
| | - Wei Wang
- Department of Pathology, Shenzhen University General Hospital, Shenzhen, Guangdong Province, China
| |
Collapse
|
37
|
Anoop B, Pavan R, Girish G, Kothari AR, Rajan J. Stack generalized deep ensemble learning for retinal layer segmentation in Optical Coherence Tomography images. Biocybern Biomed Eng 2020. [DOI: 10.1016/j.bbe.2020.07.010] [Citation(s) in RCA: 5] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/16/2022]
|
38
|
Zhang Y, Huang C, Li M, Xie S, Xie K, Ji Z, Yuan S, Chen Q. Robust Layer Segmentation Against Complex Retinal Abnormalities for en face OCTA Generation. ACTA ACUST UNITED AC 2020. [DOI: 10.1007/978-3-030-59722-1_62] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 02/16/2023]
|
39
|
Raja H, Akram MU, Shaukat A, Khan SA, Alghamdi N, Khawaja SG, Nazir N. Extraction of Retinal Layers Through Convolution Neural Network (CNN) in an OCT Image for Glaucoma Diagnosis. J Digit Imaging 2020; 33:1428-1442. [PMID: 32968881 DOI: 10.1007/s10278-020-00383-5] [Citation(s) in RCA: 14] [Impact Index Per Article: 3.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/19/2020] [Revised: 06/16/2020] [Accepted: 09/09/2020] [Indexed: 11/26/2022] Open
Abstract
Glaucoma is a progressive and deteriorating optic neuropathy that leads to visual field defects. The damage occurs as glaucoma is irreversible, so early and timely diagnosis is of significant importance. The proposed system employs the convolution neural network (CNN) for automatic segmentation of the retinal layers. The inner limiting membrane (ILM) and retinal pigmented epithelium (RPE) are used to calculate cup-to-disc ratio (CDR) for glaucoma diagnosis. The proposed system uses structure tensors to extract candidate layer pixels, and a patch across each candidate layer pixel is extracted, which is classified using CNN. The proposed framework is based upon VGG-16 architecture for feature extraction and classification of retinal layer pixels. The output feature map is merged into SoftMax layer for classification and produces probability map for central pixel of each patch and decides whether it is ILM, RPE, or background pixels. Graph search theory refines the extracted layers by interpolating the missing points, and these extracted ILM and RPE are finally used to compute CDR value and diagnose glaucoma. The proposed system is validated using a local dataset of optical coherence tomography images from 196 patients, including normal and glaucoma subjects. The dataset contains manually annotated ILM and RPE layers; manually extracted patches for ILM, RPE, and background pixels; CDR values; and eventually final finding related to glaucoma. The proposed system is able to extract ILM and RPE with a small absolute mean error of 6.03 and 5.56, respectively, and it finds CDR value within average range of ± 0.09 as compared with glaucoma expert. The proposed system achieves average sensitivity, specificity, and accuracies of 94.6, 94.07, and 94.68, respectively.
Collapse
Affiliation(s)
- Hina Raja
- Department of Computer and Software Engineering, National University of Sciences and Technology, Islamabad, Pakistan.
| | - M Usman Akram
- Department of Computer and Software Engineering, National University of Sciences and Technology, Islamabad, Pakistan
| | - Arslan Shaukat
- Department of Computer and Software Engineering, National University of Sciences and Technology, Islamabad, Pakistan
| | - Shoab Ahmed Khan
- Department of Computer and Software Engineering, National University of Sciences and Technology, Islamabad, Pakistan
| | - Norah Alghamdi
- Department of Computer Science, Princess Nora Bint Abdurahman University, Riyadh, Saudi Arabia
| | - Sajid Gul Khawaja
- Department of Computer and Software Engineering, National University of Sciences and Technology, Islamabad, Pakistan
| | - Noman Nazir
- Armed Forces Institute of Ophthalmology, Rawalpindi, Pakistan
| |
Collapse
|
40
|
Tan B, Sim R, Chua J, Wong DWK, Yao X, Garhöfer G, Schmidl D, Werkmeister RM, Schmetterer L. Approaches to quantify optical coherence tomography angiography metrics. ANNALS OF TRANSLATIONAL MEDICINE 2020; 8:1205. [PMID: 33241054 PMCID: PMC7576021 DOI: 10.21037/atm-20-3246] [Citation(s) in RCA: 21] [Impact Index Per Article: 5.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Indexed: 12/13/2022]
Abstract
Optical coherence tomography (OCT) has revolutionized the field of ophthalmology in the last three decades. As an OCT extension, OCT angiography (OCTA) utilizes a fast OCT system to detect motion contrast in ocular tissue and provides a three-dimensional representation of the ocular vasculature in a non-invasive, dye-free manner. The first OCT machine equipped with OCTA function was approved by U.S. Food and Drug Administration in 2016 and now it is widely applied in clinics. To date, numerous methods have been developed to aid OCTA interpretation and quantification. In this review, we focused on the workflow of OCTA-based interpretation, beginning from the generation of the OCTA images using signal decorrelation, which we divided into intensity-based, phase-based and phasor-based methods. We further discussed methods used to address image artifacts that are commonly observed in clinical settings, to the algorithms for image enhancement, binarization, and OCTA metrics extraction. We believe a better grasp of these technical aspects of OCTA will enhance the understanding of the technology and its potential application in disease diagnosis and management. Moreover, future studies will also explore the use of ocular OCTA as a window to link ocular vasculature to the function of other organs such as the kidney and brain.
Collapse
Affiliation(s)
- Bingyao Tan
- Institute for Health Technologies, Nanyang Technological University, Singapore, Singapore.,Singapore Eye Research Institute, Singapore National Eye Centre, Singapore, Singapore.,SERI-NTU Advanced Ocular Engineering (STANCE) Program, Nanyang Technological University, Singapore, Singapore
| | - Ralene Sim
- Singapore Eye Research Institute, Singapore National Eye Centre, Singapore, Singapore
| | - Jacqueline Chua
- Singapore Eye Research Institute, Singapore National Eye Centre, Singapore, Singapore.,Academic Clinical Program, Duke-NUS Medical School, Singapore, Singapore
| | - Damon W K Wong
- Institute for Health Technologies, Nanyang Technological University, Singapore, Singapore.,Singapore Eye Research Institute, Singapore National Eye Centre, Singapore, Singapore.,SERI-NTU Advanced Ocular Engineering (STANCE) Program, Nanyang Technological University, Singapore, Singapore
| | - Xinwen Yao
- Institute for Health Technologies, Nanyang Technological University, Singapore, Singapore.,Singapore Eye Research Institute, Singapore National Eye Centre, Singapore, Singapore.,SERI-NTU Advanced Ocular Engineering (STANCE) Program, Nanyang Technological University, Singapore, Singapore
| | - Gerhard Garhöfer
- Department of Clinical Pharmacology, Medical University of Vienna, Vienna, Austria
| | - Doreen Schmidl
- Department of Clinical Pharmacology, Medical University of Vienna, Vienna, Austria
| | - René M Werkmeister
- Center for Medical Physics and Biomedical Engineering, Medical University of Vienna, Vienna, Austria
| | - Leopold Schmetterer
- Singapore Eye Research Institute, Singapore National Eye Centre, Singapore, Singapore.,SERI-NTU Advanced Ocular Engineering (STANCE) Program, Nanyang Technological University, Singapore, Singapore.,Academic Clinical Program, Duke-NUS Medical School, Singapore, Singapore.,Department of Clinical Pharmacology, Medical University of Vienna, Vienna, Austria.,Center for Medical Physics and Biomedical Engineering, Medical University of Vienna, Vienna, Austria.,Yong Loo Lin School of Medicine, National University of Singapore and National University Health System, Singapore, Singapore.,Department of Ophthalmology, Lee Kong Chian School of Medicine, Nanyang Technological University, Singapore, Singapore.,Institute of Molecular and Clinical Ophthalmology, Basel, Switzerland
| |
Collapse
|
41
|
Xu Z, Wang W, Yang J, Zhao J, Ding D, He F, Chen D, Yang Z, Li X, Yu W, Chen Y. Automated diagnoses of age-related macular degeneration and polypoidal choroidal vasculopathy using bi-modal deep convolutional neural networks. Br J Ophthalmol 2020; 105:561-566. [PMID: 32499330 DOI: 10.1136/bjophthalmol-2020-315817] [Citation(s) in RCA: 15] [Impact Index Per Article: 3.8] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/04/2020] [Revised: 03/31/2020] [Accepted: 05/14/2020] [Indexed: 02/06/2023]
Abstract
AIMS To investigate the efficacy of a bi-modality deep convolutional neural network (DCNN) framework to categorise age-related macular degeneration (AMD) and polypoidal choroidal vasculopathy (PCV) from colour fundus images and optical coherence tomography (OCT) images. METHODS A retrospective cross-sectional study was proposed of patients with AMD or PCV who came to Peking Union Medical College Hospital. Diagnoses of all patients were confirmed by two retinal experts based on diagnostic gold standard for AMD and PCV. Patients with concurrent retinal vascular diseases were excluded. Colour fundus images and spectral domain OCT images were taken from dilated eyes of patients and healthy controls, and anonymised. All images were pre-labelled into normal, dry or wet AMD or PCV. ResNet-50 models were used as the backbone and alternate machine learning models including random forest classifiers were constructed for further comparison. For human-machine comparison, the same testing data set was diagnosed by three retinal experts independently. All images from the same participant were presented only within a single partition subset. RESULTS On a test set of 143 fundus and OCT image pairs from 80 eyes (20 eyes per-group), the bi-modal DCNN demonstrated the best performance, with accuracy 87.4%, sensitivity 88.8% and specificity 95.6%, and a perfect agreement with diagnostic gold standard (Cohen's κ 0.828), exceeds slightly over the best expert (Human1, Cohen's κ 0.810). For recognising PCV, the model outperformed the best expert as well. CONCLUSION A bi-modal DCNN for automated classification of AMD and PCV is accurate and promising in the realm of public health.
Collapse
Affiliation(s)
- Zhiyan Xu
- Department of Ophthalmology, Peking Union Medical College Hospital, Dongcheng District, Beijing, China.,Key Laboratory of Ocular Fundus Diseases, Chinese Academy of Medical Sciences and Peking Union Medical College, Beijing, China
| | - Weisen Wang
- AI & Media Computing Lab, School of Information, Renmin University of China, Beijing, China
| | - Jingyuan Yang
- Department of Ophthalmology, Peking Union Medical College Hospital, Dongcheng District, Beijing, China.,Key Laboratory of Ocular Fundus Diseases, Chinese Academy of Medical Sciences and Peking Union Medical College, Beijing, China
| | - Jianchun Zhao
- Vistel AI Lab, Visionary Intelligence Ltd, Beijing, China
| | - Dayong Ding
- Vistel AI Lab, Visionary Intelligence Ltd, Beijing, China
| | - Feng He
- Department of Ophthalmology, Peking Union Medical College Hospital, Dongcheng District, Beijing, China.,Key Laboratory of Ocular Fundus Diseases, Chinese Academy of Medical Sciences and Peking Union Medical College, Beijing, China
| | - Di Chen
- Department of Ophthalmology, Peking Union Medical College Hospital, Dongcheng District, Beijing, China.,Key Laboratory of Ocular Fundus Diseases, Chinese Academy of Medical Sciences and Peking Union Medical College, Beijing, China
| | - Zhikun Yang
- Department of Ophthalmology, Peking Union Medical College Hospital, Dongcheng District, Beijing, China.,Key Laboratory of Ocular Fundus Diseases, Chinese Academy of Medical Sciences and Peking Union Medical College, Beijing, China
| | - Xirong Li
- Department for state-of-the-art ophthalmology AI research & development, Key Lab of DEKE, Renmin University of China, Beijing, China
| | - Weihong Yu
- Department of Ophthalmology, Peking Union Medical College Hospital, Dongcheng District, Beijing, China.,Key Laboratory of Ocular Fundus Diseases, Chinese Academy of Medical Sciences and Peking Union Medical College, Beijing, China
| | - Youxin Chen
- Department of Ophthalmology, Peking Union Medical College Hospital, Dongcheng District, Beijing, China .,Key Laboratory of Ocular Fundus Diseases, Chinese Academy of Medical Sciences and Peking Union Medical College, Beijing, China
| |
Collapse
|
42
|
Stromer D, Moult EM, Chen S, Waheed NK, Maier A, Fujimoto JG. Correction propagation for user-assisted optical coherence tomography segmentation: general framework and application to Bruch's membrane segmentation. BIOMEDICAL OPTICS EXPRESS 2020; 11:2830-2848. [PMID: 32499964 PMCID: PMC7249839 DOI: 10.1364/boe.392759] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 03/16/2020] [Revised: 04/22/2020] [Accepted: 04/23/2020] [Indexed: 05/25/2023]
Abstract
Optical coherence tomography (OCT) is a commonly used ophthalmic imaging modality. While OCT has traditionally been viewed cross-sectionally (i.e., as a sequence of B-scans), higher A-scan rates have increased interest in en face OCT visualization and analysis. The recent clinical introduction of OCT angiography (OCTA) has further spurred this interest, with chorioretinal OCTA being predominantly displayed via en face projections. Although en face visualization and quantitation are natural for many retinal features (e.g., drusen and vasculature), it requires segmentation. Because manual segmentation of volumetric OCT data is prohibitively laborious in many settings, there has been significant research and commercial interest in developing automatic segmentation algorithms. While these algorithms have achieved impressive results, the variability of image qualities and the variety of ocular pathologies cause even the most robust automatic segmentation algorithms to err. In this study, we develop a user-assisted segmentation approach, complementary to fully-automatic methods, wherein correction propagation is used to reduce the burden of manually correcting automatic segmentations. The approach is evaluated for Bruch's membrane segmentation in eyes with advanced age-related macular degeneration.
Collapse
Affiliation(s)
- Daniel Stromer
- Department of Electrical Engineering and Computer Science, Research Laboratory of Electronics, Massachusetts Institute of Technology, Cambridge, MA 02139-4307, USA
- Pattern Recognition Lab, Friedrich-Alexander-Universität Erlangen-Nürnberg, Erlangen, Germany
- These authors have contributed equally to this work
| | - Eric M. Moult
- Department of Electrical Engineering and Computer Science, Research Laboratory of Electronics, Massachusetts Institute of Technology, Cambridge, MA 02139-4307, USA
- These authors have contributed equally to this work
| | - Siyu Chen
- Department of Electrical Engineering and Computer Science, Research Laboratory of Electronics, Massachusetts Institute of Technology, Cambridge, MA 02139-4307, USA
| | - Nadia K. Waheed
- New England Eye Center, Tufts Medical Center, Boston, MA 02111, USA
| | - Andreas Maier
- Pattern Recognition Lab, Friedrich-Alexander-Universität Erlangen-Nürnberg, Erlangen, Germany
| | - James G. Fujimoto
- Department of Electrical Engineering and Computer Science, Research Laboratory of Electronics, Massachusetts Institute of Technology, Cambridge, MA 02139-4307, USA
| |
Collapse
|
43
|
Xu Q, Zeng Y, Tang W, Peng W, Xia T, Li Z, Teng F, Li W, Guo J. Multi-Task Joint Learning Model for Segmenting and Classifying Tongue Images Using a Deep Neural Network. IEEE J Biomed Health Inform 2020; 24:2481-2489. [PMID: 32310809 DOI: 10.1109/jbhi.2020.2986376] [Citation(s) in RCA: 92] [Impact Index Per Article: 23.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/07/2022]
Abstract
Automatic tongue image segmentation and tongue image classification are two crucial tongue characterization tasks in traditional Chinese medicine (TCM). Due to the complexity of tongue segmentation and fine-grained traits of tongue image classification, both tasks are challenging. Fortunately, from the perspective of computer vision, these two tasks are highly interrelated, making them compatible with the idea of Multi-Task Joint learning (MTL). By sharing the underlying parameters and adding two different task loss functions, an MTL method for segmenting and classifying tongue images is proposed in this paper. Moreover, two state-of-the-art deep neural network variants (UNET and Discriminative Filter Learning (DFL)) are fused into the MTL to perform these two tasks. To the best of our knowledge, our method is the first attempt to manage both tasks simultaneously with MTL. We conducted extensive experiments with the proposed method. The experimental results show that our joint method outperforms the existing tongue characterization methods. Besides, visualizations and ablation studies are provided to aid in understanding our approach, which suggest that our method is highly consistent with human perception.
Collapse
|
44
|
Tong Y, Lu W, Yu Y, Shen Y. Application of machine learning in ophthalmic imaging modalities. EYE AND VISION 2020; 7:22. [PMID: 32322599 PMCID: PMC7160952 DOI: 10.1186/s40662-020-00183-6] [Citation(s) in RCA: 36] [Impact Index Per Article: 9.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 10/16/2019] [Accepted: 03/10/2020] [Indexed: 12/27/2022]
Abstract
In clinical ophthalmology, a variety of image-related diagnostic techniques have begun to offer unprecedented insights into eye diseases based on morphological datasets with millions of data points. Artificial intelligence (AI), inspired by the human multilayered neuronal system, has shown astonishing success within some visual and auditory recognition tasks. In these tasks, AI can analyze digital data in a comprehensive, rapid and non-invasive manner. Bioinformatics has become a focus particularly in the field of medical imaging, where it is driven by enhanced computing power and cloud storage, as well as utilization of novel algorithms and generation of data in massive quantities. Machine learning (ML) is an important branch in the field of AI. The overall potential of ML to automatically pinpoint, identify and grade pathological features in ocular diseases will empower ophthalmologists to provide high-quality diagnosis and facilitate personalized health care in the near future. This review offers perspectives on the origin, development, and applications of ML technology, particularly regarding its applications in ophthalmic imaging modalities.
Collapse
Affiliation(s)
- Yan Tong
- 1Eye Center, Renmin Hospital of Wuhan University, Wuhan, 430060 Hubei China
| | - Wei Lu
- 1Eye Center, Renmin Hospital of Wuhan University, Wuhan, 430060 Hubei China
| | - Yue Yu
- 1Eye Center, Renmin Hospital of Wuhan University, Wuhan, 430060 Hubei China
| | - Yin Shen
- 1Eye Center, Renmin Hospital of Wuhan University, Wuhan, 430060 Hubei China.,2Medical Research Institute, Wuhan University, Wuhan, Hubei China
| |
Collapse
|
45
|
Wu C, Qiao Z, Zhang N, Li X, Fan J, Song H, Ai D, Yang J, Huang Y. Phase unwrapping based on a residual en-decoder network for phase images in Fourier domain Doppler optical coherence tomography. BIOMEDICAL OPTICS EXPRESS 2020; 11:1760-1771. [PMID: 32341846 PMCID: PMC7173896 DOI: 10.1364/boe.386101] [Citation(s) in RCA: 8] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Received: 12/16/2019] [Revised: 02/19/2020] [Accepted: 02/27/2020] [Indexed: 06/01/2023]
Abstract
To solve the phase unwrapping problem for phase images in Fourier domain Doppler optical coherence tomography (DOCT), we propose a deep learning-based residual en-decoder network (REDN) method. In our approach, we reformulate the definition for obtaining the true phase as obtaining an integer multiple of 2π at each pixel by semantic segmentation. The proposed REDN architecture can provide recognition performance with pixel-level accuracy. To address the lack of phase images that are noise and wrapping free from DOCT systems for training, we used simulated images synthesized with DOCT phase image background noise features. An evaluation study on simulated images, DOCT phase images of phantom milk flowing in a plastic tube and a mouse artery, was performed. Meanwhile, a comparison study with recently proposed deep learning-based DeepLabV3+ and PhaseNet methods for signal phase unwrapping and traditional modified networking programming (MNP) method was also performed. Both visual inspection and quantitative metrical evaluation based on accuracy, specificity, sensitivity, root-mean-square-error, total-variation, and processing time demonstrate the robustness, effectiveness and superiority of our method. The proposed REDN method will benefit accurate and fast DOCT phase image-based diagnosis and evaluation when the detected phase is wrapped and will enrich the deep learning-based image processing platform for DOCT images.
Collapse
Affiliation(s)
- Chuanchao Wu
- School of Optics and Photonics, Beijing Institute of Technology, No. 5 South Zhongguancun Street, Haidian, Beijing 100081, China
| | - Zhengyu Qiao
- School of Optics and Photonics, Beijing Institute of Technology, No. 5 South Zhongguancun Street, Haidian, Beijing 100081, China
| | - Nan Zhang
- School of Optics and Photonics, Beijing Institute of Technology, No. 5 South Zhongguancun Street, Haidian, Beijing 100081, China
| | - Xiaochen Li
- School of Optics and Photonics, Beijing Institute of Technology, No. 5 South Zhongguancun Street, Haidian, Beijing 100081, China
| | - Jingfan Fan
- School of Optics and Photonics, Beijing Institute of Technology, No. 5 South Zhongguancun Street, Haidian, Beijing 100081, China
| | - Hong Song
- School of Computer Science and Technology, Beijing Institute of Technology, No. 5 South Zhongguancun Street, Haidian, Beijing 100081, China
| | - Danni Ai
- School of Optics and Photonics, Beijing Institute of Technology, No. 5 South Zhongguancun Street, Haidian, Beijing 100081, China
| | - Jian Yang
- School of Optics and Photonics, Beijing Institute of Technology, No. 5 South Zhongguancun Street, Haidian, Beijing 100081, China
| | - Yong Huang
- School of Optics and Photonics, Beijing Institute of Technology, No. 5 South Zhongguancun Street, Haidian, Beijing 100081, China
| |
Collapse
|
46
|
Stegmann H, Werkmeister RM, Pfister M, Garhöfer G, Schmetterer L, dos Santos VA. Deep learning segmentation for optical coherence tomography measurements of the lower tear meniscus. BIOMEDICAL OPTICS EXPRESS 2020; 11:1539-1554. [PMID: 32206427 PMCID: PMC7075621 DOI: 10.1364/boe.386228] [Citation(s) in RCA: 18] [Impact Index Per Article: 4.5] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Received: 12/17/2019] [Revised: 01/27/2020] [Accepted: 01/28/2020] [Indexed: 05/22/2023]
Abstract
The tear meniscus contains most of the tear fluid and therefore is a good indicator for the state of the tear film. Previously, we used a custom-built optical coherence tomography (OCT) system to study the lower tear meniscus by automatically segmenting the image data with a thresholding-based segmentation algorithm (TBSA). In this report, we investigate whether the results of this image segmentation algorithm are suitable to train a neural network in order to obtain similar or better segmentation results with shorter processing times. Considering the class imbalance problem, we compare two approaches, one directly segmenting the tear meniscus (DSA), the other first localizing the region of interest and then segmenting within the higher resolution image section (LSA). A total of 6658 images labeled by the TBSA were used to train deep convolutional neural networks with supervised learning. Five-fold cross-validation reveals a sensitivity of 96.36% and 96.43%, a specificity of 99.98% and 99.86% and a Jaccard index of 93.24% and 93.16% for the DSA and LSA, respectively. Average segmentation times are up to 228 times faster than the TBSA. Additionally, we report the behavior of the DSA and LSA in cases challenging for the TBSA and further test the applicability to measurements acquired with a commercially available OCT system. The application of deep learning for the segmentation of the tear meniscus provides a powerful tool for the assessment of the tear film, supporting studies for the investigation of the pathophysiology of dry eye-related diseases.
Collapse
Affiliation(s)
- Hannes Stegmann
- Center for Medical Physics and Biomedical Engineering, Medical University of Vienna, Austria
- Christian Doppler Laboratory for Ocular and Dermal Effects of Thiomers, Medical University of Vienna, Austria
| | - René M. Werkmeister
- Center for Medical Physics and Biomedical Engineering, Medical University of Vienna, Austria
- Christian Doppler Laboratory for Ocular and Dermal Effects of Thiomers, Medical University of Vienna, Austria
| | - Martin Pfister
- Center for Medical Physics and Biomedical Engineering, Medical University of Vienna, Austria
- Christian Doppler Laboratory for Ocular and Dermal Effects of Thiomers, Medical University of Vienna, Austria
- Institute of Applied Physics, Vienna University of Technology, Vienna, Austria
| | - Gerhard Garhöfer
- Department of Clinical Pharmacology, Medical University of Vienna, Austria
| | - Leopold Schmetterer
- Center for Medical Physics and Biomedical Engineering, Medical University of Vienna, Austria
- Christian Doppler Laboratory for Ocular and Dermal Effects of Thiomers, Medical University of Vienna, Austria
- Department of Clinical Pharmacology, Medical University of Vienna, Austria
- Singapore Eye Research Institute, Singapore National Eye Centre, Singapore
- Lee Kong Chian School of Medicine, Nanyang Technological University, Singapore
- Ophthalmology and Visual Sciences Academic Clinical Program, Duke-NUS Medical School, Singapore
| | - Valentin Aranha dos Santos
- Center for Medical Physics and Biomedical Engineering, Medical University of Vienna, Austria
- Christian Doppler Laboratory for Ocular and Dermal Effects of Thiomers, Medical University of Vienna, Austria
| |
Collapse
|
47
|
Zeng T, So HKH, Lam EY. RedCap: residual encoder-decoder capsule network for holographic image reconstruction. OPTICS EXPRESS 2020; 28:4876-4887. [PMID: 32121718 DOI: 10.1364/oe.383350] [Citation(s) in RCA: 11] [Impact Index Per Article: 2.8] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Received: 11/18/2019] [Accepted: 01/27/2020] [Indexed: 06/10/2023]
Abstract
A capsule network, as an advanced technique in deep learning, is designed to overcome information loss in the pooling operation and internal data representation of a convolutional neural network (CNN). It has shown promising results in several applications, such as digit recognition and image segmentation. In this work, we investigate for the first time the use of capsule network in digital holographic reconstruction. The proposed residual encoder-decoder capsule network, which we call RedCap, uses a novel windowed spatial dynamic routing algorithm and residual capsule block, which extends the idea of a residual block. Compared with the CNN-based neural network, RedCap exhibits much better experimental results in digital holographic reconstruction, while having a dramatic 75% reduction in the number of parameters. It indicates that RedCap is more efficient in the way it processes data and requires a much less memory storage for the learned model, which therefore makes it possible to be applied to some challenging situations with limited computational resources, such as portable devices.
Collapse
|
48
|
Li K, Liang W, Yang Z, Liang Y, Wan S. Robust, accurate depth-resolved attenuation characterization in optical coherence tomography. BIOMEDICAL OPTICS EXPRESS 2020; 11:672-687. [PMID: 32206392 PMCID: PMC7041483 DOI: 10.1364/boe.382493] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Received: 11/11/2019] [Revised: 12/21/2019] [Accepted: 12/23/2019] [Indexed: 05/06/2023]
Abstract
Depth-resolved optical attenuation coefficient is a valuable tissue parameter that complements the intensity-based structural information in optical coherent tomography (OCT) imaging. Herein we systematically analyzed the under- and over-estimation bias of existing depth-resolved methods when applied to real biological tissues, and then proposed a new algorithm that remedies these issues and accommodates general OCT data that contain incomplete decay and noise floor, thereby affording consistent estimation accuracy for practical biological samples of different scattering properties. Compared with other algorithms, our method demonstrates remarkably improved estimation accuracy and numerical robustness, as validated via numerical simulations and on experimental OCT data obtained from both silicone-TiO2 phantoms and human ventral tongue leukoplakia samples.
Collapse
Affiliation(s)
- Kaiyan Li
- School of Biological Science & Medical Engineering, Southeast University, Nanjing, Jiangsu 210096, China
- Equal contribution
| | - Wenxuan Liang
- Depart of Biomedical Engineering, Johns Hopkins University, Baltimore, Maryland 21205, USA
- Currently with the Department of Biomedical Engineering, Columbia University, New York, New York 10027, USA
- Equal contribution
| | - Zihan Yang
- Institute of Modern Optics, Nankai University, Tianjin 300350, China
| | - Yanmei Liang
- Institute of Modern Optics, Nankai University, Tianjin 300350, China
| | - Suiren Wan
- School of Biological Science & Medical Engineering, Southeast University, Nanjing, Jiangsu 210096, China
| |
Collapse
|
49
|
Cai L, Hinkle JW, Arias D, Gorniak RJ, Lakhani PC, Flanders AE, Kuriyan AE. Applications of Artificial Intelligence for the Diagnosis, Prognosis, and Treatment of Age-related Macular Degeneration. Int Ophthalmol Clin 2020; 60:147-168. [PMID: 33093323 DOI: 10.1097/iio.0000000000000334] [Citation(s) in RCA: 3] [Impact Index Per Article: 0.8] [Reference Citation Analysis] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 06/11/2023]
|
50
|
He Y, Carass A, Liu Y, Jedynak BM, Solomon SD, Saidha S, Calabresi PA, Prince JL. Deep learning based topology guaranteed surface and MME segmentation of multiple sclerosis subjects from retinal OCT. BIOMEDICAL OPTICS EXPRESS 2019; 10:5042-5058. [PMID: 31646029 PMCID: PMC6788619 DOI: 10.1364/boe.10.005042] [Citation(s) in RCA: 18] [Impact Index Per Article: 3.6] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 08/01/2019] [Revised: 09/01/2019] [Accepted: 09/02/2019] [Indexed: 05/11/2023]
Abstract
Optical coherence tomography (OCT) is a noninvasive imaging modality that can be used to obtain depth images of the retina. Patients with multiple sclerosis (MS) have thinning retinal nerve fiber and ganglion cell layers, and approximately 5% of MS patients will develop microcystic macular edema (MME) within the retina. Segmentation of both the retinal layers and MME can provide important information to help monitor MS progression. Graph-based segmentation with machine learning preprocessing is the leading method for retinal layer segmentation, providing accurate surface delineations with the correct topological ordering. However, graph methods are time-consuming and they do not optimally incorporate joint MME segmentation. This paper presents a deep network that extracts continuous, smooth, and topology-guaranteed surfaces and MMEs. The network learns shape priors automatically during training rather than being hard-coded as in graph methods. In this new approach, retinal surfaces and MMEs are segmented together with two cascaded deep networks in a single feed forward propagation. The proposed framework obtains retinal surfaces (separating the layers) with sub-pixel surface accuracy comparable to the best existing graph methods and MMEs with better accuracy than the state-of-the-art method. The full segmentation operation takes only ten seconds for a 3D volume.
Collapse
Affiliation(s)
- Yufan He
- Deptartment of Electrical and Computer Engineering, The Johns Hopkins University, Baltimore, MD 21218, USA
| | - Aaron Carass
- Deptartment of Electrical and Computer Engineering, The Johns Hopkins University, Baltimore, MD 21218, USA
- Department of Computer Science, The Johns Hopkins University, Baltimore, MD 21218, USA
| | - Yihao Liu
- Deptartment of Electrical and Computer Engineering, The Johns Hopkins University, Baltimore, MD 21218, USA
| | - Bruno M. Jedynak
- Department of Mathematics & Statistics, Portland State University, Portland, OR 97201, USA
| | - Sharon D. Solomon
- Wilmer Eye Institute, The Johns Hopkins University School of Medicine, MD 21287, USA
| | - Shiv Saidha
- Department of Neurology, The Johns Hopkins University School of Medicine, MD 21287, USA
| | - Peter A. Calabresi
- Department of Neurology, The Johns Hopkins University School of Medicine, MD 21287, USA
| | - Jerry L. Prince
- Deptartment of Electrical and Computer Engineering, The Johns Hopkins University, Baltimore, MD 21218, USA
- Department of Computer Science, The Johns Hopkins University, Baltimore, MD 21218, USA
| |
Collapse
|