1
|
Cao Y, le Yu X, Yao H, Jin Y, Lin K, Shi C, Cheng H, Lin Z, Jiang J, Gao H, Shen M. ScLNet: A cornea with scleral lens OCT layers segmentation dataset and new multi-task model. Heliyon 2024; 10:e33911. [PMID: 39071564 PMCID: PMC11283045 DOI: 10.1016/j.heliyon.2024.e33911] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/16/2024] [Revised: 06/28/2024] [Accepted: 06/28/2024] [Indexed: 07/30/2024] Open
Abstract
Objective To develop deep learning methods with high accuracy for segmenting irregular corneas and detecting the tear fluid reservoir (TFR) boundary under the scleral lens. Additionally, this study aims to provide a publicly available cornea with scleral lens OCT dataset, including manually labeled layer masks for training and validation of segmentation algorithms. This study introduces ScLNet, a dataset comprising cornea with Scleral Lens (ScL) optical coherence tomography (OCT) images with layer annotations, and a multi-task network designed to achieve rapid, accurate, automated segmentation of scleral lens with regular and irregular corneas. Methods We created a dataset comprising 31,360 OCT images with scleral lens annotations. The network architecture includes an encoder with multi-scale input and a context coding layer, along with two decoders for specific tasks. The primary task focuses on predicting ScL, TFR, and cornea regions, while the auxiliary task, aimed at predicting the boundaries of ScL, TFR, and cornea, enhances feature extraction for the main task. Segmentation results were compared with state-of-the-art methods and evaluated using Dice similarity coefficient (DSC), intersection over union (IoU), Matthews correlation coefficient (MCC), Precision, and Hausdorff distance (HD). Results ScLNet achieves 98.22 % DSC, 96.50 % IoU, 98.13 % MCC, 98.35 % Precision, and 3.6840 HD (in pixels) in segmenting ScL; 97.78 % DSC, 95.66 % IoU, 97.71 % MCC, 97.70 % Precision, and 3.7838 HD (in pixels) in segmenting TFR; and 99.22 % DSC, 98.45 % IoU, 99.15 % MCC, 99.14 % Precision, and 3.5355 HD (in pixels) in segmenting cornea. The layer interfaces recognized by ScLNet closely align with expert annotations, as evidenced by high IoU scores. Boundary metrics further confirm its effectiveness. Conclusion We constructed a dataset of corneal OCT images with ScL wearing, which includes regular and irregular cornea patients. The proposed ScLNet achieves high accuracy in extracting ScL, TFR, and corneal layer masks and boundaries from OCT images of the dataset.
Collapse
Affiliation(s)
- Yang Cao
- Oujiang Laboratory (Zhejiang Lab for Regenerative Medicine, Vision and Brain Health), Eye Hospital and School of Ophthalmology and Optometry, Wenzhou Medical University, Wenzhou, 325000, China
| | - Xiang le Yu
- Oujiang Laboratory (Zhejiang Lab for Regenerative Medicine, Vision and Brain Health), Eye Hospital and School of Ophthalmology and Optometry, Wenzhou Medical University, Wenzhou, 325000, China
| | - Han Yao
- Oujiang Laboratory (Zhejiang Lab for Regenerative Medicine, Vision and Brain Health), Eye Hospital and School of Ophthalmology and Optometry, Wenzhou Medical University, Wenzhou, 325000, China
| | - Yue Jin
- Oujiang Laboratory (Zhejiang Lab for Regenerative Medicine, Vision and Brain Health), Eye Hospital and School of Ophthalmology and Optometry, Wenzhou Medical University, Wenzhou, 325000, China
| | - Kuangqing Lin
- Oujiang Laboratory (Zhejiang Lab for Regenerative Medicine, Vision and Brain Health), Eye Hospital and School of Ophthalmology and Optometry, Wenzhou Medical University, Wenzhou, 325000, China
| | - Ce Shi
- Oujiang Laboratory (Zhejiang Lab for Regenerative Medicine, Vision and Brain Health), Eye Hospital and School of Ophthalmology and Optometry, Wenzhou Medical University, Wenzhou, 325000, China
| | - Hongling Cheng
- Oujiang Laboratory (Zhejiang Lab for Regenerative Medicine, Vision and Brain Health), Eye Hospital and School of Ophthalmology and Optometry, Wenzhou Medical University, Wenzhou, 325000, China
| | - Zhiyang Lin
- Oujiang Laboratory (Zhejiang Lab for Regenerative Medicine, Vision and Brain Health), Eye Hospital and School of Ophthalmology and Optometry, Wenzhou Medical University, Wenzhou, 325000, China
| | - Jun Jiang
- Oujiang Laboratory (Zhejiang Lab for Regenerative Medicine, Vision and Brain Health), Eye Hospital and School of Ophthalmology and Optometry, Wenzhou Medical University, Wenzhou, 325000, China
| | - Hebei Gao
- Oujiang Laboratory (Zhejiang Lab for Regenerative Medicine, Vision and Brain Health), Eye Hospital and School of Ophthalmology and Optometry, Wenzhou Medical University, Wenzhou, 325000, China
- School of Artificial Intelligence, Wenzhou Polytechnic, Wenzhou, 325035, China
| | - Meixiao Shen
- Oujiang Laboratory (Zhejiang Lab for Regenerative Medicine, Vision and Brain Health), Eye Hospital and School of Ophthalmology and Optometry, Wenzhou Medical University, Wenzhou, 325000, China
| |
Collapse
|
2
|
Hosseini F, Asadi F, Rabiei R, Kiani F, Harari RE. Applications of artificial intelligence in diagnosis of uncommon cystoid macular edema using optical coherence tomography imaging: A systematic review. Surv Ophthalmol 2024:S0039-6257(24)00073-0. [PMID: 38942125 DOI: 10.1016/j.survophthal.2024.06.005] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/07/2024] [Revised: 06/20/2024] [Accepted: 06/24/2024] [Indexed: 06/30/2024]
Abstract
Cystoid macular edema (CME) is a sight-threatening condition often associated with inflammatory and diabetic diseases. Early detection is crucial to prevent irreversible vision loss. Artificial intelligence (AI) has shown promise in automating CME diagnosis through optical coherence tomography (OCT) imaging, but its utility needs critical evaluation. This systematic review assesses the application of AI to diagnosis CME, specifically focusing on disorders like postoperative CME (Irvine Gass syndrome) and retinitis pigmentosa without obvious vasculopathy, using OCT imaging. A comprehensive search was conducted across 6 databases (PubMed, Scopus, Web of Science, Wiley, ScienceDirect, and IEEE) from 2018 to November, 2023. Twenty-three articles met the inclusion criteria and were selected for in-depth analysis. We evaluate AI's role in CME diagnosis and its performance in "detection", "classification", and "segmentation" of OCT retinal images. We found that convolutional neural network (CNN)-based methods consistently outperformed other machine learning techniques, achieving an average accuracy of over 96 % in detecting and identifying CME from OCT images. Despite certain limitations such as dataset size and ethical concerns, the synergy between AI and OCT, particularly through CNNs, holds promise for significantly advancing CME diagnostics.
Collapse
Affiliation(s)
- Farhang Hosseini
- Department of Health Information Technology and Management, School of Allied Medical Sciences, Shahid Beheshti University of Medical Sciences, Tehran, Iran
| | - Farkhondeh Asadi
- Department of Health Information Technology and Management, School of Allied Medical Sciences, Shahid Beheshti University of Medical Sciences, Tehran, Iran.
| | - Reza Rabiei
- Department of Health Information Technology and Management, School of Allied Medical Sciences, Shahid Beheshti University of Medical Sciences, Tehran, Iran.
| | - Fatemeh Kiani
- Department of Health Information Technology and Management, School of Allied Medical Sciences, Shahid Beheshti University of Medical Sciences, Tehran, Iran
| | - Rayan Ebnali Harari
- STRATUS Center for Medical Simulation, Harvard Medical School, Boston, MA, USA.
| |
Collapse
|
3
|
Demuth S, Paris J, Faddeenkov I, De Sèze J, Gourraud PA. Clinical applications of deep learning in neuroinflammatory diseases: A scoping review. Rev Neurol (Paris) 2024:S0035-3787(24)00522-8. [PMID: 38772806 DOI: 10.1016/j.neurol.2024.04.004] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/18/2024] [Revised: 03/26/2024] [Accepted: 04/09/2024] [Indexed: 05/23/2024]
Abstract
BACKGROUND Deep learning (DL) is an artificial intelligence technology that has aroused much excitement for predictive medicine due to its ability to process raw data modalities such as images, text, and time series of signals. OBJECTIVES Here, we intend to give the clinical reader elements to understand this technology, taking neuroinflammatory diseases as an illustrative use case of clinical translation efforts. We reviewed the scope of this rapidly evolving field to get quantitative insights about which clinical applications concentrate the efforts and which data modalities are most commonly used. METHODS We queried the PubMed database for articles reporting DL algorithms for clinical applications in neuroinflammatory diseases and the radiology.healthairegister.com website for commercial algorithms. RESULTS The review included 148 articles published between 2018 and 2024 and five commercial algorithms. The clinical applications could be grouped as computer-aided diagnosis, individual prognosis, functional assessment, the segmentation of radiological structures, and the optimization of data acquisition. Our review highlighted important discrepancies in efforts. The segmentation of radiological structures and computer-aided diagnosis currently concentrate most efforts with an overrepresentation of imaging. Various model architectures have addressed different applications, relatively low volume of data, and diverse data modalities. We report the high-level technical characteristics of the algorithms and synthesize narratively the clinical applications. Predictive performances and some common a priori on this topic are finally discussed. CONCLUSION The currently reported efforts position DL as an information processing technology, enhancing existing modalities of paraclinical investigations and bringing perspectives to make innovative ones actionable for healthcare.
Collapse
Affiliation(s)
- S Demuth
- Inserm U1064, CR2TI - Center for Research in Transplantation and Translational Immunology, Nantes University, 44000 Nantes, France; Inserm U1119 : biopathologie de la myéline, neuroprotection et stratégies thérapeutiques, University of Strasbourg, 1, rue Eugène-Boeckel - CS 60026, 67084 Strasbourg, France.
| | - J Paris
- Inserm U1064, CR2TI - Center for Research in Transplantation and Translational Immunology, Nantes University, 44000 Nantes, France
| | - I Faddeenkov
- Inserm U1064, CR2TI - Center for Research in Transplantation and Translational Immunology, Nantes University, 44000 Nantes, France
| | - J De Sèze
- Inserm U1119 : biopathologie de la myéline, neuroprotection et stratégies thérapeutiques, University of Strasbourg, 1, rue Eugène-Boeckel - CS 60026, 67084 Strasbourg, France; Department of Neurology, University Hospital of Strasbourg, 1, avenue Molière, 67200 Strasbourg, France; Inserm CIC 1434 Clinical Investigation Center, University Hospital of Strasbourg, 1, avenue Molière, 67200 Strasbourg, France
| | - P-A Gourraud
- Inserm U1064, CR2TI - Center for Research in Transplantation and Translational Immunology, Nantes University, 44000 Nantes, France; "Data clinic", Department of Public Health, University Hospital of Nantes, Nantes, France
| |
Collapse
|
4
|
Liu X, Li X, Zhang Y, Wang M, Yao J, Tang J. Boundary-Repairing Dual-Path Network for Retinal Layer Segmentation in OCT Image with Pigment Epithelial Detachment. JOURNAL OF IMAGING INFORMATICS IN MEDICINE 2024:10.1007/s10278-024-01093-y. [PMID: 38740662 DOI: 10.1007/s10278-024-01093-y] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 10/05/2023] [Revised: 03/05/2024] [Accepted: 03/06/2024] [Indexed: 05/16/2024]
Abstract
Automatic retinal layer segmentation in optical coherence tomography (OCT) images is crucial for the diagnosis of ocular diseases. Currently, automatic retinal layer segmentation works well with normal OCT images. However, pigment epithelial detachment (PED) dramatically alters the retinal structure, causing blurred boundaries and partial disappearance of the Bruch's Membrane (BM), thus posing challenges to the segmentation. To tackle these problems, we propose a novel dual-path U-shaped network for simultaneous layer segmentation and boundary regression. This network first designs a feature interaction fusion (FIF) module to strengthen the boundary shape constraints in the layer path. To address the challenge posed by partial BM disappearance and boundary-blurring, we propose a layer boundary repair (LBR) module. This module aims to use contrastive loss to enhance the confidence of blurred boundary regions and refine the segmentation of layer boundaries through the re-prediction head. In addition, we introduce a novel bilateral threshold distance map (BTDM) designed for the boundary path. The BTDM serves to emphasize information within boundary regions. This map, combined with the updated probability map, culminates in topology-guaranteed segmentation results achieved through a topology correction (TC) module. We investigated the proposed network on two severely deformed datasets (i.e., OCTA-500 and Aier-PED) and one slightly deformed dataset (i.e., DUKE). The proposed method achieves an average Dice score of 94.26% on the OCTA-500 dataset, which was 1.5% higher than BAU-Net and outperformed other methods. In the DUKE and Aier-PED datasets, the proposed method achieved average Dice scores of 91.65% and 95.75%, respectively.
Collapse
Affiliation(s)
- Xiaoming Liu
- School of Computer Science and Technology, Wuhan University of Science and Technology, Wuhan, 430065, China.
- Hubei Province Key Laboratory of Intelligent Information Processing and Real-Time Industrial System, Wuhan, 430065, China.
| | - Xiao Li
- School of Computer Science and Technology, Wuhan University of Science and Technology, Wuhan, 430065, China
- Hubei Province Key Laboratory of Intelligent Information Processing and Real-Time Industrial System, Wuhan, 430065, China
| | - Ying Zhang
- Wuhan Aier Eye Hospital of Wuhan University, Wuhan, China
| | - Man Wang
- Wuhan Aier Eye Hospital of Wuhan University, Wuhan, China
| | - Junping Yao
- Department of Ophthalmology, Tianyou Hospital Affiliated to Wuhan University of Science and Technology, Wuhan, China
| | - Jinshan Tang
- Department of Health Administration and Policy, College of Health and Human Services, George Mason University, Fairfax, VA, 22030, USA
| |
Collapse
|
5
|
Li D, Ran AR, Cheung CY, Prince JL. Deep learning in optical coherence tomography: Where are the gaps? Clin Exp Ophthalmol 2023; 51:853-863. [PMID: 37245525 PMCID: PMC10825778 DOI: 10.1111/ceo.14258] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/31/2023] [Revised: 04/24/2023] [Accepted: 05/03/2023] [Indexed: 05/30/2023]
Abstract
Optical coherence tomography (OCT) is a non-invasive optical imaging modality, which provides rapid, high-resolution and cross-sectional morphology of macular area and optic nerve head for diagnosis and managing of different eye diseases. However, interpreting OCT images requires experts in both OCT images and eye diseases since many factors such as artefacts and concomitant diseases can affect the accuracy of quantitative measurements made by post-processing algorithms. Currently, there is a growing interest in applying deep learning (DL) methods to analyse OCT images automatically. This review summarises the trends in DL-based OCT image analysis in ophthalmology, discusses the current gaps, and provides potential research directions. DL in OCT analysis shows promising performance in several tasks: (1) layers and features segmentation and quantification; (2) disease classification; (3) disease progression and prognosis; and (4) referral triage level prediction. Different studies and trends in the development of DL-based OCT image analysis are described and the following challenges are identified and described: (1) public OCT data are scarce and scattered; (2) models show performance discrepancies in real-world settings; (3) models lack of transparency; (4) there is a lack of societal acceptance and regulatory standards; and (5) OCT is still not widely available in underprivileged areas. More work is needed to tackle the challenges and gaps, before DL is further applied in OCT image analysis for clinical use.
Collapse
Affiliation(s)
- Dawei Li
- College of Future Technology, Peking University, Beijing, China
| | - An Ran Ran
- Department of Ophthalmology and Visual Sciences, The Chinese University of Hong Kong, Hong Kong SAR, China
| | - Carol Y. Cheung
- Department of Ophthalmology and Visual Sciences, The Chinese University of Hong Kong, Hong Kong SAR, China
| | - Jerry L. Prince
- Department of Electrical and Computer Engineering, Johns Hopkins University, Baltimore, Maryland, USA
| |
Collapse
|
6
|
Shen Y, Li J, Zhu W, Yu K, Wang M, Peng Y, Zhou Y, Guan L, Chen X. Graph Attention U-Net for Retinal Layer Surface Detection and Choroid Neovascularization Segmentation in OCT Images. IEEE TRANSACTIONS ON MEDICAL IMAGING 2023; 42:3140-3154. [PMID: 37022267 DOI: 10.1109/tmi.2023.3240757] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/19/2023]
Abstract
Choroidal neovascularization (CNV) is a typical symptom of age-related macular degeneration (AMD) and is one of the leading causes for blindness. Accurate segmentation of CNV and detection of retinal layers are critical for eye disease diagnosis and monitoring. In this paper, we propose a novel graph attention U-Net (GA-UNet) for retinal layer surface detection and CNV segmentation in optical coherence tomography (OCT) images. Due to retinal layer deformation caused by CNV, it is challenging for existing models to segment CNV and detect retinal layer surfaces with the correct topological order. We propose two novel modules to address the challenge. The first module is a graph attention encoder (GAE) in a U-Net model that automatically integrates topological and pathological knowledge of retinal layers into the U-Net structure to achieve effective feature embedding. The second module is a graph decorrelation module (GDM) that takes reconstructed features by the decoder of the U-Net as inputs, it then decorrelates and removes information unrelated to retinal layer for improved retinal layer surface detection. In addition, we propose a new loss function to maintain the correct topological order of retinal layers and the continuity of their boundaries. The proposed model learns graph attention maps automatically during training and performs retinal layer surface detection and CNV segmentation simultaneously with the attention maps during inference. We evaluated the proposed model on our private AMD dataset and another public dataset. Experiment results show that the proposed model outperformed the competing methods for retinal layer surface detection and CNV segmentation and achieved new state of the arts on the datasets.
Collapse
|
7
|
Gende M, Mallen V, de Moura J, Cordon B, Garcia-Martin E, Sanchez CI, Novo J, Ortega M. Automatic Segmentation of Retinal Layers in Multiple Neurodegenerative Disorder Scenarios. IEEE J Biomed Health Inform 2023; 27:5483-5494. [PMID: 37682646 DOI: 10.1109/jbhi.2023.3313392] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 09/10/2023]
Abstract
Retinal Optical Coherence Tomography (OCT) allows the non-invasive direct observation of the central nervous system, enabling the measurement and extraction of biomarkers from neural tissue that can be helpful in the assessment of ocular, systemic and Neurological Disorders (ND). Deep learning models can be trained to segment the retinal layers for biomarker extraction. However, the onset of ND can have an impact on the neural tissue, which can lead to the degraded performance of models not exposed to images displaying signs of disease during training. We present a fully automatic approach for the retinal layer segmentation in multiple neurodegenerative disorder scenarios, using an annotated dataset of patients of the most prevalent NDs: Alzheimer's disease, Parkinson's disease, multiple sclerosis and essential tremor, along with healthy control patients. Furthermore, we present a two-part, comprehensive study on the effects of ND on the performance of these models. The results show that images of healthy patients may not be sufficient for the robust training of automated segmentation models intended for the analysis of ND patients, and that using images representative of different NDs can increase the model performance. These results indicate that the presence or absence of patients of ND in datasets should be taken into account when training deep learning models for retinal layer segmentation, and that the proposed approach can provide a valuable tool for the robust and reliable diagnosis in multiple scenarios of ND.
Collapse
|
8
|
Vázquez-Marrufo M, Sarrias-Arrabal E, García-Torres M, Martín-Clemente R, Izquierdo G. A systematic review of the application of machine-learning algorithms in multiple sclerosis. Neurologia 2023; 38:577-590. [PMID: 35843587 DOI: 10.1016/j.nrleng.2020.10.013] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/05/2020] [Accepted: 10/11/2020] [Indexed: 10/17/2022] Open
Abstract
INTRODUCTION The applications of artificial intelligence, and in particular automatic learning or "machine learning" (ML), constitute both a challenge and a great opportunity in numerous scientific, technical, and clinical disciplines. Specific applications in the study of multiple sclerosis (MS) have been no exception, and constitute an area of increasing interest in recent years. OBJECTIVE We present a systematic review of the application of ML algorithms in MS. MATERIALS AND METHODS We used the PubMed search engine, which allows free access to the MEDLINE medical database, to identify studies including the keywords "machine learning" and "multiple sclerosis." We excluded review articles, studies written in languages other than English or Spanish, and studies that were mainly technical and did not specifically apply to MS. The final selection included 76 articles, and 38 were rejected. CONCLUSIONS After the review process, we established 4 main applications of ML in MS: 1) classifying MS subtypes; 2) distinguishing patients with MS from healthy controls and individuals with other diseases; 3) predicting progression and response to therapeutic interventions; and 4) other applications. Results found to date have shown that ML algorithms may offer great support for health professionals both in clinical settings and in research into MS.
Collapse
Affiliation(s)
- M Vázquez-Marrufo
- Departamento de Psicología Experimental, Facultad de Psicología, Universidad de Sevilla, Sevilla, Spain.
| | - E Sarrias-Arrabal
- Departamento de Psicología Experimental, Facultad de Psicología, Universidad de Sevilla, Sevilla, Spain
| | - M García-Torres
- Escuela Politécnica Superior, Universidad Pablo de Olavide, Sevilla, Spain
| | - R Martín-Clemente
- Departamento de Teoría de la Señal y Comunicaciones, Escuela Técnica Superior de Ingeniería, Universidad de Sevilla, Sevilla, Spain
| | - G Izquierdo
- Unidad de Esclerosis Múltiple, Hospital VITHAS, Sevilla, Spain
| |
Collapse
|
9
|
He Y, Carass A, Liu Y, Calabresi PA, Saidha S, Prince JL. Longitudinal deep network for consistent OCT layer segmentation. BIOMEDICAL OPTICS EXPRESS 2023; 14:1874-1893. [PMID: 37206119 PMCID: PMC10191669 DOI: 10.1364/boe.487518] [Citation(s) in RCA: 5] [Impact Index Per Article: 5.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Received: 02/10/2023] [Revised: 03/11/2023] [Accepted: 03/17/2023] [Indexed: 05/21/2023]
Abstract
Retinal layer thickness is an important bio-marker for people with multiple sclerosis (PwMS). In clinical practice, retinal layer thickness changes in optical coherence tomography (OCT) are widely used for monitoring multiple sclerosis (MS) progression. Recent developments in automated retinal layer segmentation algorithms allow cohort-level retina thinning to be observed in a large study of PwMS. However, variability in these results make it difficult to identify patient-level trends; this prevents patient specific disease monitoring and treatment planning using OCT. Deep learning based retinal layer segmentation algorithms have achieved state-of-the-art accuracy, but the segmentation is performed on each individual scan without utilizing longitudinal information, which can be important in reducing segmentation error and reveal subtle changes in retinal layers. In this paper, we propose a longitudinal OCT segmentation network which achieves more accurate and consistent layer thickness measurements for PwMS.
Collapse
Affiliation(s)
- Yufan He
- Dept. of Electrical and Computer Engineering, The Johns Hopkins University, Baltimore, MD 21218, USA
| | - Aaron Carass
- Dept. of Electrical and Computer Engineering, The Johns Hopkins University, Baltimore, MD 21218, USA
| | - Yihao Liu
- Dept. of Electrical and Computer Engineering, The Johns Hopkins University, Baltimore, MD 21218, USA
| | - Peter A. Calabresi
- Dept. of Neurology, The Johns Hopkins University School of Medicine, MD 21287, USA
| | - Shiv Saidha
- Dept. of Neurology, The Johns Hopkins University School of Medicine, MD 21287, USA
| | - Jerry L. Prince
- Dept. of Electrical and Computer Engineering, The Johns Hopkins University, Baltimore, MD 21218, USA
| |
Collapse
|
10
|
Lou S, Chen X, Wang Y, Cai H, Chen S, Liu L. Multiscale joint segmentation method for retinal optical coherence tomography images using a bidirectional wave algorithm and improved graph theory. OPTICS EXPRESS 2023; 31:6862-6876. [PMID: 36823933 DOI: 10.1364/oe.472154] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 08/01/2022] [Accepted: 12/16/2022] [Indexed: 06/18/2023]
Abstract
Morphology and functional metrics of retinal layers are important biomarkers for many human ophthalmic diseases. Automatic and accurate segmentation of retinal layers is crucial for disease diagnosis and research. To improve the performance of retinal layer segmentation, a multiscale joint segmentation framework for retinal optical coherence tomography (OCT) images based on bidirectional wave algorithm and improved graph theory is proposed. In this framework, the bidirectional wave algorithm was used to segment edge information in multiscale images, and the improved graph theory was used to modify edge information globally, to realize automatic and accurate segmentation of eight retinal layer boundaries. This framework was tested on two public datasets and two OCT imaging systems. The test results show that, compared with other state-of-the-art methods, this framework does not need data pre-training and parameter pre-adjustment on different datasets, and can achieve sub-pixel retinal layer segmentation on a low-configuration computer.
Collapse
|
11
|
Yang H, Chen C, Jiang M, Liu Q, Cao J, Heng PA, Dou Q. DLTTA: Dynamic Learning Rate for Test-Time Adaptation on Cross-Domain Medical Images. IEEE TRANSACTIONS ON MEDICAL IMAGING 2022; 41:3575-3586. [PMID: 35839185 DOI: 10.1109/tmi.2022.3191535] [Citation(s) in RCA: 6] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/15/2023]
Abstract
Test-time adaptation (TTA) has increasingly been an important topic to efficiently tackle the cross-domain distribution shift at test time for medical images from different institutions. Previous TTA methods have a common limitation of using a fixed learning rate for all the test samples. Such a practice would be sub-optimal for TTA, because test data may arrive sequentially therefore the scale of distribution shift would change frequently. To address this problem, we propose a novel dynamic learning rate adjustment method for test-time adaptation, called DLTTA, which dynamically modulates the amount of weights update for each test image to account for the differences in their distribution shift. Specifically, our DLTTA is equipped with a memory bank based estimation scheme to effectively measure the discrepancy of a given test sample. Based on this estimated discrepancy, a dynamic learning rate adjustment strategy is then developed to achieve a suitable degree of adaptation for each test sample. The effectiveness and general applicability of our DLTTA is extensively demonstrated on three tasks including retinal optical coherence tomography (OCT) segmentation, histopathological image classification, and prostate 3D MRI segmentation. Our method achieves effective and fast test-time adaptation with consistent performance improvement over current state-of-the-art test-time adaptation methods. Code is available at https://github.com/med-air/DLTTA.
Collapse
|
12
|
|
13
|
Yadav SK, Kafieh R, Zimmermann HG, Kauer-Bonin J, Nouri-Mahdavi K, Mohammadzadeh V, Shi L, Kadas EM, Paul F, Motamedi S, Brandt AU. Intraretinal Layer Segmentation Using Cascaded Compressed U-Nets. J Imaging 2022; 8:139. [PMID: 35621903 PMCID: PMC9146486 DOI: 10.3390/jimaging8050139] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/31/2022] [Revised: 04/23/2022] [Accepted: 05/03/2022] [Indexed: 12/24/2022] Open
Abstract
Reliable biomarkers quantifying neurodegeneration and neuroinflammation in central nervous system disorders such as Multiple Sclerosis, Alzheimer's dementia or Parkinson's disease are an unmet clinical need. Intraretinal layer thicknesses on macular optical coherence tomography (OCT) images are promising noninvasive biomarkers querying neuroretinal structures with near cellular resolution. However, changes are typically subtle, while tissue gradients can be weak, making intraretinal segmentation a challenging task. A robust and efficient method that requires no or minimal manual correction is an unmet need to foster reliable and reproducible research as well as clinical application. Here, we propose and validate a cascaded two-stage network for intraretinal layer segmentation, with both networks being compressed versions of U-Net (CCU-INSEG). The first network is responsible for retinal tissue segmentation from OCT B-scans. The second network segments eight intraretinal layers with high fidelity. At the post-processing stage, we introduce Laplacian-based outlier detection with layer surface hole filling by adaptive non-linear interpolation. Additionally, we propose a weighted version of focal loss to minimize the foreground-background pixel imbalance in the training data. We train our method using 17,458 B-scans from patients with autoimmune optic neuropathies, i.e., multiple sclerosis, and healthy controls. Voxel-wise comparison against manual segmentation produces a mean absolute error of 2.3 μm, outperforming current state-of-the-art methods on the same data set. Voxel-wise comparison against external glaucoma data leads to a mean absolute error of 2.6 μm when using the same gold standard segmentation approach, and 3.7 μm mean absolute error in an externally segmented data set. In scans from patients with severe optic atrophy, 3.5% of B-scan segmentation results were rejected by an experienced grader, whereas this was the case in 41.4% of B-scans segmented with a graph-based reference method. The validation results suggest that the proposed method can robustly segment macular scans from eyes with even severe neuroretinal changes.
Collapse
Affiliation(s)
- Sunil Kumar Yadav
- Experimental and Clinical Research Center, Max Delbrück Center for Molecular Medicine and Charité-Universitätsmedizin Berlin, Corporate Member of Freie Universität Berlin and Humboldt-Universität zu Berlin, 13125 Berlin, Germany; (S.K.Y.); (R.K.); (H.G.Z.); (J.K.-B.); (F.P.); (S.M.)
- Nocturne GmbH, 10119 Berlin, Germany;
| | - Rahele Kafieh
- Experimental and Clinical Research Center, Max Delbrück Center for Molecular Medicine and Charité-Universitätsmedizin Berlin, Corporate Member of Freie Universität Berlin and Humboldt-Universität zu Berlin, 13125 Berlin, Germany; (S.K.Y.); (R.K.); (H.G.Z.); (J.K.-B.); (F.P.); (S.M.)
| | - Hanna Gwendolyn Zimmermann
- Experimental and Clinical Research Center, Max Delbrück Center for Molecular Medicine and Charité-Universitätsmedizin Berlin, Corporate Member of Freie Universität Berlin and Humboldt-Universität zu Berlin, 13125 Berlin, Germany; (S.K.Y.); (R.K.); (H.G.Z.); (J.K.-B.); (F.P.); (S.M.)
| | - Josef Kauer-Bonin
- Experimental and Clinical Research Center, Max Delbrück Center for Molecular Medicine and Charité-Universitätsmedizin Berlin, Corporate Member of Freie Universität Berlin and Humboldt-Universität zu Berlin, 13125 Berlin, Germany; (S.K.Y.); (R.K.); (H.G.Z.); (J.K.-B.); (F.P.); (S.M.)
- Nocturne GmbH, 10119 Berlin, Germany;
| | - Kouros Nouri-Mahdavi
- Glaucoma Division, Stein Eye Institute, David Geffen School of Medicine, University of California Los Angeles, Los Angeles, CA 90095, USA; (K.N.-M.); (V.M.); (L.S.)
| | - Vahid Mohammadzadeh
- Glaucoma Division, Stein Eye Institute, David Geffen School of Medicine, University of California Los Angeles, Los Angeles, CA 90095, USA; (K.N.-M.); (V.M.); (L.S.)
| | - Lynn Shi
- Glaucoma Division, Stein Eye Institute, David Geffen School of Medicine, University of California Los Angeles, Los Angeles, CA 90095, USA; (K.N.-M.); (V.M.); (L.S.)
| | | | - Friedemann Paul
- Experimental and Clinical Research Center, Max Delbrück Center for Molecular Medicine and Charité-Universitätsmedizin Berlin, Corporate Member of Freie Universität Berlin and Humboldt-Universität zu Berlin, 13125 Berlin, Germany; (S.K.Y.); (R.K.); (H.G.Z.); (J.K.-B.); (F.P.); (S.M.)
- Department of Neurology, Charité-Universitätsmedizin Berlin, Corporate Member of Freie Universität Berlin and Humboldt-Universität zu Berlin, 10098 Berlin, Germany
| | - Seyedamirhosein Motamedi
- Experimental and Clinical Research Center, Max Delbrück Center for Molecular Medicine and Charité-Universitätsmedizin Berlin, Corporate Member of Freie Universität Berlin and Humboldt-Universität zu Berlin, 13125 Berlin, Germany; (S.K.Y.); (R.K.); (H.G.Z.); (J.K.-B.); (F.P.); (S.M.)
| | - Alexander Ulrich Brandt
- Experimental and Clinical Research Center, Max Delbrück Center for Molecular Medicine and Charité-Universitätsmedizin Berlin, Corporate Member of Freie Universität Berlin and Humboldt-Universität zu Berlin, 13125 Berlin, Germany; (S.K.Y.); (R.K.); (H.G.Z.); (J.K.-B.); (F.P.); (S.M.)
- Department of Neurology, University of California Irvine, Irvine, CA 92697, USA
| |
Collapse
|
14
|
He W, Sainju AM, Jiang Z, Yan D, Zhou Y. Earth Imagery Segmentation on Terrain Surface with Limited Training Labels: A Semi-supervised Approach based on Physics-Guided Graph Co-Training. ACM T INTEL SYST TEC 2022. [DOI: 10.1145/3481043] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/19/2022]
Abstract
Given earth imagery with spectral features on a terrain surface, this paper studies surface segmentation based on both explanatory features and surface topology. The problem is important in many spatial and spatiotemporal applications such as flood extent mapping in hydrology. The problem is uniquely challenging for several reasons: first, the size of earth imagery on a terrain surface is often much larger than the input of popular deep convolutional neural networks; second, there exists topological structure dependency between pixel classes on the surface, and such dependency can follow an unknown and non-linear distribution; third, there are often limited training labels. Existing methods for earth imagery segmentation often divide the imagery into patches and consider the elevation as an additional feature channel. These methods do not fully incorporate the spatial topological structural constraint within and across surface patches and thus often show poor results, especially when training labels are limited. Existing methods on semi-supervised and unsupervised learning for earth imagery often focus on learning representation without explicitly incorporating surface topology. In contrast, we propose a novel framework that explicitly models the topological skeleton of a terrain surface with a contour tree from computational topology, which is guided by the physical constraint (e.g., water flow direction on terrains). Our framework consists of two neural networks: a
convolutional neural network (CNN)
to learn spatial contextual features on a 2D image grid, and a
graph neural network (GNN)
to learn the statistical distribution of physics-guided spatial topological dependency on the contour tree. The two models are co-trained via variational EM. Evaluations on the real-world flood mapping datasets show that the proposed models outperform baseline methods in classification accuracy, especially when training labels are limited.
Collapse
Affiliation(s)
- Wenchong He
- Department of Computer & Information Science & Engineering, The University of Florida, Gainesville, Florida, USA
| | - Arpan Man Sainju
- Department of Computer Science, Middle Tennessee State University, Murfreesboro, TN, USA
| | - Zhe Jiang
- Department of Computer & Information Science & Engineering, The University of Florida, Gainesville, Florida, USA
| | - Da Yan
- Department of Computer Science, University of Alabama, Birmingham, AL, USA
| | - Yang Zhou
- Department of Computer Science and Software Engineering, Auburn University, Auburn, AL, USA
| |
Collapse
|
15
|
OCT Retinal and Choroidal Layer Instance Segmentation Using Mask R-CNN. SENSORS 2022; 22:s22052016. [PMID: 35271165 PMCID: PMC8914986 DOI: 10.3390/s22052016] [Citation(s) in RCA: 5] [Impact Index Per Article: 2.5] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 12/31/2021] [Revised: 03/01/2022] [Accepted: 03/02/2022] [Indexed: 11/16/2022]
Abstract
Optical coherence tomography (OCT) of the posterior segment of the eye provides high-resolution cross-sectional images that allow visualization of individual layers of the posterior eye tissue (the retina and choroid), facilitating the diagnosis and monitoring of ocular diseases and abnormalities. The manual analysis of retinal OCT images is a time-consuming task; therefore, the development of automatic image analysis methods is important for both research and clinical applications. In recent years, deep learning methods have emerged as an alternative method to perform this segmentation task. A large number of the proposed segmentation methods in the literature focus on the use of encoder–decoder architectures, such as U-Net, while other architectural modalities have not received as much attention. In this study, the application of an instance segmentation method based on region proposal architecture, called the Mask R-CNN, is explored in depth in the context of retinal OCT image segmentation. The importance of adequate hyper-parameter selection is examined, and the performance is compared with commonly used techniques. The Mask R-CNN provides a suitable method for the segmentation of OCT images with low segmentation boundary errors and high Dice coefficients, with segmentation performance comparable with the commonly used U-Net method. The Mask R-CNN has the advantage of a simpler extraction of the boundary positions, especially avoiding the need for a time-consuming graph search method to extract boundaries, which reduces the inference time by 2.5 times compared to U-Net, while segmenting seven retinal layers.
Collapse
|
16
|
López-Dorado A, Ortiz M, Satue M, Rodrigo MJ, Barea R, Sánchez-Morla EM, Cavaliere C, Rodríguez-Ascariz JM, Orduna-Hospital E, Boquete L, Garcia-Martin E. Early Diagnosis of Multiple Sclerosis Using Swept-Source Optical Coherence Tomography and Convolutional Neural Networks Trained with Data Augmentation. SENSORS (BASEL, SWITZERLAND) 2021; 22:167. [PMID: 35009710 PMCID: PMC8747672 DOI: 10.3390/s22010167] [Citation(s) in RCA: 10] [Impact Index Per Article: 3.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 11/03/2021] [Revised: 12/21/2021] [Accepted: 12/22/2021] [Indexed: 05/07/2023]
Abstract
BACKGROUND The aim of this paper is to implement a system to facilitate the diagnosis of multiple sclerosis (MS) in its initial stages. It does so using a convolutional neural network (CNN) to classify images captured with swept-source optical coherence tomography (SS-OCT). METHODS SS-OCT images from 48 control subjects and 48 recently diagnosed MS patients have been used. These images show the thicknesses (45 × 60 points) of the following structures: complete retina, retinal nerve fiber layer, two ganglion cell layers (GCL+, GCL++) and choroid. The Cohen distance is used to identify the structures and the regions within them with greatest discriminant capacity. The original database of OCT images is augmented by a deep convolutional generative adversarial network to expand the CNN's training set. RESULTS The retinal structures with greatest discriminant capacity are the GCL++ (44.99% of image points), complete retina (26.71%) and GCL+ (22.93%). Thresholding these images and using them as inputs to a CNN comprising two convolution modules and one classification module obtains sensitivity = specificity = 1.0. CONCLUSIONS Feature pre-selection and the use of a convolutional neural network may be a promising, nonharmful, low-cost, easy-to-perform and effective means of assisting the early diagnosis of MS based on SS-OCT thickness data.
Collapse
Affiliation(s)
- Almudena López-Dorado
- Biomedical Engineering Group, Department of Electronics, University of Alcalá, 28801 Alcalá de Henares, Spain; (A.L.-D.); (R.B.); (C.C.); (J.M.R.-A.)
| | - Miguel Ortiz
- Computer Vision, Imaging and Machine Intelligence Research Group, Interdisciplinary Center for Security, Reliability and Trust (SnT), University of Luxembourg, 4365 Luxembourg, Luxembourg;
| | - María Satue
- Miguel Servet Ophthalmology Innovation and Research Group (GIMSO), Department of Ophthalmology, Aragon Institute for Health Research (IIS Aragon), Miguel Servet University Hospital, University of Zaragoza, 50018 Zaragoza, Spain; (M.S.); (M.J.R.); (E.O.-H.)
| | - María J. Rodrigo
- Miguel Servet Ophthalmology Innovation and Research Group (GIMSO), Department of Ophthalmology, Aragon Institute for Health Research (IIS Aragon), Miguel Servet University Hospital, University of Zaragoza, 50018 Zaragoza, Spain; (M.S.); (M.J.R.); (E.O.-H.)
| | - Rafael Barea
- Biomedical Engineering Group, Department of Electronics, University of Alcalá, 28801 Alcalá de Henares, Spain; (A.L.-D.); (R.B.); (C.C.); (J.M.R.-A.)
| | - Eva M. Sánchez-Morla
- Department of Psychiatry, Hospital 12 de Octubre Research Institute (i+12), 28041 Madrid, Spain;
- Faculty of Medicine, Complutense University of Madrid, 28040 Madrid, Spain
- Biomedical Research Networking Centre in Mental Health (CIBERSAM), 28029 Madrid, Spain
| | - Carlo Cavaliere
- Biomedical Engineering Group, Department of Electronics, University of Alcalá, 28801 Alcalá de Henares, Spain; (A.L.-D.); (R.B.); (C.C.); (J.M.R.-A.)
| | - José M. Rodríguez-Ascariz
- Biomedical Engineering Group, Department of Electronics, University of Alcalá, 28801 Alcalá de Henares, Spain; (A.L.-D.); (R.B.); (C.C.); (J.M.R.-A.)
| | - Elvira Orduna-Hospital
- Miguel Servet Ophthalmology Innovation and Research Group (GIMSO), Department of Ophthalmology, Aragon Institute for Health Research (IIS Aragon), Miguel Servet University Hospital, University of Zaragoza, 50018 Zaragoza, Spain; (M.S.); (M.J.R.); (E.O.-H.)
| | - Luciano Boquete
- Biomedical Engineering Group, Department of Electronics, University of Alcalá, 28801 Alcalá de Henares, Spain; (A.L.-D.); (R.B.); (C.C.); (J.M.R.-A.)
| | - Elena Garcia-Martin
- Miguel Servet Ophthalmology Innovation and Research Group (GIMSO), Department of Ophthalmology, Aragon Institute for Health Research (IIS Aragon), Miguel Servet University Hospital, University of Zaragoza, 50018 Zaragoza, Spain; (M.S.); (M.J.R.); (E.O.-H.)
| |
Collapse
|
17
|
Stankiewicz A, Marciniak T, Dabrowski A, Stopa M, Marciniak E, Obara B. Segmentation of Preretinal Space in Optical Coherence Tomography Images Using Deep Neural Networks. SENSORS 2021; 21:s21227521. [PMID: 34833597 PMCID: PMC8623441 DOI: 10.3390/s21227521] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 09/28/2021] [Revised: 11/08/2021] [Accepted: 11/09/2021] [Indexed: 02/01/2023]
Abstract
This paper proposes an efficient segmentation of the preretinal area between the inner limiting membrane (ILM) and posterior cortical vitreous (PCV) of the human eye in an image obtained with the use of optical coherence tomography (OCT). The research was carried out using a database of three-dimensional OCT imaging scans obtained with the Optovue RTVue XR Avanti device. Various types of neural networks (UNet, Attention UNet, ReLayNet, LFUNet) were tested for semantic segmentation, their effectiveness was assessed using the Dice coefficient and compared to the graph theory techniques. Improvement in segmentation efficiency was achieved through the use of relative distance maps. We also show that selecting a larger kernel size for convolutional layers can improve segmentation quality depending on the neural network model. In the case of PVC, we obtain the effectiveness reaching up to 96.35%. The proposed solution can be widely used to diagnose vitreomacular traction changes, which is not yet available in scientific or commercial OCT imaging solutions.
Collapse
Affiliation(s)
- Agnieszka Stankiewicz
- Division of Electronic Systems and Signal Processing, Institute of Automatic Control and Robotics, Poznan University of Technology, 60-965 Poznan, Poland; (A.S.); (A.D.)
| | - Tomasz Marciniak
- Division of Electronic Systems and Signal Processing, Institute of Automatic Control and Robotics, Poznan University of Technology, 60-965 Poznan, Poland; (A.S.); (A.D.)
- Correspondence:
| | - Adam Dabrowski
- Division of Electronic Systems and Signal Processing, Institute of Automatic Control and Robotics, Poznan University of Technology, 60-965 Poznan, Poland; (A.S.); (A.D.)
| | - Marcin Stopa
- Department of Ophthalmology, Chair of Ophthalmology and Optometry, Heliodor Swiecicki University Hospital, Poznan University of Medical Sciences, 60-780 Poznan, Poland; (M.S.); (E.M.)
| | - Elzbieta Marciniak
- Department of Ophthalmology, Chair of Ophthalmology and Optometry, Heliodor Swiecicki University Hospital, Poznan University of Medical Sciences, 60-780 Poznan, Poland; (M.S.); (E.M.)
| | - Boguslaw Obara
- School of Computing, Newcastle University, Newcastle upon Tyne NE4 5TG, UK;
- Biosciences Institute, Newcastle University, Newcastle upon Tyne NE2 4HH, UK
| |
Collapse
|
18
|
Assignment Flow for Order-Constrained OCT Segmentation. Int J Comput Vis 2021. [DOI: 10.1007/s11263-021-01520-5] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/20/2022]
Abstract
AbstractAt the present time optical coherence tomography (OCT) is among the most commonly used non-invasive imaging methods for the acquisition of large volumetric scans of human retinal tissues and vasculature. The substantial increase of accessible highly resolved 3D samples at the optic nerve head and the macula is directly linked to medical advancements in early detection of eye diseases. To resolve decisive information from extracted OCT volumes and to make it applicable for further diagnostic analysis, the exact measurement of retinal layer thicknesses serves as an essential task be done for each patient separately. However, manual examination of OCT scans is a demanding and time consuming task, which is typically made difficult by the presence of tissue-dependent speckle noise. Therefore, the elaboration of automated segmentation models has become an important task in the field of medical image processing. We propose a novel, purely data driven geometric approach to order-constrained 3D OCT retinal cell layer segmentation which takes as input data in any metric space and can be implemented using only simple, highly parallelizable operations. As opposed to many established retinal layer segmentation methods, we use only locally extracted features as input and do not employ any global shape prior. The physiological order of retinal cell layers and membranes is achieved through the introduction of a smoothed energy term. This is combined with additional regularization of local smoothness to yield highly accurate 3D segmentations. The approach thereby systematically avoid bias pertaining to global shape and is hence suited for the detection of anatomical changes of retinal tissue structure. To demonstrate its robustness, we compare two different choices of features on a data set of manually annotated 3D OCT volumes of healthy human retina. The quality of computed segmentations is compared to the state of the art in automatic retinal layer segmention as well as to manually annotated ground truth data in terms of mean absolute error and Dice similarity coefficient. Visualizations of segmented volumes are also provided.
Collapse
|
19
|
Wang B, Wei W, Qiu S, Wang S, Li D, He H. Boundary Aware U-Net for Retinal Layers Segmentation in Optical Coherence Tomography Images. IEEE J Biomed Health Inform 2021; 25:3029-3040. [PMID: 33729959 DOI: 10.1109/jbhi.2021.3066208] [Citation(s) in RCA: 9] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/07/2022]
Abstract
Retinal layers segmentation in optical coherence tomography (OCT) images is a critical step in the diagnosis of numerous ocular diseases. Automatic layers segmentation requires separating each individual layer instance with accurate boundary detection, but remains a challenging task since it suffers from speckle noise, intensity inhomogeneity, and the low contrast around boundary. In this work, we proposed a boundary aware U-Net (BAU-Net) for retinal layers segmentation by detecting accurate boundary. Based on encoder-decoder architecture, we design a dual tasks framework with low-level outputs for boundary detection and high-level outputs for layers segmentation. Specifically, we first use the multi-scale input strategy to enrich the spatial information in the deep features of encoder. For low-level features from encoder, we design an edge aware (EA) module in skip connection to extract the pure edge features. Then, a U-structure feature enhanced (UFE) module is designed in all skip connections to enlarge the features receptive fields from the encoder. Besides, a canny edge fusion (CEF) module is introduced to aforementioned architecture, which can fuse the priory edge information from segmentation task to boundary detection branch for a better predication. Furthermore, we model each boundary as a vertical coordinates distribution for boundary detection. Based on this distribution, a topology guarantee loss with combined A-scan regression loss and structure loss is proposed to make an accurate and guaranteed topological boundary set. The method is evaluated on two public datasets and the results demonstrate that the BAU-Net achieves promising performance than other state-of-the-art methods.
Collapse
|
20
|
Wang J, Li W, Chen Y, Fang W, Kong W, He Y, Shi G. Weakly supervised anomaly segmentation in retinal OCT images using an adversarial learning approach. BIOMEDICAL OPTICS EXPRESS 2021; 12:4713-4729. [PMID: 34513220 PMCID: PMC8407839 DOI: 10.1364/boe.426803] [Citation(s) in RCA: 14] [Impact Index Per Article: 4.7] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Received: 04/02/2021] [Revised: 06/17/2021] [Accepted: 06/26/2021] [Indexed: 05/09/2023]
Abstract
Lesion detection is a critical component of disease diagnosis, but the manual segmentation of lesions in medical images is time-consuming and experience-demanding. These issues have recently been addressed through deep learning models. However, most of the existing algorithms were developed using supervised training, which requires time-intensive manual labeling and prevents the model from detecting unaware lesions. As such, this study proposes a weakly supervised learning network based on CycleGAN for lesions segmentation in full-width optical coherence tomography (OCT) images. The model was trained to reconstruct underlying normal anatomic structures from abnormal input images, then the lesions can be detected by calculating the difference between the input and output images. A customized network architecture and a multi-scale similarity perceptual reconstruction loss were used to extend the CycleGAN model to transfer between objects exhibiting shape deformations. The proposed technique was validated using an open-source retinal OCT image dataset. Image-level anomaly detection and pixel-level lesion detection results were assessed using area-under-curve (AUC) and the Dice similarity coefficient, producing results of 96.94% and 0.8239, respectively, higher than all comparative methods. The average test time required to generate a single full-width image was 0.039 s, which is shorter than that reported in recent studies. These results indicate that our model can accurately detect and segment retinopathy lesions in real-time, without the need for supervised labeling. And we hope this method will be helpful to accelerate the clinical diagnosis process and reduce the misdiagnosis rate.
Collapse
Affiliation(s)
- Jing Wang
- Jiangsu Key Laboratory of Medical Optics, Suzhou Institute of Biomedical Engineering and Technology, Chinese Academy of Science, Suzhou 215163, China
- School of Biomedical Engineering (Suzhou), Division of Life Sciences and Medicine, University of Science and Technology of China, Hefei 230026, China
| | - Wanyue Li
- Jiangsu Key Laboratory of Medical Optics, Suzhou Institute of Biomedical Engineering and Technology, Chinese Academy of Science, Suzhou 215163, China
- School of Biomedical Engineering (Suzhou), Division of Life Sciences and Medicine, University of Science and Technology of China, Hefei 230026, China
| | - Yiwei Chen
- School of Biomedical Engineering (Suzhou), Division of Life Sciences and Medicine, University of Science and Technology of China, Hefei 230026, China
| | - Wangyi Fang
- Department of Ophthalmology and Vision Science, Eye and ENT Hospital, Fudan University, Shanghai 201112, China
- Key Laboratory of Myopia of State Health Ministry, and Key Laboratory of Visual Impairment and Restoration of Shanghai, Shanghai 200003, China
| | - Wen Kong
- Jiangsu Key Laboratory of Medical Optics, Suzhou Institute of Biomedical Engineering and Technology, Chinese Academy of Science, Suzhou 215163, China
- School of Biomedical Engineering (Suzhou), Division of Life Sciences and Medicine, University of Science and Technology of China, Hefei 230026, China
| | - Yi He
- School of Biomedical Engineering (Suzhou), Division of Life Sciences and Medicine, University of Science and Technology of China, Hefei 230026, China
| | - Guohua Shi
- School of Biomedical Engineering (Suzhou), Division of Life Sciences and Medicine, University of Science and Technology of China, Hefei 230026, China
- Center for Excellence in Brain Science and Intelligence Technology, Chinese Academy of Science, Shanghai 200031, China
| |
Collapse
|
21
|
Past, present and future role of retinal imaging in neurodegenerative disease. Prog Retin Eye Res 2021; 83:100938. [PMID: 33460813 PMCID: PMC8280255 DOI: 10.1016/j.preteyeres.2020.100938] [Citation(s) in RCA: 55] [Impact Index Per Article: 18.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/21/2020] [Revised: 12/11/2020] [Accepted: 12/17/2020] [Indexed: 02/08/2023]
Abstract
Retinal imaging technology is rapidly advancing and can provide ever-increasing amounts of information about the structure, function and molecular composition of retinal tissue in humans in vivo. Most importantly, this information can be obtained rapidly, non-invasively and in many cases using Food and Drug Administration-approved devices that are commercially available. Technologies such as optical coherence tomography have dramatically changed our understanding of retinal disease and in many cases have significantly improved their clinical management. Since the retina is an extension of the brain and shares a common embryological origin with the central nervous system, there has also been intense interest in leveraging the expanding armamentarium of retinal imaging technology to understand, diagnose and monitor neurological diseases. This is particularly appealing because of the high spatial resolution, relatively low-cost and wide availability of retinal imaging modalities such as fundus photography or OCT compared to brain imaging modalities such as magnetic resonance imaging or positron emission tomography. The purpose of this article is to review and synthesize current research about retinal imaging in neurodegenerative disease by providing examples from the literature and elaborating on limitations, challenges and future directions. We begin by providing a general background of the most relevant retinal imaging modalities to ensure that the reader has a foundation on which to understand the clinical studies that are subsequently discussed. We then review the application and results of retinal imaging methodologies to several prevalent neurodegenerative diseases where extensive work has been done including sporadic late onset Alzheimer's Disease, Parkinson's Disease and Huntington's Disease. We also discuss Autosomal Dominant Alzheimer's Disease and cerebrovascular small vessel disease, where the application of retinal imaging holds promise but data is currently scarce. Although cerebrovascular disease is not generally considered a neurodegenerative process, it is both a confounder and contributor to neurodegenerative disease processes that requires more attention. Finally, we discuss ongoing efforts to overcome the limitations in the field and unmet clinical and scientific needs.
Collapse
|
22
|
Saad G, Jaber B, Al-hajri M, Househ M, Ahmed A, Abd-alrazaq A. Artificial Intelligence in Diagnosis and Prediction of the Multiple Sclerosis Progression: A Scoping Review (Preprint).. [DOI: 10.2196/preprints.29720] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 09/02/2023]
Abstract
BACKGROUND
Multiple Sclerosis (MS) is an autoimmune disease that results from the demyelination of the nerves in the Central Nervous System. The diagnosis depends on clinical history, neurological examination, and radiological images. Artificial Intelligence proved to be an effective tool in enhancing the diagnostic tools of MS.
OBJECTIVE
To explore how AI assisted in diagnosis and predicting the progression of MS.
METHODS
We used three bibliographic databases in our search: PubMed IEEE Xplore and Cochrane in our search. The study selection process included: removal of duplicated articles, screening titles and abstracts, and reading the full text. This process was performed by two reviewers. The data extracted from the included studies have been filled in an Excel sheet. This step had been done by each reviewer accordingly to the assigned articles. The extracted data sheet was checked by two reviewers to have accuracy ensured. The narrative approach is applied in data synthesis.
RESULTS
The search conducted resulted in 320 articles Removing duplicates and excluding the ineligible articles due to irrelevancy to the population, intervention, and outcomes resulted in excluding 299 articles. Thus, our review will include 21 articles for data extraction and data synthesis.
CONCLUSIONS
Artificial Intelligence is becoming a trend in the medical field. Its contribution in enhancing the diagnostic tools of many diseases, as in MS, is prominent and can be built on in further development plans. However, the implementation of Artificial Intelligence in Multiple Sclerosis is not widespread to confirm the benefits gained, and the datasets involved in the current practice are relatively small. It is recommended to have more studies that focus on the relationship between the employment of AI in diagnosis and monitoring progression and the accuracy gained by this employment.
Collapse
|
23
|
Li J, Jin P, Zhu J, Zou H, Xu X, Tang M, Zhou M, Gan Y, He J, Ling Y, Su Y. Multi-scale GCN-assisted two-stage network for joint segmentation of retinal layers and discs in peripapillary OCT images. BIOMEDICAL OPTICS EXPRESS 2021; 12:2204-2220. [PMID: 33996224 PMCID: PMC8086482 DOI: 10.1364/boe.417212] [Citation(s) in RCA: 22] [Impact Index Per Article: 7.3] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Received: 12/11/2020] [Revised: 03/15/2021] [Accepted: 03/16/2021] [Indexed: 05/03/2023]
Abstract
An accurate and automated tissue segmentation algorithm for retinal optical coherence tomography (OCT) images is crucial for the diagnosis of glaucoma. However, due to the presence of the optic disc, the anatomical structure of the peripapillary region of the retina is complicated and is challenging for segmentation. To address this issue, we develop a novel graph convolutional network (GCN)-assisted two-stage framework to simultaneously label the nine retinal layers and the optic disc. Specifically, a multi-scale global reasoning module is inserted between the encoder and decoder of a U-shape neural network to exploit anatomical prior knowledge and perform spatial reasoning. We conduct experiments on human peripapillary retinal OCT images. We also provide public access to the collected dataset, which might contribute to the research in the field of biomedical image processing. The Dice score of the proposed segmentation network is 0.820 ± 0.001 and the pixel accuracy is 0.830 ± 0.002, both of which outperform those from other state-of-the-art techniques.
Collapse
Affiliation(s)
- Jiaxuan Li
- John Hopcroft Center for Computer Science, Shanghai Jiao Tong University, Shanghai 200240, China
| | - Peiyao Jin
- Department of Preventative Ophthalmology, Shanghai Eye Disease Prevention and Treatment Center, Shanghai Eye Hospital, Shanghai 200040, China
- Department of Ophthalmology, Shanghai General Hospital, Shanghai Jiao Tong University, Shanghai 200080, China
| | - Jianfeng Zhu
- Department of Preventative Ophthalmology, Shanghai Eye Disease Prevention and Treatment Center, Shanghai Eye Hospital, Shanghai 200040, China
| | - Haidong Zou
- Department of Preventative Ophthalmology, Shanghai Eye Disease Prevention and Treatment Center, Shanghai Eye Hospital, Shanghai 200040, China
- Department of Ophthalmology, Shanghai General Hospital, Shanghai Jiao Tong University, Shanghai 200080, China
| | - Xun Xu
- Department of Preventative Ophthalmology, Shanghai Eye Disease Prevention and Treatment Center, Shanghai Eye Hospital, Shanghai 200040, China
- Department of Ophthalmology, Shanghai General Hospital, Shanghai Jiao Tong University, Shanghai 200080, China
| | - Min Tang
- Department of Ophthalmology, Shanghai General Hospital, Shanghai Jiao Tong University, Shanghai 200080, China
| | - Minwen Zhou
- Department of Ophthalmology, Shanghai General Hospital, Shanghai Jiao Tong University, Shanghai 200080, China
| | - Yu Gan
- Department of Electrical and Computer Engineering, The University of Alabama, AL 35487, USA
| | - Jiangnan He
- Department of Preventative Ophthalmology, Shanghai Eye Disease Prevention and Treatment Center, Shanghai Eye Hospital, Shanghai 200040, China
| | - Yuye Ling
- John Hopcroft Center for Computer Science, Shanghai Jiao Tong University, Shanghai 200240, China
| | - Yikai Su
- State Key Lab of Advanced Optical Communication Systems and Networks, Department of Electronic Engineering, Shanghai Jiao Tong University, Shanghai 200240, China
| |
Collapse
|
24
|
Vázquez-Marrufo M, Sarrias-Arrabal E, García-Torres M, Martín-Clemente R, Izquierdo G. A systematic review of the application of machine-learning algorithms in multiple sclerosis. Neurologia 2021; 38:S0213-4853(20)30431-X. [PMID: 33549371 DOI: 10.1016/j.nrl.2020.10.017] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/05/2020] [Revised: 08/20/2020] [Accepted: 10/11/2020] [Indexed: 11/24/2022] Open
Abstract
INTRODUCTION The applications of artificial intelligence, and in particular automatic learning or "machine learning" (ML), constitute both a challenge and a great opportunity in numerous scientific, technical, and clinical disciplines. Specific applications in the study of multiple sclerosis (MS) have been no exception, and constitute an area of increasing interest in recent years. OBJECTIVE We present a systematic review of the application of ML algorithms in MS. MATERIALS AND METHODS We used the PubMed search engine, which allows free access to the MEDLINE medical database, to identify studies including the keywords "machine learning" and "multiple sclerosis." We excluded review articles, studies written in languages other than English or Spanish, and studies that were mainly technical and did not specifically apply to MS. The final selection included 76 articles, and 38 were rejected. CONCLUSIONS After the review process, we established 4 main applications of ML in MS: 1) classifying MS subtypes; 2) distinguishing patients with MS from healthy controls and individuals with other diseases; 3) predicting progression and response to therapeutic interventions; and 4) other applications. Results found to date have shown that ML algorithms may offer great support for health professionals both in clinical settings and in research into MS.
Collapse
Affiliation(s)
- M Vázquez-Marrufo
- Departamento de Psicología Experimental, Facultad de Psicología, Universidad de Sevilla, Sevilla, España.
| | - E Sarrias-Arrabal
- Departamento de Psicología Experimental, Facultad de Psicología, Universidad de Sevilla, Sevilla, España
| | - M García-Torres
- Escuela Politécnica Superior, Universidad Pablo de Olavide, Sevilla, España
| | - R Martín-Clemente
- Departamento de Teoría de la Señal y Comunicaciones, Escuela Técnica Superior de Ingeniería, Universidad de Sevilla, Sevilla, España
| | - G Izquierdo
- Unidad de Esclerosis Múltiple, Hospital VITHAS, Sevilla, España
| |
Collapse
|
25
|
He Y, Carass A, Liu Y, Jedynak BM, Solomon SD, Saidha S, Calabresi PA, Prince JL. Structured layer surface segmentation for retina OCT using fully convolutional regression networks. Med Image Anal 2021; 68:101856. [PMID: 33260113 PMCID: PMC7855873 DOI: 10.1016/j.media.2020.101856] [Citation(s) in RCA: 33] [Impact Index Per Article: 11.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/29/2020] [Revised: 08/27/2020] [Accepted: 09/14/2020] [Indexed: 12/18/2022]
Abstract
Optical coherence tomography (OCT) is a noninvasive imaging modality with micrometer resolution which has been widely used for scanning the retina. Retinal layers are important biomarkers for many diseases. Accurate automated algorithms for segmenting smooth continuous layer surfaces with correct hierarchy (topology) are important for automated retinal thickness and surface shape analysis. State-of-the-art methods typically use a two step process. Firstly, a trained classifier is used to label each pixel into either background and layers or boundaries and non-boundaries. Secondly, the desired smooth surfaces with the correct topology are extracted by graph methods (e.g., graph cut). Data driven methods like deep networks have shown great ability for the pixel classification step, but to date have not been able to extract structured smooth continuous surfaces with topological constraints in the second step. In this paper, we combine these two steps into a unified deep learning framework by directly modeling the distribution of the surface positions. Smooth, continuous, and topologically correct surfaces are obtained in a single feed forward operation. The proposed method was evaluated on two publicly available data sets of healthy controls and subjects with either multiple sclerosis or diabetic macular edema, and is shown to achieve state-of-the art performance with sub-pixel accuracy.
Collapse
Affiliation(s)
- Yufan He
- Department of Electrical and Computer Engineering, The Johns Hopkins University, Baltimore, MD 21218, USA.
| | - Aaron Carass
- Department of Electrical and Computer Engineering, The Johns Hopkins University, Baltimore, MD 21218, USA; Department of Computer Science, The Johns Hopkins University, Baltimore, MD 21218, USA
| | - Yihao Liu
- Department of Electrical and Computer Engineering, The Johns Hopkins University, Baltimore, MD 21218, USA
| | - Bruno M Jedynak
- Department of Mathematics & Statistics, Portland State University, Portland, OR 97201, USA
| | - Sharon D Solomon
- Wilmer Eye Institute, The Johns Hopkins University School of Medicine, MD 21287, USA
| | - Shiv Saidha
- Department of Neurology, The Johns Hopkins University School of Medicine, MD 21287, USA
| | - Peter A Calabresi
- Department of Neurology, The Johns Hopkins University School of Medicine, MD 21287, USA
| | - Jerry L Prince
- Department of Electrical and Computer Engineering, The Johns Hopkins University, Baltimore, MD 21218, USA; Department of Computer Science, The Johns Hopkins University, Baltimore, MD 21218, USA
| |
Collapse
|
26
|
Kugelman J, Alonso-Caneiro D, Chen Y, Arunachalam S, Huang D, Vallis N, Collins MJ, Chen FK. Retinal Boundary Segmentation in Stargardt Disease Optical Coherence Tomography Images Using Automated Deep Learning. Transl Vis Sci Technol 2020; 9:12. [PMID: 33133774 PMCID: PMC7581491 DOI: 10.1167/tvst.9.11.12] [Citation(s) in RCA: 16] [Impact Index Per Article: 4.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/26/2020] [Accepted: 09/11/2020] [Indexed: 12/13/2022] Open
Abstract
Purpose To use a deep learning model to develop a fully automated method (fully semantic network and graph search [FS-GS]) of retinal segmentation for optical coherence tomography (OCT) images from patients with Stargardt disease. Methods Eighty-seven manually segmented (ground truth) OCT volume scan sets (5171 B-scans) from 22 patients with Stargardt disease were used for training, validation and testing of a novel retinal boundary detection approach (FS-GS) that combines a fully semantic deep learning segmentation method, which generates a per-pixel class prediction map with a graph-search method to extract retinal boundary positions. The performance was evaluated using the mean absolute boundary error and the differences in two clinical metrics (retinal thickness and volume) compared with the ground truth. The performance of a separate deep learning method and two publicly available software algorithms were also evaluated against the ground truth. Results FS-GS showed an excellent agreement with the ground truth, with a boundary mean absolute error of 0.23 and 1.12 pixels for the internal limiting membrane and the base of retinal pigment epithelium or Bruch's membrane, respectively. The mean difference in thickness and volume across the central 6 mm zone were 2.10 µm and 0.059 mm3. The performance of the proposed method was more accurate and consistent than the publicly available OCTExplorer and AURA tools. Conclusions The FS-GS method delivers good performance in segmentation of OCT images of pathologic retina in Stargardt disease. Translational Relevance Deep learning models can provide a robust method for retinal segmentation and support a high-throughput analysis pipeline for measuring retinal thickness and volume in Stargardt disease.
Collapse
Affiliation(s)
- Jason Kugelman
- Queensland University of Technology (QUT), Contact Lens and Visual Optics Laboratory, Centre for Vision and Eye Research, School of Optometry and Vision Science, Queensland, Australia
| | - David Alonso-Caneiro
- Queensland University of Technology (QUT), Contact Lens and Visual Optics Laboratory, Centre for Vision and Eye Research, School of Optometry and Vision Science, Queensland, Australia.,Centre for Ophthalmology and Visual Science (incorporating Lions Eye Institute), The University of Western Australia, Perth, Western Australia, Australia
| | - Yi Chen
- Centre for Ophthalmology and Visual Science (incorporating Lions Eye Institute), The University of Western Australia, Perth, Western Australia, Australia
| | - Sukanya Arunachalam
- Centre for Ophthalmology and Visual Science (incorporating Lions Eye Institute), The University of Western Australia, Perth, Western Australia, Australia
| | - Di Huang
- Centre for Ophthalmology and Visual Science (incorporating Lions Eye Institute), The University of Western Australia, Perth, Western Australia, Australia.,Centre for Molecular Medicine and Innovative Therapeutics, Murdoch University, Murdoch, Western Australia, Australia.,Centre for Neuromuscular and Neurological Disorders, The University of Western Australia and Perron Institute for Neurological and Translational Science, Nedlands, Western Australia, Australia
| | - Natasha Vallis
- Centre for Ophthalmology and Visual Science (incorporating Lions Eye Institute), The University of Western Australia, Perth, Western Australia, Australia
| | - Michael J Collins
- Queensland University of Technology (QUT), Contact Lens and Visual Optics Laboratory, Centre for Vision and Eye Research, School of Optometry and Vision Science, Queensland, Australia
| | - Fred K Chen
- Centre for Ophthalmology and Visual Science (incorporating Lions Eye Institute), The University of Western Australia, Perth, Western Australia, Australia.,Department of Ophthalmology, Royal Perth Hospital, Perth, Western Australia, Australia.,Department of Ophthalmology, Perth Children's Hospital, Nedlands, Western Australia, Australia
| |
Collapse
|
27
|
Tan B, Sim R, Chua J, Wong DWK, Yao X, Garhöfer G, Schmidl D, Werkmeister RM, Schmetterer L. Approaches to quantify optical coherence tomography angiography metrics. ANNALS OF TRANSLATIONAL MEDICINE 2020; 8:1205. [PMID: 33241054 PMCID: PMC7576021 DOI: 10.21037/atm-20-3246] [Citation(s) in RCA: 21] [Impact Index Per Article: 5.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Indexed: 12/13/2022]
Abstract
Optical coherence tomography (OCT) has revolutionized the field of ophthalmology in the last three decades. As an OCT extension, OCT angiography (OCTA) utilizes a fast OCT system to detect motion contrast in ocular tissue and provides a three-dimensional representation of the ocular vasculature in a non-invasive, dye-free manner. The first OCT machine equipped with OCTA function was approved by U.S. Food and Drug Administration in 2016 and now it is widely applied in clinics. To date, numerous methods have been developed to aid OCTA interpretation and quantification. In this review, we focused on the workflow of OCTA-based interpretation, beginning from the generation of the OCTA images using signal decorrelation, which we divided into intensity-based, phase-based and phasor-based methods. We further discussed methods used to address image artifacts that are commonly observed in clinical settings, to the algorithms for image enhancement, binarization, and OCTA metrics extraction. We believe a better grasp of these technical aspects of OCTA will enhance the understanding of the technology and its potential application in disease diagnosis and management. Moreover, future studies will also explore the use of ocular OCTA as a window to link ocular vasculature to the function of other organs such as the kidney and brain.
Collapse
Affiliation(s)
- Bingyao Tan
- Institute for Health Technologies, Nanyang Technological University, Singapore, Singapore.,Singapore Eye Research Institute, Singapore National Eye Centre, Singapore, Singapore.,SERI-NTU Advanced Ocular Engineering (STANCE) Program, Nanyang Technological University, Singapore, Singapore
| | - Ralene Sim
- Singapore Eye Research Institute, Singapore National Eye Centre, Singapore, Singapore
| | - Jacqueline Chua
- Singapore Eye Research Institute, Singapore National Eye Centre, Singapore, Singapore.,Academic Clinical Program, Duke-NUS Medical School, Singapore, Singapore
| | - Damon W K Wong
- Institute for Health Technologies, Nanyang Technological University, Singapore, Singapore.,Singapore Eye Research Institute, Singapore National Eye Centre, Singapore, Singapore.,SERI-NTU Advanced Ocular Engineering (STANCE) Program, Nanyang Technological University, Singapore, Singapore
| | - Xinwen Yao
- Institute for Health Technologies, Nanyang Technological University, Singapore, Singapore.,Singapore Eye Research Institute, Singapore National Eye Centre, Singapore, Singapore.,SERI-NTU Advanced Ocular Engineering (STANCE) Program, Nanyang Technological University, Singapore, Singapore
| | - Gerhard Garhöfer
- Department of Clinical Pharmacology, Medical University of Vienna, Vienna, Austria
| | - Doreen Schmidl
- Department of Clinical Pharmacology, Medical University of Vienna, Vienna, Austria
| | - René M Werkmeister
- Center for Medical Physics and Biomedical Engineering, Medical University of Vienna, Vienna, Austria
| | - Leopold Schmetterer
- Singapore Eye Research Institute, Singapore National Eye Centre, Singapore, Singapore.,SERI-NTU Advanced Ocular Engineering (STANCE) Program, Nanyang Technological University, Singapore, Singapore.,Academic Clinical Program, Duke-NUS Medical School, Singapore, Singapore.,Department of Clinical Pharmacology, Medical University of Vienna, Vienna, Austria.,Center for Medical Physics and Biomedical Engineering, Medical University of Vienna, Vienna, Austria.,Yong Loo Lin School of Medicine, National University of Singapore and National University Health System, Singapore, Singapore.,Department of Ophthalmology, Lee Kong Chian School of Medicine, Nanyang Technological University, Singapore, Singapore.,Institute of Molecular and Clinical Ophthalmology, Basel, Switzerland
| |
Collapse
|
28
|
Heisler M, Bhalla M, Lo J, Mammo Z, Lee S, Ju MJ, Beg MF, Sarunic MV. Semi-supervised deep learning based 3D analysis of the peripapillary region. BIOMEDICAL OPTICS EXPRESS 2020; 11:3843-3856. [PMID: 33014570 PMCID: PMC7510893 DOI: 10.1364/boe.392648] [Citation(s) in RCA: 11] [Impact Index Per Article: 2.8] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Received: 03/18/2020] [Revised: 04/30/2020] [Accepted: 05/01/2020] [Indexed: 05/08/2023]
Abstract
Optical coherence tomography (OCT) has become an essential tool in the evaluation of glaucoma, typically through analyzing retinal nerve fiber layer changes in circumpapillary scans. Three-dimensional OCT volumes enable a much more thorough analysis of the optic nerve head (ONH) region, which may be the site of initial glaucomatous optic nerve damage. Automated analysis of this region is of great interest, though large anatomical variations and the termination of layers make the requisite peripapillary layer and Bruch's membrane opening (BMO) segmentation a challenging task. Several machine learning-based segmentation methods have been proposed for retinal layer segmentation, and a few for the ONH region, but they typically depend on either heavily averaged or pre-processed B-scans or a large amount of annotated data, which is a tedious task and resource-intensive. We evaluated a semi-supervised adversarial deep learning method for segmenting peripapillary retinal layers in OCT B-scans to take advantage of unlabeled data. We show that the use of a generative adversarial network and unlabeled data can improve the performance of segmentation. Additionally, we use a Faster R-CNN architecture to automatically segment the BMO. The proposed methods are then used for the 3D morphometric analysis of both control and glaucomatous ONH volumes to demonstrate the potential for clinical utility.
Collapse
Affiliation(s)
- Morgan Heisler
- Simon Fraser University, Department of
Engineering Science, 8888 University Drive, Burnaby, BC, V5A 1S6,
Canada
| | - Mahadev Bhalla
- University of British Columbia, Faculty of
Medicine, 317-2194 Health Sciences Mall, Vancouver, BC, V6 T 1Z3,
Canada
| | - Julian Lo
- Simon Fraser University, Department of
Engineering Science, 8888 University Drive, Burnaby, BC, V5A 1S6,
Canada
| | - Zaid Mammo
- University of British Columbia, Department
of Ophthalmology and Vision Science, 2550 Willow Street, Vancouver,
BC, V5Z 3N9, Canada
| | - Sieun Lee
- Simon Fraser University, Department of
Engineering Science, 8888 University Drive, Burnaby, BC, V5A 1S6,
Canada
| | - Myeong Jin Ju
- University of British Columbia, Department
of Ophthalmology and Vision Science, 2550 Willow Street, Vancouver,
BC, V5Z 3N9, Canada
- University of British Columbia, School of
Biomedical Engineering, 251-2222 Health Sciences Mall, Vancouver, BC,
V6 T 1Z3, Canada
| | - Mirza Faisal Beg
- Simon Fraser University, Department of
Engineering Science, 8888 University Drive, Burnaby, BC, V5A 1S6,
Canada
| | - Marinko V. Sarunic
- Simon Fraser University, Department of
Engineering Science, 8888 University Drive, Burnaby, BC, V5A 1S6,
Canada
| |
Collapse
|
29
|
Wang C, Gan M, Zhang M, Li D. Adversarial convolutional network for esophageal tissue segmentation on OCT images. BIOMEDICAL OPTICS EXPRESS 2020; 11:3095-3110. [PMID: 32637244 PMCID: PMC7316031 DOI: 10.1364/boe.394715] [Citation(s) in RCA: 8] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Received: 04/08/2020] [Revised: 05/08/2020] [Accepted: 05/08/2020] [Indexed: 05/20/2023]
Abstract
Automatic segmentation is important for esophageal OCT image processing, which is able to provide tissue characteristics such as shape and thickness for disease diagnosis. Existing automatical segmentation methods based on deep convolutional networks may not generate accurate segmentation results due to limited training set and various layer shapes. This study proposed a novel adversarial convolutional network (ACN) to segment esophageal OCT images using a convolutional network trained by adversarial learning. The proposed framework includes a generator and a discriminator, both with U-Net alike fully convolutional architecture. The discriminator is a hybrid network that discriminates whether the generated results are real and implements pixel classification at the same time. Leveraging on the adversarial training, the discriminator becomes more powerful. In addition, the adversarial loss is able to encode high order relationships of pixels, thus eliminating the requirements of post-processing. Experiments on segmenting esophageal OCT images from guinea pigs confirmed that the ACN outperforms several deep learning frameworks in pixel classification accuracy and improves the segmentation result. The potential clinical application of ACN for detecting eosinophilic esophagitis (EoE), an esophageal disease, is also presented in the experiment.
Collapse
Affiliation(s)
- Cong Wang
- Jiangsu Key Laboratory of Medical Optics, Suzhou Institute of Biomedical Engineering and Technology, Chinese Academy of Sciences, Suzhou 215163, China
- These authors contributed equally to this work and should be considered co-first authors
| | - Meng Gan
- Jiangsu Key Laboratory of Medical Optics, Suzhou Institute of Biomedical Engineering and Technology, Chinese Academy of Sciences, Suzhou 215163, China
- These authors contributed equally to this work and should be considered co-first authors
| | - Miao Zhang
- School of Electronic and Information Engineering, Soochow University, Suzhou 215006, China
| | - Deyin Li
- School of Electronic and Information Engineering, Soochow University, Suzhou 215006, China
| |
Collapse
|
30
|
Wang C, Gan M, Zhang M, Li D. Adversarial convolutional network for esophageal tissue segmentation on OCT images. BIOMEDICAL OPTICS EXPRESS 2020; 11:3095-3110. [PMID: 32637244 DOI: 10.1109/access.2020.3041767] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Received: 04/08/2020] [Revised: 05/08/2020] [Accepted: 05/08/2020] [Indexed: 05/26/2023]
Abstract
Automatic segmentation is important for esophageal OCT image processing, which is able to provide tissue characteristics such as shape and thickness for disease diagnosis. Existing automatical segmentation methods based on deep convolutional networks may not generate accurate segmentation results due to limited training set and various layer shapes. This study proposed a novel adversarial convolutional network (ACN) to segment esophageal OCT images using a convolutional network trained by adversarial learning. The proposed framework includes a generator and a discriminator, both with U-Net alike fully convolutional architecture. The discriminator is a hybrid network that discriminates whether the generated results are real and implements pixel classification at the same time. Leveraging on the adversarial training, the discriminator becomes more powerful. In addition, the adversarial loss is able to encode high order relationships of pixels, thus eliminating the requirements of post-processing. Experiments on segmenting esophageal OCT images from guinea pigs confirmed that the ACN outperforms several deep learning frameworks in pixel classification accuracy and improves the segmentation result. The potential clinical application of ACN for detecting eosinophilic esophagitis (EoE), an esophageal disease, is also presented in the experiment.
Collapse
Affiliation(s)
- Cong Wang
- Jiangsu Key Laboratory of Medical Optics, Suzhou Institute of Biomedical Engineering and Technology, Chinese Academy of Sciences, Suzhou 215163, China
- These authors contributed equally to this work and should be considered co-first authors
| | - Meng Gan
- Jiangsu Key Laboratory of Medical Optics, Suzhou Institute of Biomedical Engineering and Technology, Chinese Academy of Sciences, Suzhou 215163, China
- These authors contributed equally to this work and should be considered co-first authors
| | - Miao Zhang
- School of Electronic and Information Engineering, Soochow University, Suzhou 215006, China
| | - Deyin Li
- School of Electronic and Information Engineering, Soochow University, Suzhou 215006, China
| |
Collapse
|
31
|
Carass A, Roy S, Gherman A, Reinhold JC, Jesson A, Arbel T, Maier O, Handels H, Ghafoorian M, Platel B, Birenbaum A, Greenspan H, Pham DL, Crainiceanu CM, Calabresi PA, Prince JL, Roncal WRG, Shinohara RT, Oguz I. Evaluating White Matter Lesion Segmentations with Refined Sørensen-Dice Analysis. Sci Rep 2020; 10:8242. [PMID: 32427874 PMCID: PMC7237671 DOI: 10.1038/s41598-020-64803-w] [Citation(s) in RCA: 56] [Impact Index Per Article: 14.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/03/2019] [Accepted: 04/20/2020] [Indexed: 11/09/2022] Open
Abstract
The Sørensen-Dice index (SDI) is a widely used measure for evaluating medical image segmentation algorithms. It offers a standardized measure of segmentation accuracy which has proven useful. However, it offers diminishing insight when the number of objects is unknown, such as in white matter lesion segmentation of multiple sclerosis (MS) patients. We present a refinement for finer grained parsing of SDI results in situations where the number of objects is unknown. We explore these ideas with two case studies showing what can be learned from our two presented studies. Our first study explores an inter-rater comparison, showing that smaller lesions cannot be reliably identified. In our second case study, we demonstrate fusing multiple MS lesion segmentation algorithms based on the insights into the algorithms provided by our analysis to generate a segmentation that exhibits improved performance. This work demonstrates the wealth of information that can be learned from refined analysis of medical image segmentations.
Collapse
Affiliation(s)
- Aaron Carass
- Department of Electrical and Computer Engineering, The Johns Hopkins University, Baltimore, MD, 21218, USA.
- Department of Computer Science, The Johns Hopkins University, Baltimore, MD, 21218, USA.
| | - Snehashis Roy
- CNRM, The Henry M. Jackson Foundation for the Advancement of Military Medicine, Bethesda, MD, 20817, USA
| | - Adrian Gherman
- Department of Biostatistics, The Johns Hopkins University, Baltimore, MD, 21205, USA
| | - Jacob C Reinhold
- Department of Electrical and Computer Engineering, The Johns Hopkins University, Baltimore, MD, 21218, USA
| | - Andrew Jesson
- Centre For Intelligent Machines, McGill University, Montréal, QC, H3A 0E9, Canada
| | - Tal Arbel
- Centre For Intelligent Machines, McGill University, Montréal, QC, H3A 0E9, Canada
| | - Oskar Maier
- Institute of Medical Informatics, University of Lübeck, 23538, Lübeck, Germany
| | - Heinz Handels
- Institute of Medical Informatics, University of Lübeck, 23538, Lübeck, Germany
| | - Mohsen Ghafoorian
- Institute for Computing and Information Sciences, Radboud University, 6525, HP, Nijmegen, Netherlands
| | - Bram Platel
- Diagnostic Image Analysis Group, Radboud University Medical Center, 6525, GA, Nijmegen, Netherlands
| | - Ariel Birenbaum
- Department of Electrical Engineering, Tel-Aviv University, Tel-Aviv, 69978, Israel
| | - Hayit Greenspan
- Department of Biomedical Engineering, Tel-Aviv University, Tel-Aviv, 69978, Israel
| | - Dzung L Pham
- CNRM, The Henry M. Jackson Foundation for the Advancement of Military Medicine, Bethesda, MD, 20817, USA
| | - Ciprian M Crainiceanu
- Department of Biostatistics, The Johns Hopkins University, Baltimore, MD, 21205, USA
| | - Peter A Calabresi
- Department of Neurology, The Johns Hopkins University School of Medicine, Baltimore, MD, 21287, USA
| | - Jerry L Prince
- Department of Electrical and Computer Engineering, The Johns Hopkins University, Baltimore, MD, 21218, USA
- Department of Computer Science, The Johns Hopkins University, Baltimore, MD, 21218, USA
| | - William R Gray Roncal
- Department of Computer Science, The Johns Hopkins University, Baltimore, MD, 21218, USA
| | - Russell T Shinohara
- Penn Statistics in Imaging and Visualization Center, Department of Biostatistics & Epidemiology, University of Pennsylvania, Philadelphia, PA, 19104, USA
- Center for Biomedical Image Computing and Analytics, Department of Radiology, University of Pennsylvania, Philadelphia, PA, 19104, USA
| | - Ipek Oguz
- Department of Electrical Engineering and Computer Science, Vanderbilt University, Nashville, TN, 37203, USA
| |
Collapse
|