1
|
Chen Z, Zhang H, Linton EF, Johnson BA, Choi YJ, Kupersmith MJ, Sonka M, Garvin MK, Kardon RH, Wang JK. Hybrid deep learning and optimal graph search method for optical coherence tomography layer segmentation in diseases affecting the optic nerve. BIOMEDICAL OPTICS EXPRESS 2024; 15:3681-3698. [PMID: 38867777 PMCID: PMC11166436 DOI: 10.1364/boe.516045] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 12/14/2023] [Revised: 03/09/2024] [Accepted: 05/02/2024] [Indexed: 06/14/2024]
Abstract
Accurate segmentation of retinal layers in optical coherence tomography (OCT) images is critical for assessing diseases that affect the optic nerve, but existing automated algorithms often fail when pathology causes irregular layer topology, such as extreme thinning of the ganglion cell-inner plexiform layer (GCIPL). Deep LOGISMOS, a hybrid approach that combines the strengths of deep learning and 3D graph search to overcome their limitations, was developed to improve the accuracy, robustness and generalizability of retinal layer segmentation. The method was trained on 124 OCT volumes from both eyes of 31 non-arteritic anterior ischemic optic neuropathy (NAION) patients and tested on three cross-sectional datasets with available reference tracings: Test-NAION (40 volumes from both eyes of 20 NAION subjects), Test-G (29 volumes from 29 glaucoma subjects/eyes), and Test-JHU (35 volumes from 21 multiple sclerosis and 14 control subjects/eyes) and one longitudinal dataset without reference tracings: Test-G-L (155 volumes from 15 glaucoma patients/eyes). In the three test datasets with reference tracings (Test-NAION, Test-G, and Test-JHU), Deep LOGISMOS achieved very high Dice similarity coefficients (%) on GCIPL: 89.97±3.59, 90.63±2.56, and 94.06±1.76, respectively. In the same context, Deep LOGISMOS outperformed the Iowa reference algorithms by improving the Dice score by 17.5, 5.4, and 7.5, and also surpassed the deep learning framework nnU-Net with improvements of 4.4, 3.7, and 1.0. For the 15 severe glaucoma eyes with marked GCIPL thinning (Test-G-L), it demonstrated reliable regional GCIPL thickness measurement over five years. The proposed Deep LOGISMOS approach has potential to enhance precise quantification of retinal structures, aiding diagnosis and treatment management of optic nerve diseases.
Collapse
Affiliation(s)
- Zhi Chen
- Iowa Institute for Biomedical Imaging, University of Iowa, Iowa City, IA 52242, USA
- Department of Electrical and Computer
Engineering, University of Iowa, Iowa City, IA 52242, USA
| | - Honghai Zhang
- Iowa Institute for Biomedical Imaging, University of Iowa, Iowa City, IA 52242, USA
- Department of Electrical and Computer
Engineering, University of Iowa, Iowa City, IA 52242, USA
| | - Edward F. Linton
- Department of Ophthalmology and Visual
Sciences, University of Iowa, Iowa City, IA 52242, USA
| | - Brett A. Johnson
- Department of Ophthalmology and Visual
Sciences, University of Iowa, Iowa City, IA 52242, USA
| | - Yun Jae Choi
- Department of Ophthalmology and Visual
Sciences, University of Iowa, Iowa City, IA 52242, USA
- Department of Biomedical Engineering, University of Iowa, Iowa City, IA 52242, USA
| | - Mark J. Kupersmith
- Departments of Neurology, Ophthalmology and
Neurosurgery, Icahn School of Medicine at Mount
Sinai, New York, NY 10029, USA
| | - Milan Sonka
- Iowa Institute for Biomedical Imaging, University of Iowa, Iowa City, IA 52242, USA
- Department of Electrical and Computer
Engineering, University of Iowa, Iowa City, IA 52242, USA
| | - Mona K. Garvin
- Iowa Institute for Biomedical Imaging, University of Iowa, Iowa City, IA 52242, USA
- Department of Electrical and Computer
Engineering, University of Iowa, Iowa City, IA 52242, USA
- Center for the Prevention and
Treatment of Visual Loss, Iowa City VA Health Care
System, Iowa City, IA 52242, USA
| | - Randy H. Kardon
- Department of Ophthalmology and Visual
Sciences, University of Iowa, Iowa City, IA 52242, USA
- Center for the Prevention and
Treatment of Visual Loss, Iowa City VA Health Care
System, Iowa City, IA 52242, USA
| | - Jui-Kai Wang
- Department of Electrical and Computer
Engineering, University of Iowa, Iowa City, IA 52242, USA
- Department of Ophthalmology and Visual
Sciences, University of Iowa, Iowa City, IA 52242, USA
- Center for the Prevention and
Treatment of Visual Loss, Iowa City VA Health Care
System, Iowa City, IA 52242, USA
| |
Collapse
|
2
|
Tanthanathewin R, Wongrattanapipat W, Khaing TT, Aimmanee P. Automatic exudate and aneurysm segmentation in OCT images using UNET++ and hyperreflective-foci feature based bagged tree ensemble. PLoS One 2024; 19:e0304146. [PMID: 38787844 PMCID: PMC11125471 DOI: 10.1371/journal.pone.0304146] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/28/2022] [Accepted: 05/07/2024] [Indexed: 05/26/2024] Open
Abstract
Diabetic retinopathy's signs, such as exudates (EXs) and aneurysms (ANs), initially develop from under the retinal surface detectable from optical coherence tomography (OCT) images. Detecting these signs helps ophthalmologists diagnose DR sooner. Detecting and segmenting exudates (EXs) and aneurysms (ANs) in medical images is challenging due to their small size, similarity to other hyperreflective regions, noise presence, and low background contrast. Furthermore, the scarcity of public OCT images featuring these abnormalities has limited the number of studies related to the automatic segmentation of EXs and ANs, and the reported performance of such studies has not been satisfactory. This work proposes an efficient algorithm that can automatically segment these anomalies by improving key steps in the process. The potential area where these hyper-reflective EXs and ANs occur was scoped by our method using a deep-learning U-Net++ program. From this area, the candidates for EX-AN were segmented using the adaptive thresholding method. Nine features based on appearances, locations, and shadow markers were extracted from these candidates. They were trained and tested using bagged tree ensemble classifiers to obtain only EX-AN blobs. The proposed method was tested on a collection of a public dataset comprising 80 images with hand-drawn ground truths. The experimental results showed that our method could segment EX-AN blobs with average recall, precision, and F1-measure as 87.9%, 86.1%, and 87.0%, respectively. Its F1-measure drastically outperformed two comparative methods, binary thresholding and watershed (BT-WS) and adaptive thresholding with shadow tracking (AT-ST), by 78.0% and 82.1%, respectively.
Collapse
Affiliation(s)
- Rinrada Tanthanathewin
- School of Information, Computer and Communication Technology, Sirindhorn International Institute of Technology, Thammasat University, Meung, Patumthani, Thailand
| | - Warissaporn Wongrattanapipat
- School of Information, Computer and Communication Technology, Sirindhorn International Institute of Technology, Thammasat University, Meung, Patumthani, Thailand
| | - Tin Tin Khaing
- School of Information, Computer and Communication Technology, Sirindhorn International Institute of Technology, Thammasat University, Meung, Patumthani, Thailand
| | - Pakinee Aimmanee
- School of Information, Computer and Communication Technology, Sirindhorn International Institute of Technology, Thammasat University, Meung, Patumthani, Thailand
| |
Collapse
|
3
|
Riazi-Esfahani H, Jafari B, Azimi H, Rahimi M, Saeidian J, Pouya P, Faghihi H, Mirzaei A, Asadi Khameneh E, Khalili Pour E. Assessment of area and structural irregularity of retinal layers in diabetic retinopathy using machine learning and image processing techniques. Sci Rep 2024; 14:4013. [PMID: 38369610 PMCID: PMC10874958 DOI: 10.1038/s41598-024-54535-6] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/26/2023] [Accepted: 02/13/2024] [Indexed: 02/20/2024] Open
Abstract
Diabetes retinopathy prevention necessitates early detection, monitoring, and treatment. Non-invasive optical coherence tomography (OCT) shows structural changes in the retinal layer. OCT image evaluation necessitates retinal layer segmentation. The ability of our automated retinal layer segmentation to distinguish between normal, non-proliferative (NPDR), and proliferative diabetic retinopathy (PDR) was investigated in this study using quantifiable biomarkers such as retina layer smoothness index (SI) and area (S) in horizontal and vertical OCT images for each zone (fovea, superior, inferior, nasal, and temporal). This research includes 84 eyes from 57 individuals. The study shows a significant difference in the Area (S) of inner nuclear layer (INL) and outer nuclear layer (ONL) in the horizontal foveal zone across the three groups (p < 0.001). In the horizontal scan, there is a significant difference in the smoothness index (SI) of the inner plexiform layer (IPL) and the upper border of the outer plexiform layer (OPL) among three groups (p < 0.05). There is also a significant difference in the area (S) of the OPL in the foveal zone among the three groups (p = 0.003). The area (S) of the INL in the foveal region of horizontal slabs performed best for distinguishing diabetic patients (NPDR and PDR) from normal individuals, with an accuracy of 87.6%. The smoothness index (SI) of IPL in the nasal zone of horizontal foveal slabs was the most accurate at 97.2% in distinguishing PDR from NPDR. The smoothness index of the top border of the OPL in the nasal zone of horizontal slabs was 84.1% accurate in distinguishing NPDR from PDR. Smoothness index of IPL in the temporal zone of horizontal slabs was 89.8% accurate in identifying NPDR from PDR patients. In conclusion, optical coherence tomography can assess the smoothness index and irregularity of the inner and outer plexiform layers, particularly in the nasal and temporal regions of horizontal foveal slabs, to distinguish non-proliferative from proliferative diabetic retinopathy. The evolution of diabetic retinopathy throughout severity levels and its effects on retinal layer irregularity need more study.
Collapse
Affiliation(s)
- Hamid Riazi-Esfahani
- Retina Ward, Farabi Eye Hospital, Tehran University of Medical Sciences, South Kargar Street, Qazvin Square, Tehran, Iran
| | - Behzad Jafari
- Retina Ward, Farabi Eye Hospital, Tehran University of Medical Sciences, South Kargar Street, Qazvin Square, Tehran, Iran
| | - Hossein Azimi
- Faculty of Mathematical Sciences and Computer, Kharazmi University, No. 50, Taleghani Ave, Tehran, Iran
| | - Masoud Rahimi
- Retina Ward, Farabi Eye Hospital, Tehran University of Medical Sciences, South Kargar Street, Qazvin Square, Tehran, Iran
| | - Jamshid Saeidian
- Faculty of Mathematical Sciences and Computer, Kharazmi University, No. 50, Taleghani Ave, Tehran, Iran
| | - Parnia Pouya
- Research Center for Evidence-Based Medicine, Health Management and Safety Promotion Research Institute, Tabriz University of Medical Sciences, Tabriz, Iran
| | - Hooshang Faghihi
- Retina Ward, Farabi Eye Hospital, Tehran University of Medical Sciences, South Kargar Street, Qazvin Square, Tehran, Iran
| | - Arash Mirzaei
- Retina Ward, Farabi Eye Hospital, Tehran University of Medical Sciences, South Kargar Street, Qazvin Square, Tehran, Iran
| | - Esmaeil Asadi Khameneh
- Retina Ward, Farabi Eye Hospital, Tehran University of Medical Sciences, South Kargar Street, Qazvin Square, Tehran, Iran
| | - Elias Khalili Pour
- Retina Ward, Farabi Eye Hospital, Tehran University of Medical Sciences, South Kargar Street, Qazvin Square, Tehran, Iran.
| |
Collapse
|
4
|
Chen C, Fu Z, Ye S, Zhao C, Golovko V, Ye S, Bai Z. Study on high-precision three-dimensional reconstruction of pulmonary lesions and surrounding blood vessels based on CT images. OPTICS EXPRESS 2024; 32:1371-1390. [PMID: 38297691 DOI: 10.1364/oe.510398] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 10/27/2023] [Accepted: 12/15/2023] [Indexed: 02/02/2024]
Abstract
The adoption of computerized tomography (CT) technology has significantly elevated the role of pulmonary CT imaging in diagnosing and treating pulmonary diseases. However, challenges persist due to the complex relationship between lesions within pulmonary tissue and the surrounding blood vessels. These challenges involve achieving precise three-dimensional reconstruction while maintaining accurate relative positioning of these elements. To effectively address this issue, this study employs a semi-automatic precise labeling process for the target region. This procedure ensures a high level of consistency in the relative positions of lesions and the surrounding blood vessels. Additionally, a morphological gradient interpolation algorithm, combined with Gaussian filtering, is applied to facilitate high-precision three-dimensional reconstruction of both lesions and blood vessels. Furthermore, this technique enables post-reconstruction slicing at any layer, facilitating intuitive exploration of the correlation between blood vessels and lesion layers. Moreover, the study utilizes physiological knowledge to simulate real-world blood vessel intersections, determining the range of blood vessel branch angles and achieving seamless continuity at internal blood vessel branch points. The experimental results achieved a satisfactory reconstruction with an average Hausdorff distance of 1.5 mm and an average Dice coefficient of 92%, obtained by comparing the reconstructed shape with the original shape,the approach also achieves a high level of accuracy in three-dimensional reconstruction and visualization. In conclusion, this study is a valuable source of technical support for the diagnosis and treatment of pulmonary diseases and holds promising potential for widespread adoption in clinical practice.
Collapse
|
5
|
Lu J, Cheng Y, Hiya FE, Shen M, Herrera G, Zhang Q, Gregori G, Rosenfeld PJ, Wang RK. Deep-learning-based automated measurement of outer retinal layer thickness for use in the assessment of age-related macular degeneration, applicable to both swept-source and spectral-domain OCT imaging. BIOMEDICAL OPTICS EXPRESS 2024; 15:413-427. [PMID: 38223170 PMCID: PMC10783897 DOI: 10.1364/boe.512359] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 11/16/2023] [Revised: 12/17/2023] [Accepted: 12/17/2023] [Indexed: 01/16/2024]
Abstract
Effective biomarkers are required for assessing the progression of age-related macular degeneration (AMD), a prevalent and progressive eye disease. This paper presents a deep learning-based automated algorithm, applicable to both swept-source OCT (SS-OCT) and spectral-domain OCT (SD-OCT) scans, for measuring outer retinal layer (ORL) thickness as a surrogate biomarker for outer retinal degeneration, e.g., photoreceptor disruption, to assess AMD progression. The algorithm was developed based on a modified TransUNet model with clinically annotated retinal features manifested in the progression of AMD. The algorithm demonstrates a high accuracy with an intersection of union (IoU) of 0.9698 in the testing dataset for segmenting ORL using both SS-OCT and SD-OCT datasets. The robustness and applicability of the algorithm are indicated by strong correlation (r = 0.9551, P < 0.0001 in the central-fovea 3 mm-circle, and r = 0.9442, P < 0.0001 in the 5 mm-circle) and agreement (the mean bias = 0.5440 um in the 3-mm circle, and 1.392 um in the 5-mm circle) of the ORL thickness measurements between SS-OCT and SD-OCT scans. Comparative analysis reveals significant differences (P < 0.0001) in ORL thickness among 80 normal eyes, 30 intermediate AMD eyes with reticular pseudodrusen, 49 intermediate AMD eyes with drusen, and 40 late AMD eyes with geographic atrophy, highlighting its potential as an independent biomarker for predicting AMD progression. The findings provide valuable insights into the ORL alterations associated with different stages of AMD and emphasize the potential of ORL thickness as a sensitive indicator of AMD severity and progression.
Collapse
Affiliation(s)
- Jie Lu
- Department of Bioengineering, University of Washington, Seattle, Washington, USA
| | - Yuxuan Cheng
- Department of Bioengineering, University of Washington, Seattle, Washington, USA
| | - Farhan E. Hiya
- Department of Ophthalmology, Bascom Palmer Eye Institute, University of Miami Miller School of Medicine, Miami, Florida, USA
| | - Mengxi Shen
- Department of Ophthalmology, Bascom Palmer Eye Institute, University of Miami Miller School of Medicine, Miami, Florida, USA
| | - Gissel Herrera
- Department of Ophthalmology, Bascom Palmer Eye Institute, University of Miami Miller School of Medicine, Miami, Florida, USA
| | - Qinqin Zhang
- Research and Development, Carl Zeiss Meditec, Inc., Dublin, CA, USA
| | - Giovanni Gregori
- Department of Ophthalmology, Bascom Palmer Eye Institute, University of Miami Miller School of Medicine, Miami, Florida, USA
| | - Philip J. Rosenfeld
- Department of Ophthalmology, Bascom Palmer Eye Institute, University of Miami Miller School of Medicine, Miami, Florida, USA
| | - Ruikang K. Wang
- Department of Bioengineering, University of Washington, Seattle, Washington, USA
- Department of Ophthalmology, University of Washington, Seattle, Washington, USA
| |
Collapse
|
6
|
Verejan V. Advancing Diabetic Retinopathy Diagnosis: Leveraging Optical Coherence Tomography Imaging with Convolutional Neural Networks. Rom J Ophthalmol 2023; 67:398-402. [PMID: 38239418 PMCID: PMC10793374 DOI: 10.22336/rjo.2023.63] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Accepted: 12/14/2023] [Indexed: 01/22/2024] Open
Abstract
Diabetic retinopathy (DR) is a vision-threatening complication of diabetes, necessitating early and accurate diagnosis. The combination of optical coherence tomography (OCT) imaging with convolutional neural networks (CNNs) has emerged as a promising approach for enhancing DR diagnosis. OCT provides detailed retinal morphology information, while CNNs analyze OCT images for automated detection and classification of DR. This paper reviews the current research on OCT imaging and CNNs for DR diagnosis, discussing their technical aspects and suitability. It explores CNN applications in detecting lesions, segmenting microaneurysms, and assessing disease severity, showing high sensitivity and accuracy. CNN models outperform traditional methods and rival expert ophthalmologists' results. However, challenges such as dataset availability and model interpretability remain. Future directions include multimodal imaging integration and real-time, point-of-care CNN systems for DR screening. The integration of OCT imaging with CNNs has transformative potential in DR diagnosis, facilitating early intervention, personalized treatments, and improved patient outcomes. Abbreviations: DR = Diabetic Retinopathy, OCT = Optical Coherence Tomography, CNN = Convolutional Neural Network, CMV = Cytomegalovirus, PDR = Proliferative Diabetic Retinopathy, AMD = Age-Related Macular Degeneration, VEGF = vascular endothelial growth factor, RAP = Retinal Angiomatous Proliferation, OCTA = OCT Angiography, AI = Artificial Intelligence.
Collapse
Affiliation(s)
- Victoria Verejan
- Department of Ophthalmology, “N. Testemițanu” State University of Medicine and Pharmacy, Chişinău, Republic of Moldova
| |
Collapse
|
7
|
Xie H, Xu W, Wang YX, Wu X. Deep learning network with differentiable dynamic programming for retina OCT surface segmentation. BIOMEDICAL OPTICS EXPRESS 2023; 14:3190-3202. [PMID: 37497505 PMCID: PMC10368040 DOI: 10.1364/boe.492670] [Citation(s) in RCA: 3] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Received: 04/10/2023] [Revised: 05/19/2023] [Accepted: 05/23/2023] [Indexed: 07/28/2023]
Abstract
Multiple-surface segmentation in optical coherence tomography (OCT) images is a challenging problem, further complicated by the frequent presence of weak image boundaries. Recently, many deep learning-based methods have been developed for this task and yield remarkable performance. Unfortunately, due to the scarcity of training data in medical imaging, it is challenging for deep learning networks to learn the global structure of the target surfaces, including surface smoothness. To bridge this gap, this study proposes to seamlessly unify a U-Net for feature learning with a constrained differentiable dynamic programming module to achieve end-to-end learning for retina OCT surface segmentation to explicitly enforce surface smoothness. It effectively utilizes the feedback from the downstream model optimization module to guide feature learning, yielding better enforcement of global structures of the target surfaces. Experiments on Duke AMD (age-related macular degeneration) and JHU MS (multiple sclerosis) OCT data sets for retinal layer segmentation demonstrated that the proposed method was able to achieve subvoxel accuracy on both datasets, with the mean absolute surface distance (MASD) errors of 1.88 ± 1.96μm and 2.75 ± 0.94μm, respectively, over all the segmented surfaces.
Collapse
Affiliation(s)
- Hui Xie
- Department of Electrical and Computer Engineering, The University of Iowa, Iowa City, IA, USA
| | - Weiyu Xu
- Department of Electrical and Computer Engineering, The University of Iowa, Iowa City, IA, USA
| | - Ya Xing Wang
- Beijing Institute of Ophthalmology, Beijing Tongren Hospital, Capital University of Medical Science, Beijing Ophthalmology and Visual Sciences Key Laboratory, Beijing, China
| | - Xiaodong Wu
- Department of Electrical and Computer Engineering, The University of Iowa, Iowa City, IA, USA
| |
Collapse
|
8
|
Mellak Y, Achim A, Ward A, Nicholson L, Descombes X. A machine learning framework for the quantification of experimental uveitis in murine OCT. BIOMEDICAL OPTICS EXPRESS 2023; 14:3413-3432. [PMID: 37497491 PMCID: PMC10368067 DOI: 10.1364/boe.489271] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 03/07/2023] [Revised: 05/11/2023] [Accepted: 05/22/2023] [Indexed: 07/28/2023]
Abstract
This paper presents methods for the detection and assessment of non-infectious uveitis, a leading cause of vision loss in working age adults. In the first part, we propose a classification model that can accurately predict the presence of uveitis and differentiate between different stages of the disease using optical coherence tomography (OCT) images. We utilize the Grad-CAM visualization technique to elucidate the decision-making process of the classifier and gain deeper insights into the results obtained. In the second part, we apply and compare three methods for the detection of detached particles in the retina that are indicative of uveitis. The first is a fully supervised detection method, the second is a marked point process (MPP) technique, and the third is a weakly supervised segmentation that produces per-pixel masks as output. The segmentation model is used as a backbone for a fully automated pipeline that can segment small particles of uveitis in two-dimensional (2-D) slices of the retina, reconstruct the volume, and produce centroids as points distribution in space. The number of particles in retinas is used to grade the disease, and point process analysis on centroids in three-dimensional (3-D) shows clustering patterns in the distribution of the particles on the retina.
Collapse
Affiliation(s)
- Youness Mellak
- Université Côte d’Azur, INRIA, CNRS, I3S, Sophia Antipolis, France
| | - Alin Achim
- University of Bristol, Bristol, United Kingdom
| | - Amy Ward
- University of Bristol, Bristol, United Kingdom
| | | | - Xavier Descombes
- Université Côte d’Azur, INRIA, CNRS, I3S, Sophia Antipolis, France
| |
Collapse
|
9
|
Dehghani S, Sommersperger M, Zhang P, Martin-Gomez A, Busam B, Gehlbach P, Navab N, Nasseri MA, Iordachita I. Robotic Navigation Autonomy for Subretinal Injection via Intelligent Real-Time Virtual iOCT Volume Slicing. IEEE INTERNATIONAL CONFERENCE ON ROBOTICS AND AUTOMATION : ICRA : [PROCEEDINGS]. IEEE INTERNATIONAL CONFERENCE ON ROBOTICS AND AUTOMATION 2023; 2023:4724-4731. [PMID: 38125032 PMCID: PMC10732544 DOI: 10.1109/icra48891.2023.10160372] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 12/23/2023]
Abstract
In the last decade, various robotic platforms have been introduced that could support delicate retinal surgeries. Concurrently, to provide semantic understanding of the surgical area, recent advances have enabled microscope-integrated intraoperative Optical Coherent Tomography (iOCT) with high-resolution 3D imaging at near video rate. The combination of robotics and semantic understanding enables task autonomy in robotic retinal surgery, such as for subretinal injection. This procedure requires precise needle insertion for best treatment outcomes. However, merging robotic systems with iOCT introduces new challenges. These include, but are not limited to high demands on data processing rates and dynamic registration of these systems during the procedure. In this work, we propose a framework for autonomous robotic navigation for subretinal injection, based on intelligent real-time processing of iOCT volumes. Our method consists of an instrument pose estimation method, an online registration between the robotic and the iOCT system, and trajectory planning tailored for navigation to an injection target. We also introduce intelligent virtual B-scans, a volume slicing approach for rapid instrument pose estimation, which is enabled by Convolutional Neural Networks (CNNs). Our experiments on ex-vivo porcine eyes demonstrate the precision and repeatability of the method. Finally, we discuss identified challenges in this work and suggest potential solutions to further the development of such systems.
Collapse
Affiliation(s)
- Shervin Dehghani
- Department of Computer Science, Technische Universität München, München 85748 Germany
- Laboratory for Computational Sensing and Robotics, Johns Hopkins University, Baltimore, MD, USA
| | - Michael Sommersperger
- Department of Computer Science, Technische Universität München, München 85748 Germany
| | - Peiyao Zhang
- Laboratory for Computational Sensing and Robotics, Johns Hopkins University, Baltimore, MD, USA
| | - Alejandro Martin-Gomez
- Laboratory for Computational Sensing and Robotics, Johns Hopkins University, Baltimore, MD, USA
| | - Benjamin Busam
- Department of Computer Science, Technische Universität München, München 85748 Germany
| | - Peter Gehlbach
- Wilmer Eye Institute, Johns Hopkins Hospital, Baltimore, MD, USA
| | - Nassir Navab
- Computer Aided Medical Procedures & Augmented Reality, Technical University of Munich, 85748 Munich, Germany, and an adjunct professor at the Whiting School of Engineering, Johns Hopkins University, Baltimore, MD, USA
| | - M. Ali Nasseri
- Department of Computer Science, Technische Universität München, München 85748 Germany
- Augenklinik und Poliklinik, Klinikum rechts der Isar der Technische Universität München, München 81675 Germany
| | - Iulian Iordachita
- Laboratory for Computational Sensing and Robotics, Johns Hopkins University, Baltimore, MD, USA
| |
Collapse
|
10
|
He Y, Carass A, Liu Y, Calabresi PA, Saidha S, Prince JL. Longitudinal deep network for consistent OCT layer segmentation. BIOMEDICAL OPTICS EXPRESS 2023; 14:1874-1893. [PMID: 37206119 PMCID: PMC10191669 DOI: 10.1364/boe.487518] [Citation(s) in RCA: 5] [Impact Index Per Article: 5.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Received: 02/10/2023] [Revised: 03/11/2023] [Accepted: 03/17/2023] [Indexed: 05/21/2023]
Abstract
Retinal layer thickness is an important bio-marker for people with multiple sclerosis (PwMS). In clinical practice, retinal layer thickness changes in optical coherence tomography (OCT) are widely used for monitoring multiple sclerosis (MS) progression. Recent developments in automated retinal layer segmentation algorithms allow cohort-level retina thinning to be observed in a large study of PwMS. However, variability in these results make it difficult to identify patient-level trends; this prevents patient specific disease monitoring and treatment planning using OCT. Deep learning based retinal layer segmentation algorithms have achieved state-of-the-art accuracy, but the segmentation is performed on each individual scan without utilizing longitudinal information, which can be important in reducing segmentation error and reveal subtle changes in retinal layers. In this paper, we propose a longitudinal OCT segmentation network which achieves more accurate and consistent layer thickness measurements for PwMS.
Collapse
Affiliation(s)
- Yufan He
- Dept. of Electrical and Computer Engineering, The Johns Hopkins University, Baltimore, MD 21218, USA
| | - Aaron Carass
- Dept. of Electrical and Computer Engineering, The Johns Hopkins University, Baltimore, MD 21218, USA
| | - Yihao Liu
- Dept. of Electrical and Computer Engineering, The Johns Hopkins University, Baltimore, MD 21218, USA
| | - Peter A. Calabresi
- Dept. of Neurology, The Johns Hopkins University School of Medicine, MD 21287, USA
| | - Shiv Saidha
- Dept. of Neurology, The Johns Hopkins University School of Medicine, MD 21287, USA
| | - Jerry L. Prince
- Dept. of Electrical and Computer Engineering, The Johns Hopkins University, Baltimore, MD 21218, USA
| |
Collapse
|
11
|
Zhou Y, Lin G, Yu X, Cao Y, Cheng H, Shi C, Jiang J, Gao H, Lu F, Shen M. Deep learning segmentation of the tear fluid reservoir under the sclera lens in optical coherence tomography images. BIOMEDICAL OPTICS EXPRESS 2023; 14:1848-1861. [PMID: 37206122 PMCID: PMC10191653 DOI: 10.1364/boe.480247] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Received: 12/19/2022] [Revised: 01/31/2023] [Accepted: 02/05/2023] [Indexed: 05/21/2023]
Abstract
The tear fluid reservoir (TFR) under the sclera lens is a unique characteristic providing optical neutralization of any aberrations from corneal irregularities. Anterior segment optical coherence tomography (AS-OCT) has become an important imaging modality for sclera lens fitting and visual rehabilitation therapy in both optometry and ophthalmology. Herein, we aimed to investigate whether deep learning can be used to segment the TFR from healthy and keratoconus eyes, with irregular corneal surfaces, in OCT images. Using AS-OCT, a dataset of 31850 images from 52 healthy and 46 keratoconus eyes, during sclera lens wear, was obtained and labeled with our previously developed algorithm of semi-automatic segmentation. A custom-improved U-shape network architecture with a full-range multi-scale feature-enhanced module (FMFE-Unet) was designed and trained. A hybrid loss function was designed to focus training on the TFR, to tackle the class imbalance problem. The experiments on our database showed an IoU, precision, specificity, and recall of 0.9426, 0.9678, 0.9965, and 0.9731, respectively. Furthermore, FMFE-Unet was found to outperform the other two state-of-the-art methods and ablation models, suggesting its strength in segmenting the TFR under the sclera lens depicted on OCT images. The application of deep learning for TFR segmentation in OCT images provides a powerful tool to assess changes in the dynamic tear film under the sclera lens, improving the efficiency and accuracy of lens fitting, and thus supporting the promotion of sclera lenses in clinical practice.
Collapse
Affiliation(s)
- Yuheng Zhou
- Eye Hospital and School of Ophthalmology and Optometry, Wenzhou Medical University, Wenzhou, Zhejiang 325000, China
| | - Guangqing Lin
- Eye Hospital and School of Ophthalmology and Optometry, Wenzhou Medical University, Wenzhou, Zhejiang 325000, China
| | - Xiangle Yu
- Eye Hospital and School of Ophthalmology and Optometry, Wenzhou Medical University, Wenzhou, Zhejiang 325000, China
| | - Yang Cao
- Eye Hospital and School of Ophthalmology and Optometry, Wenzhou Medical University, Wenzhou, Zhejiang 325000, China
| | - Hongling Cheng
- Eye Hospital and School of Ophthalmology and Optometry, Wenzhou Medical University, Wenzhou, Zhejiang 325000, China
| | - Ce Shi
- Eye Hospital and School of Ophthalmology and Optometry, Wenzhou Medical University, Wenzhou, Zhejiang 325000, China
| | - Jun Jiang
- Eye Hospital and School of Ophthalmology and Optometry, Wenzhou Medical University, Wenzhou, Zhejiang 325000, China
| | - Hebei Gao
- Oujiang Laboratory (Zhejiang Lab for Regenerative Medicine, Vision and Brain Health), Eye Hospital and School of Ophthalmology and Optometry, Wenzhou Medical University, Wenzhou, 325000, China
| | - Fan Lu
- Eye Hospital and School of Ophthalmology and Optometry, Wenzhou Medical University, Wenzhou, Zhejiang 325000, China
- Oujiang Laboratory (Zhejiang Lab for Regenerative Medicine, Vision and Brain Health), Eye Hospital and School of Ophthalmology and Optometry, Wenzhou Medical University, Wenzhou, 325000, China
| | - Meixiao Shen
- Eye Hospital and School of Ophthalmology and Optometry, Wenzhou Medical University, Wenzhou, Zhejiang 325000, China
- Oujiang Laboratory (Zhejiang Lab for Regenerative Medicine, Vision and Brain Health), Eye Hospital and School of Ophthalmology and Optometry, Wenzhou Medical University, Wenzhou, 325000, China
| |
Collapse
|
12
|
Lu J, Cheng Y, Li J, Liu Z, Shen M, Zhang Q, Liu J, Herrera G, Hiya FE, Morin R, Joseph J, Gregori G, Rosenfeld PJ, Wang RK. Automated segmentation and quantification of calcified drusen in 3D swept source OCT imaging. BIOMEDICAL OPTICS EXPRESS 2023; 14:1292-1306. [PMID: 36950236 PMCID: PMC10026581 DOI: 10.1364/boe.485999] [Citation(s) in RCA: 2] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Received: 01/18/2023] [Revised: 02/18/2023] [Accepted: 02/19/2023] [Indexed: 06/18/2023]
Abstract
Qualitative and quantitative assessments of calcified drusen are clinically important for determining the risk of disease progression in age-related macular degeneration (AMD). This paper reports the development of an automated algorithm to segment and quantify calcified drusen on swept-source optical coherence tomography (SS-OCT) images. The algorithm leverages the higher scattering property of calcified drusen compared with soft drusen. Calcified drusen have a higher optical attenuation coefficient (OAC), which results in a choroidal hypotransmission defect (hypoTD) below the calcified drusen. We show that it is possible to automatically segment calcified drusen from 3D SS-OCT scans by combining the OAC within drusen and the hypoTDs under drusen. We also propose a correction method for the segmentation of the retina pigment epithelium (RPE) overlying calcified drusen by automatically correcting the RPE by an amount of the OAC peak width along each A-line, leading to more accurate segmentation and quantification of drusen in general, and the calcified drusen in particular. A total of 29 eyes with nonexudative AMD and calcified drusen imaged with SS-OCT using the 6 × 6 mm2 scanning pattern were used in this study to test the performance of the proposed automated method. We demonstrated that the method achieved good agreement with the human expert graders in identifying the area of calcified drusen (Dice similarity coefficient: 68.27 ± 11.09%, correlation coefficient of the area measurements: r = 0.9422, the mean bias of the area measurements = 0.04781 mm2).
Collapse
Affiliation(s)
- Jie Lu
- Department of Bioengineering, University of Washington, Seattle, Washington, USA
| | - Yuxuan Cheng
- Department of Bioengineering, University of Washington, Seattle, Washington, USA
| | - Jianqing Li
- Department of Ophthalmology, Bascom Palmer Eye Institute, University of Miami Miller School of Medicine, Miami, Florida, USA
| | - Ziyu Liu
- Department of Bioengineering, University of Washington, Seattle, Washington, USA
| | - Mengxi Shen
- Department of Ophthalmology, Bascom Palmer Eye Institute, University of Miami Miller School of Medicine, Miami, Florida, USA
| | - Qinqin Zhang
- Department of Bioengineering, University of Washington, Seattle, Washington, USA
- Research and Development, Carl Zeiss Meditec, Inc., Dublin, CA, USA
| | - Jeremy Liu
- Department of Ophthalmology, Bascom Palmer Eye Institute, University of Miami Miller School of Medicine, Miami, Florida, USA
| | - Gissel Herrera
- Department of Ophthalmology, Bascom Palmer Eye Institute, University of Miami Miller School of Medicine, Miami, Florida, USA
| | - Farhan E. Hiya
- Department of Ophthalmology, Bascom Palmer Eye Institute, University of Miami Miller School of Medicine, Miami, Florida, USA
| | - Rosalyn Morin
- Department of Ophthalmology, Bascom Palmer Eye Institute, University of Miami Miller School of Medicine, Miami, Florida, USA
| | - Joan Joseph
- Department of Ophthalmology, Bascom Palmer Eye Institute, University of Miami Miller School of Medicine, Miami, Florida, USA
| | - Giovanni Gregori
- Department of Ophthalmology, Bascom Palmer Eye Institute, University of Miami Miller School of Medicine, Miami, Florida, USA
| | - Philip J. Rosenfeld
- Department of Ophthalmology, Bascom Palmer Eye Institute, University of Miami Miller School of Medicine, Miami, Florida, USA
| | - Ruikang K. Wang
- Department of Bioengineering, University of Washington, Seattle, Washington, USA
- Department of Ophthalmology, University of Washington, Seattle, Washington, USA
| |
Collapse
|
13
|
Zhou H, Liu J, Laiginhas R, Zhang Q, Cheng Y, Zhang Y, Shi Y, Shen M, Gregori G, Rosenfeld PJ, Wang RK. Depth-resolved visualization and automated quantification of hyperreflective foci on OCT scans using optical attenuation coefficients. BIOMEDICAL OPTICS EXPRESS 2022; 13:4175-4189. [PMID: 36032584 PMCID: PMC9408241 DOI: 10.1364/boe.467623] [Citation(s) in RCA: 9] [Impact Index Per Article: 4.5] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Received: 06/13/2022] [Revised: 06/25/2022] [Accepted: 06/25/2022] [Indexed: 05/11/2023]
Abstract
An automated depth-resolved algorithm using optical attenuation coefficients (OACs) was developed to visualize, localize, and quantify hyperreflective foci (HRF) seen on OCT imaging that are associated with macular hyperpigmentation and represent an increased risk of disease progression in age related macular degeneration. To achieve this, we first transformed the OCT scans to linear representation, which were then contrasted by OACs. HRF were visualized and localized within the entire scan by differentiating HRF within the retina from HRF along the retinal pigment epithelium (RPE). The total pigment burden was quantified using the en face sum projection of an OAC slab between the inner limiting membrane (ILM) to Bruch's membrane (BM). The manual total pigment burden measurements were also obtained by combining manual outlines of HRF in the B-scans with the total area of hypotransmission defects outlined on sub-RPE slabs, which was used as the reference to compare with those obtained from the automated algorithm. 6×6 mm swept-source OCT scans were collected from a total of 49 eyes from 42 patients with macular HRF. We demonstrate that the algorithm was able to automatically distinguish between HRF within the retina and HRF along the RPE. In 24 test eyes, the total pigment burden measurements by the automated algorithm were compared with measurements obtained from manual segmentations. A significant correlation was found between the total pigment area measurements from the automated and manual segmentations (P < 0.001). The proposed automated algorithm based on OACs should be useful in studying eye diseases involving HRF.
Collapse
Affiliation(s)
- Hao Zhou
- Department of Bioengineering, University of Washington, Seattle, WA 98105, USA
| | - Jeremy Liu
- Department of Ophthalmology, Bascom Palmer Eye Institute, University of Miami Miller School of Medicine, Miami, FL 33136, USA
| | - Rita Laiginhas
- Department of Ophthalmology, Bascom Palmer Eye Institute, University of Miami Miller School of Medicine, Miami, FL 33136, USA
| | - Qinqin Zhang
- Department of Bioengineering, University of Washington, Seattle, WA 98105, USA
| | - Yuxuan Cheng
- Department of Bioengineering, University of Washington, Seattle, WA 98105, USA
| | - Yi Zhang
- Department of Bioengineering, University of Washington, Seattle, WA 98105, USA
| | - Yingying Shi
- Department of Ophthalmology, Bascom Palmer Eye Institute, University of Miami Miller School of Medicine, Miami, FL 33136, USA
| | - Mengxi Shen
- Department of Ophthalmology, Bascom Palmer Eye Institute, University of Miami Miller School of Medicine, Miami, FL 33136, USA
| | - Giovanni Gregori
- Department of Ophthalmology, Bascom Palmer Eye Institute, University of Miami Miller School of Medicine, Miami, FL 33136, USA
| | - Philip J. Rosenfeld
- Department of Ophthalmology, Bascom Palmer Eye Institute, University of Miami Miller School of Medicine, Miami, FL 33136, USA
| | - Ruikang K. Wang
- Department of Bioengineering, University of Washington, Seattle, WA 98105, USA
- Karalis Johnson Retina Center, Department of Ophthalmology, University of Washington, Seattle, WA 98105, USA
| |
Collapse
|