1
|
Huang J, Zhang X, Jin R, Xu T, Jin Z, Shen M, Lv F, Chen J, Liu J. Wavelet-based selection-and-recalibration network for Parkinson's disease screening in OCT images. COMPUTER METHODS AND PROGRAMS IN BIOMEDICINE 2024; 256:108368. [PMID: 39154408 DOI: 10.1016/j.cmpb.2024.108368] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 01/30/2024] [Revised: 07/30/2024] [Accepted: 08/07/2024] [Indexed: 08/20/2024]
Abstract
BACKGROUND AND OBJECTIVE Parkinson's disease (PD) is one of the most prevalent neurodegenerative brain diseases worldwide. Therefore, accurate PD screening is crucial for early clinical intervention and treatment. Recent clinical research indicates that changes in pathology, such as the texture and thickness of the retinal layers, can serve as biomarkers for clinical PD diagnosis based on optical coherence tomography (OCT) images. However, the pathological manifestations of PD in the retinal layers are subtle compared to the more salient lesions associated with retinal diseases. METHODS Inspired by textural edge feature extraction in frequency domain learning, we aim to explore a potential approach to enhance the distinction between the feature distributions in retinal layers of PD cases and healthy controls. In this paper, we introduce a simple yet novel wavelet-based selection and recalibration module to effectively enhance the feature representations of the deep neural network by aggregating the unique clinical properties, such as the retinal layers in each frequency band. We combine this module with the residual block to form a deep network named Wavelet-based Selection and Recalibration Network (WaveSRNet) for automatic PD screening. RESULTS The extensive experiments on a clinical PD-OCT dataset and two publicly available datasets demonstrate that our approach outperforms state-of-the-art methods. Visualization analysis and ablation studies are conducted to enhance the explainability of WaveSRNet in the decision-making process. CONCLUSIONS Our results suggest the potential role of the retina as an assessment tool for PD. Visual analysis shows that PD-related elements include not only certain retinal layers but also the location of the fovea in OCT images.
Collapse
Affiliation(s)
- Jingqi Huang
- Research Institute of Trustworthy Autonomous Systems and Department of Computer Science and Engineering, Southern University of Science and Technology, Shenzhen, 518055, China
| | - Xiaoqing Zhang
- Research Institute of Trustworthy Autonomous Systems and Department of Computer Science and Engineering, Southern University of Science and Technology, Shenzhen, 518055, China; Center for High Performance Computing and Shenzhen Key Laboratory of Intelligent Bioinformatics, Shenzhen institute of Advanced Technology, Chinese Academy of Sciences, Shenzhen, 518055, China
| | - Richu Jin
- Research Institute of Trustworthy Autonomous Systems and Department of Computer Science and Engineering, Southern University of Science and Technology, Shenzhen, 518055, China
| | - Tao Xu
- The State Key Laboratory of Ophthalmology, Optometry and Vision Science, Wenzhou Medical University, Wenzhou, Zhejiang, China; The Oujiang Laboratory; The Affiliated Eye Hospital, Wenzhou Medical University, 270 Xueyuan Road, Wenzhou, Zhejiang, China
| | - Zi Jin
- National Engineering Research Center of Ophthalmology and Optometry, Eye Hospital, Wenzhou Medical University, Wenzhou, China; National Clinical Research Center for Ocular Diseases, Eye Hospital, Wenzhou Medical University, Wenzhou, China
| | - Meixiao Shen
- National Engineering Research Center of Ophthalmology and Optometry, Eye Hospital, Wenzhou Medical University, Wenzhou, China; National Clinical Research Center for Ocular Diseases, Eye Hospital, Wenzhou Medical University, Wenzhou, China
| | - Fan Lv
- The Oujiang Laboratory; The Affiliated Eye Hospital, Wenzhou Medical University, 270 Xueyuan Road, Wenzhou, Zhejiang, China; National Engineering Research Center of Ophthalmology and Optometry, Eye Hospital, Wenzhou Medical University, Wenzhou, China; National Clinical Research Center for Ocular Diseases, Eye Hospital, Wenzhou Medical University, Wenzhou, China
| | - Jiangfan Chen
- The State Key Laboratory of Ophthalmology, Optometry and Vision Science, Wenzhou Medical University, Wenzhou, Zhejiang, China; The Oujiang Laboratory; The Affiliated Eye Hospital, Wenzhou Medical University, 270 Xueyuan Road, Wenzhou, Zhejiang, China
| | - Jiang Liu
- Research Institute of Trustworthy Autonomous Systems and Department of Computer Science and Engineering, Southern University of Science and Technology, Shenzhen, 518055, China; The State Key Laboratory of Ophthalmology, Optometry and Vision Science, Wenzhou Medical University, Wenzhou, Zhejiang, China; Singapore Eye Research Institute, 169856, Singapore.
| |
Collapse
|
2
|
Akpinar MH, Sengur A, Faust O, Tong L, Molinari F, Acharya UR. Artificial intelligence in retinal screening using OCT images: A review of the last decade (2013-2023). COMPUTER METHODS AND PROGRAMS IN BIOMEDICINE 2024; 254:108253. [PMID: 38861878 DOI: 10.1016/j.cmpb.2024.108253] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 12/22/2023] [Revised: 04/22/2024] [Accepted: 05/25/2024] [Indexed: 06/13/2024]
Abstract
BACKGROUND AND OBJECTIVES Optical coherence tomography (OCT) has ushered in a transformative era in the domain of ophthalmology, offering non-invasive imaging with high resolution for ocular disease detection. OCT, which is frequently used in diagnosing fundamental ocular pathologies, such as glaucoma and age-related macular degeneration (AMD), plays an important role in the widespread adoption of this technology. Apart from glaucoma and AMD, we will also investigate pertinent pathologies, such as epiretinal membrane (ERM), macular hole (MH), macular dystrophy (MD), vitreomacular traction (VMT), diabetic maculopathy (DMP), cystoid macular edema (CME), central serous chorioretinopathy (CSC), diabetic macular edema (DME), diabetic retinopathy (DR), drusen, glaucomatous optic neuropathy (GON), neovascular AMD (nAMD), myopia macular degeneration (MMD) and choroidal neovascularization (CNV) diseases. This comprehensive review examines the role that OCT-derived images play in detecting, characterizing, and monitoring eye diseases. METHOD The 2020 PRISMA guideline was used to structure a systematic review of research on various eye conditions using machine learning (ML) or deep learning (DL) techniques. A thorough search across IEEE, PubMed, Web of Science, and Scopus databases yielded 1787 publications, of which 1136 remained after removing duplicates. Subsequent exclusion of conference papers, review papers, and non-open-access articles reduced the selection to 511 articles. Further scrutiny led to the exclusion of 435 more articles due to lower-quality indexing or irrelevance, resulting in 76 journal articles for the review. RESULTS During our investigation, we found that a major challenge for ML-based decision support is the abundance of features and the determination of their significance. In contrast, DL-based decision support is characterized by a plug-and-play nature rather than relying on a trial-and-error approach. Furthermore, we observed that pre-trained networks are practical and especially useful when working on complex images such as OCT. Consequently, pre-trained deep networks were frequently utilized for classification tasks. Currently, medical decision support aims to reduce the workload of ophthalmologists and retina specialists during routine tasks. In the future, it might be possible to create continuous learning systems that can predict ocular pathologies by identifying subtle changes in OCT images.
Collapse
Affiliation(s)
- Muhammed Halil Akpinar
- Department of Electronics and Automation, Vocational School of Technical Sciences, Istanbul University-Cerrahpasa, Istanbul, Turkey
| | - Abdulkadir Sengur
- Electrical-Electronics Engineering Department, Technology Faculty, Firat University, Elazig, Turkey.
| | - Oliver Faust
- School of Computing and Information Science, Anglia Ruskin University Cambridge Campus, United Kingdom
| | - Louis Tong
- Singapore Eye Research Institute, Singapore, Singapore
| | - Filippo Molinari
- Biolab, PolitoBIOMedLab, Department of Electronics and Telecommunications, Politecnico di Torino, Turin, Italy
| | - U Rajendra Acharya
- School of Mathematics, Physics and Computing, University of Southern Queensland, Springfield, Australia
| |
Collapse
|
3
|
Pang S, Zou B, Xiao X, Peng Q, Yan J, Zhang W, Yue K. A novel approach for automatic classification of macular degeneration OCT images. Sci Rep 2024; 14:19285. [PMID: 39164445 PMCID: PMC11335908 DOI: 10.1038/s41598-024-70175-2] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/14/2024] [Accepted: 08/13/2024] [Indexed: 08/22/2024] Open
Abstract
Age-related macular degeneration (AMD) and diabetic macular edema (DME) are significant causes of blindness worldwide. The prevalence of these diseases is steadily increasing due to population aging. Therefore, early diagnosis and prevention are crucial for effective treatment. Classification of Macular Degeneration OCT Images is a widely used method for assessing retinal lesions. However, there are two main challenges in OCT image classification: incomplete image feature extraction and lack of prominence in important positional features. To address these challenges, we proposed a deep learning neural network model called MSA-Net, which incorporates our proposed multi-scale architecture and spatial attention mechanism. Our multi-scale architecture is based on depthwise separable convolution, which ensures comprehensive feature extraction from multiple scales while minimizing the growth of model parameters. The spatial attention mechanism is aim to highlight the important positional features in the images, which emphasizes the representation of macular region features in OCT images. We test MSA-NET on the NEH dataset and the UCSD dataset, performing three-class (CNV, DURSEN, and NORMAL) and four-class (CNV, DURSEN, DME, and NORMAL) classification tasks. On the NEH dataset, the accuracy, sensitivity, and specificity are 98.1%, 97.9%, and 98.0%, respectively. After fine-tuning on the UCSD dataset, the accuracy, sensitivity, and specificity are 96.7%, 96.7%, and 98.9%, respectively. Experimental results demonstrate the excellent classification performance and generalization ability of our model compared to previous models and recent well-known OCT classification models, establishing it as a highly competitive intelligence classification approach in the field of macular degeneration.
Collapse
Affiliation(s)
- Shilong Pang
- School of Informatics, Hunan University of Chinese Medicine, Changsha, 410208, Hunan, China
| | - Beiji Zou
- School of Informatics, Hunan University of Chinese Medicine, Changsha, 410208, Hunan, China
- School of Computer Science and Engineering, Central South University, Changsha, 410083, Hunan, China
| | - Xiaoxia Xiao
- School of Informatics, Hunan University of Chinese Medicine, Changsha, 410208, Hunan, China.
| | - Qinghua Peng
- School of Traditional Chinese Medicine, Hunan University of Chinese Medicine, Changsha, 410208, Hunan, China
| | - Junfeng Yan
- School of Informatics, Hunan University of Chinese Medicine, Changsha, 410208, Hunan, China
| | - Wensheng Zhang
- School of Informatics, Hunan University of Chinese Medicine, Changsha, 410208, Hunan, China
- University of Chinese Academy of Sciences (UCAS), Beijing, 100049, China
- Research Center of Precision Sensing and Control, Institute of Automation, Chinese Academy of Sciences, Beijing, 100190, China
| | - Kejuan Yue
- School of Computer Science, Hunan First Normal University, Changsha, 410205, Hunan, China
| |
Collapse
|
4
|
Drakopoulos M, Hooshmand D, Machlab LA, Bryar PJ, Hammond KJ, Mirza RG. Machine Teaching Allows for Rapid Development of Automated Systems for Retinal Lesion Detection From Small Image Datasets. Ophthalmic Surg Lasers Imaging Retina 2024; 55:475-478. [PMID: 38752915 DOI: 10.3928/23258160-20240410-01] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 08/17/2024]
Abstract
Machine teaching, a machine learning subfield, may allow for rapid development of artificial intelligence systems able to automatically identify emerging ocular biomarkers from small imaging datasets. We sought to use machine teaching to automatically identify retinal ischemic perivascular lesions (RIPLs) and subretinal drusenoid deposits (SDDs), two emerging ocular biomarkers of cardiovascular disease. IRB approval was obtained. Four small datasets of SD-OCT B-scans were used to train and test two distinct automated systems, one identifying RIPLs and the other identifying SDDs. An open-source interactive machine-learning software program, RootPainter, was used to perform annotation and training simultaneously over a 6-hour period. For SDDs at the B-scan level, test-set accuracy = 92%, sensitivity = 100%, specificity = 88%, positive predictive value (PPV) = 82%, and negative predictive value (NPV) = 100%. For RIPLs at the B-scan level, test-set accuracy = 90%, sensitivity = 60%, specificity = 93%, PPV = 50%, and NPV = 95%. Machine teaching demonstrates promise within ophthalmic imaging to rapidly allow for automated identification of novel biomarkers from small image datasets. [Ophthalmic Surg Lasers Imaging Retina 2024;55:475-478.].
Collapse
|
5
|
Rozhyna A, Somfai GM, Atzori M, DeBuc DC, Saad A, Zoellin J, Müller H. Exploring Publicly Accessible Optical Coherence Tomography Datasets: A Comprehensive Overview. Diagnostics (Basel) 2024; 14:1668. [PMID: 39125544 PMCID: PMC11312046 DOI: 10.3390/diagnostics14151668] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/31/2024] [Revised: 07/15/2024] [Accepted: 07/25/2024] [Indexed: 08/12/2024] Open
Abstract
Artificial intelligence has transformed medical diagnostic capabilities, particularly through medical image analysis. AI algorithms perform well in detecting abnormalities with a strong performance, enabling computer-aided diagnosis by analyzing the extensive amounts of patient data. The data serve as a foundation upon which algorithms learn and make predictions. Thus, the importance of data cannot be underestimated, and clinically corresponding datasets are required. Many researchers face a lack of medical data due to limited access, privacy concerns, or the absence of available annotations. One of the most widely used diagnostic tools in ophthalmology is Optical Coherence Tomography (OCT). Addressing the data availability issue is crucial for enhancing AI applications in the field of OCT diagnostics. This review aims to provide a comprehensive analysis of all publicly accessible retinal OCT datasets. Our main objective is to compile a list of OCT datasets and their properties, which can serve as an accessible reference, facilitating data curation for medical image analysis tasks. For this review, we searched through the Zenodo repository, Mendeley Data repository, MEDLINE database, and Google Dataset search engine. We systematically evaluated all the identified datasets and found 23 open-access datasets containing OCT images, which significantly vary in terms of size, scope, and ground-truth labels. Our findings indicate the need for improvement in data-sharing practices and standardized documentation. Enhancing the availability and quality of OCT datasets will support the development of AI algorithms and ultimately improve diagnostic capabilities in ophthalmology. By providing a comprehensive list of accessible OCT datasets, this review aims to facilitate better utilization and development of AI in medical image analysis.
Collapse
Affiliation(s)
- Anastasiia Rozhyna
- Informatics Institute, University of Applied Sciences Western Switzerland (HES-SO), 3960 Sierre, Switzerland
- Medical Informatics, University of Geneva, 1205 Geneva, Switzerland
| | - Gábor Márk Somfai
- Department of Ophthalmology, Stadtspital Zürich, 8063 Zurich, Switzerland
- Spross Research Institute, 8063 Zurich, Switzerland
| | - Manfredo Atzori
- Informatics Institute, University of Applied Sciences Western Switzerland (HES-SO), 3960 Sierre, Switzerland
- Department of Neuroscience, University of Padua, 35121 Padova, Italy
| | - Delia Cabrera DeBuc
- Bascom Palmer Eye Institute, Miller School of Medicine, University of Miami, Miami, FL 33136, USA
| | - Amr Saad
- Department of Ophthalmology, Stadtspital Zürich, 8063 Zurich, Switzerland
- Spross Research Institute, 8063 Zurich, Switzerland
| | - Jay Zoellin
- Department of Ophthalmology, Stadtspital Zürich, 8063 Zurich, Switzerland
- Spross Research Institute, 8063 Zurich, Switzerland
| | - Henning Müller
- Informatics Institute, University of Applied Sciences Western Switzerland (HES-SO), 3960 Sierre, Switzerland
- Medical Informatics, University of Geneva, 1205 Geneva, Switzerland
- The Sense Research and Innovation Center, 1007 Lausanne, Switzerland
| |
Collapse
|
6
|
Salaheldin AM, Abdel Wahed M, Saleh N. A hybrid model for the detection of retinal disorders using artificial intelligence techniques. Biomed Phys Eng Express 2024; 10:055005. [PMID: 38955139 DOI: 10.1088/2057-1976/ad5db2] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/17/2024] [Accepted: 07/02/2024] [Indexed: 07/04/2024]
Abstract
The prevalence of vision impairment is increasing at an alarming rate. The goal of the study was to create an automated method that uses optical coherence tomography (OCT) to classify retinal disorders into four categories: choroidal neovascularization, diabetic macular edema, drusen, and normal cases. This study proposed a new framework that combines machine learning and deep learning-based techniques. The utilized classifiers were support vector machine (SVM), K-nearest neighbor (K-NN), decision tree (DT), and ensemble model (EM). A feature extractor, the InceptionV3 convolutional neural network, was also employed. The performance of the models was evaluated against nine criteria using a dataset of 18000 OCT images. For the SVM, K-NN, DT, and EM classifiers, the analysis exhibited state-of-the-art performance, with classification accuracies of 99.43%, 99.54%, 97.98%, and 99.31%, respectively. A promising methodology has been introduced for the automatic identification and classification of retinal disorders, leading to reduced human error and saved time.
Collapse
Affiliation(s)
- Ahmed M Salaheldin
- Systems and Biomedical Engineering Department, Faculty of Engineering, Cairo University, Giza, Egypt
- Systems and Biomedical Engineering Department, Higher Institute of Engineering, EL Shorouk Academy, Cairo, Egypt
| | - Manal Abdel Wahed
- Systems and Biomedical Engineering Department, Faculty of Engineering, Cairo University, Giza, Egypt
| | - Neven Saleh
- Systems and Biomedical Engineering Department, Higher Institute of Engineering, EL Shorouk Academy, Cairo, Egypt
- Electrical Communication and Electronic Systems Engineering Department, Engineering Faculty, October University for Modern Sciences and Arts, Giza, Egypt
| |
Collapse
|
7
|
Azizi MM, Abhari S, Sajedi H. Stitched vision transformer for age-related macular degeneration detection using retinal optical coherence tomography images. PLoS One 2024; 19:e0304943. [PMID: 38837967 DOI: 10.1371/journal.pone.0304943] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/23/2023] [Accepted: 05/21/2024] [Indexed: 06/07/2024] Open
Abstract
Age-related macular degeneration (AMD) is an eye disease that leads to the deterioration of the central vision area of the eye and can gradually result in vision loss in elderly individuals. Early identification of this disease can significantly impact patient treatment outcomes. Furthermore, given the increasing elderly population globally, the importance of automated methods for rapidly monitoring at-risk individuals and accurately diagnosing AMD is growing daily. One standard method for diagnosing AMD is using optical coherence tomography (OCT) images as a non-invasive imaging technology. In recent years, numerous deep neural networks have been proposed for the classification of OCT images. Utilizing pre-trained neural networks can speed up model deployment in related tasks without compromising accuracy. However, most previous methods overlook the feasibility of leveraging pre-existing trained networks to search for an optimal architecture for AMD staging on a new target dataset. In this study, our objective was to achieve an optimal architecture in the efficiency-accuracy trade-off for classifying retinal OCT images. To this end, we employed pre-trained medical vision transformer (MedViT) models. MedViT combines convolutional and transformer neural networks, explicitly designed for medical image classification. Our approach involved pre-training two distinct MedViT models on a source dataset with labels identical to those in the target dataset. This pre-training was conducted in a supervised manner. Subsequently, we evaluated the performance of the pre-trained MedViT models for classifying retinal OCT images from the target Noor Eye Hospital (NEH) dataset into the normal, drusen, and choroidal neovascularization (CNV) classes in zero-shot settings and through five-fold cross-validation. Then, we proposed a stitching approach to search for an optimal model from two MedViT family models. The proposed stitching method is an efficient architecture search algorithm known as stitchable neural networks. Stitchable neural networks create a candidate model in search space for each pair of stitchable layers by inserting a linear layer between them. A pair of stitchable layers consists of layers, each selected from one input model. While stitchable neural networks had previously been tested on more extensive and general datasets, this study demonstrated that stitching networks could also be helpful in smaller medical datasets. The results of this approach indicate that when pre-trained models were available for OCT images from another dataset, it was possible to achieve a model in 100 epochs with an accuracy of over 94.9% in classifying images from the NEH dataset. The results of this study demonstrate the efficacy of stitchable neural networks as a fine-tuning method for OCT image classification. This approach not only leads to higher accuracy but also considers architecture optimization at a reasonable computational cost.
Collapse
Affiliation(s)
- Mohammad Mahdi Azizi
- Department of Mathematics, Statistics and Computer Science, College of Science, University of Tehran, Tehran, Iran
| | - Setareh Abhari
- Department of Mathematics, Statistics and Computer Science, College of Science, University of Tehran, Tehran, Iran
| | - Hedieh Sajedi
- Department of Mathematics, Statistics and Computer Science, College of Science, University of Tehran, Tehran, Iran
| |
Collapse
|
8
|
Azoad Ahnaf SM, Saha S, Frost S, Atiqur Rahaman GM. Understanding and interpreting CNN's decision in optical coherence tomography-based AMD detection. Eur J Ophthalmol 2024; 34:803-815. [PMID: 37671441 DOI: 10.1177/11206721231199126] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 09/07/2023]
Abstract
INTRODUCTION Automated assessment of age-related macular degeneration (AMD) using optical coherence tomography (OCT) has gained significant research attention in recent years. Though a list of convolutional neural network (CNN)-based methods has been proposed recently, methods that uncover the decision-making process of CNNs or critically interpret CNNs' decisions in the context are scant. This study aims to bridge this research gap. METHODS We independently trained several state-of-the-art CNN models, namely, VGG16, VGG19, Xception, ResNet50, InceptionResNetV2 for AMD detection and applied CNN visualization techniques, namely, Grad-CAM, Grad-CAM++, Score CAM, Faster Score CAM to highlight the regions of interest utilized by the CNNs in the context. Retinal layer segmentation methods were also developed to explore how the CNN regions of interest related to the layers of the retinal structure. Extensive experiments involving 2130 SD-OCT scans collected from Duke University were performed. RESULTS Experimental analysis shows that Outer Nuclear Layer to Inner Segment Myeloid (ONL-ISM) influences the AMD detection decision heavily as evident from the normalized intersection (NI) scores. For AMD cases the obtained average NI scores were respectively 13.13%, 17.2%, 9.7%, 10.95%, and 11.31% for VGG16, VGG19, ResNet50, Xception, and Inception ResNet V2, whereas, for normal cases, these values were respectively 21.7%, 21.3%, 16.85%, 10.175% and 16%. CONCLUSION Critical analysis reveals that the ONL-ISM is the most contributing layer in determining AMD, followed by Nerve Fiber Layer to Inner Plexiform Layer (NFL-IPL).
Collapse
Affiliation(s)
- S M Azoad Ahnaf
- Computational Color and Spectral Image Analysis Lab, Computer Science and Engineering Discipline, Khulna University, Khulna 9208, Bangladesh
| | - Sajib Saha
- Australian e-Health Research Centre, Commonwealth Scientific and Industrial Research Organisation (CSIRO), Perth, Australia
| | - Shaun Frost
- Australian e-Health Research Centre, Commonwealth Scientific and Industrial Research Organisation (CSIRO), Perth, Australia
| | - G M Atiqur Rahaman
- Computational Color and Spectral Image Analysis Lab, Computer Science and Engineering Discipline, Khulna University, Khulna 9208, Bangladesh
| |
Collapse
|
9
|
Poddar R, Shukla V, Alam Z, Mohan M. Automatic segmentation of layers in chorio-retinal complex using Graph-based method for ultra-speed 1.7 MHz wide field swept source FDML optical coherence tomography. Med Biol Eng Comput 2024; 62:1375-1393. [PMID: 38191981 DOI: 10.1007/s11517-023-03007-6] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/23/2023] [Accepted: 12/20/2023] [Indexed: 01/10/2024]
Abstract
The posterior segment of the human eye complex contains two discrete microstructure and vasculature network systems, namely, the retina and choroid. We present a single segmentation framework technique for segmenting the entire layers present in the chorio-retinal complex of the human eye using optical coherence tomography (OCT) images. This automatic program is based on the graph theory method. This single program is capable of segmenting seven layers of the retina and choroid scleral interface. The graph theory was utilized to find the probability matrix and subsequent boundaries of different layers. The program was also implemented to segment angiographic maps of different chorio-retinal layers using "segmentation matrices." The method was tested and successfully validated on OCT images from six normal human eyes as well as eyes with non-exudative age-related macular degeneration (AMD). The thickness of microstructure and microvasculature for different layers located in the chorio-retinal segment of the eye was also generated and compared. A decent efficiency in terms of processing time, sensitivity, and accuracy was observed compared to the manual segmentation and other existing methods. The proposed method automatically segments whole OCT images of chorio-retinal complex with augmented probability maps generation in OCT volume dataset. We have also evaluated the segmentation results using quantitative metrics such as Dice coefficient and Hausdorff distance This method realizes a mean descent Dice similarity coefficient (DSC) value of 0.82 (range, 0.816-0.864) for RPE and CSI layer.
Collapse
Affiliation(s)
- Raju Poddar
- Biophotonics Lab, Department of Bioengineering & Biotechnology, Birla Institute of Technology-Mesra, Ranchi, JH, 835 215, India.
| | - Vinita Shukla
- Biophotonics Lab, Department of Bioengineering & Biotechnology, Birla Institute of Technology-Mesra, Ranchi, JH, 835 215, India
| | - Zoya Alam
- Biophotonics Lab, Department of Bioengineering & Biotechnology, Birla Institute of Technology-Mesra, Ranchi, JH, 835 215, India
| | - Muktesh Mohan
- Biophotonics Lab, Department of Bioengineering & Biotechnology, Birla Institute of Technology-Mesra, Ranchi, JH, 835 215, India
| |
Collapse
|
10
|
Yang J, Wang G, Xiao X, Bao M, Tian G. Explainable ensemble learning method for OCT detection with transfer learning. PLoS One 2024; 19:e0296175. [PMID: 38517913 PMCID: PMC10959366 DOI: 10.1371/journal.pone.0296175] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/02/2023] [Accepted: 12/07/2023] [Indexed: 03/24/2024] Open
Abstract
The accuracy and interpretability of artificial intelligence (AI) are crucial for the advancement of optical coherence tomography (OCT) image detection, as it can greatly reduce the manual labor required by clinicians. By prioritizing these aspects during development and application, we can make significant progress towards streamlining the clinical workflow. In this paper, we propose an explainable ensemble approach that utilizes transfer learning to detect fundus lesion diseases through OCT imaging. Our study utilized a publicly available OCT dataset consisting of normal subjects, patients with dry age-related macular degeneration (AMD), and patients with diabetic macular edema (DME), each with 15 samples. The impact of pre-trained weights on the performance of individual networks was first compared, and then these networks were ensemble using majority soft polling. Finally, the features learned by the networks were visualized using Grad-CAM and CAM. The use of pre-trained ImageNet weights improved the performance from 68.17% to 92.89%. The ensemble model consisting of the three CNN models with pre-trained parameters loaded performed best, correctly distinguishing between AMD patients, DME patients and normal subjects 100% of the time. Visualization results showed that Grad-CAM could display the lesion area more accurately. It is demonstrated that the proposed approach could have good performance of both accuracy and interpretability in retinal OCT image detection.
Collapse
Affiliation(s)
- Jiasheng Yang
- Academician Workstation, Changsha Medical University, Changsha, Hunan, China
| | - Guanfang Wang
- School of Mathematics and Statistics, Nanjing University of Information Science and Technology, Nanjing, Jiangsu, China
- Geneis Beijing Co. Ltd., Beijing, China
| | - Xu Xiao
- School of International Education, Anhui University of Technology, Maanshan, Anhui, China
| | - Meihua Bao
- Academician Workstation, Changsha Medical University, Changsha, Hunan, China
| | - Geng Tian
- Geneis Beijing Co. Ltd., Beijing, China
| |
Collapse
|
11
|
Abubakar H, Al-Turjman F, Ameen ZS, Mubarak AS, Altrjman C. A hybridized feature extraction for COVID-19 multi-class classification on computed tomography images. Heliyon 2024; 10:e26939. [PMID: 38463848 PMCID: PMC10920381 DOI: 10.1016/j.heliyon.2024.e26939] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/12/2023] [Revised: 02/20/2024] [Accepted: 02/21/2024] [Indexed: 03/12/2024] Open
Abstract
COVID-19 has killed more than 5 million individuals worldwide within a short time. It is caused by SARS-CoV-2 which continuously mutates and produces more transmissible new different strains. It is therefore of great significance to diagnose COVID-19 early to curb its spread and reduce the death rate. Owing to the COVID-19 pandemic, traditional diagnostic methods such as reverse-transcription polymerase chain reaction (RT-PCR) are ineffective for diagnosis. Medical imaging is among the most effective techniques of respiratory disorders detection through machine learning and deep learning. However, conventional machine learning methods depend on extracted and engineered features, whereby the optimum features influence the classifier's performance. In this study, Histogram of Oriented Gradient (HOG) and eight deep learning models were utilized for feature extraction while K-Nearest Neighbour (KNN) and Support Vector Machines (SVM) were used for classification. A combined feature of HOG and deep learning feature was proposed to improve the performance of the classifiers. VGG-16 + HOG achieved 99.4 overall accuracy with SVM. This indicates that our proposed concatenated feature can enhance the SVM classifier's performance in COVID-19 detection.
Collapse
Affiliation(s)
- Hassana Abubakar
- Biomedical Engineering Department, Faculty of Engineering, Near East University, Mersin 10, Turkey
| | - Fadi Al-Turjman
- Artificial Intelligence Engineering Department, AI and Robotics Institute, Near East University, Mersin 10, Turkey
- Research Center for AI and IoT, Faculty of Engineering, University of Kyrenia, Mersin 10, Turkey
| | - Zubaida S. Ameen
- Operational Research Center in Healthcare, Near East University, Mersin 10, Turkey
| | - Auwalu S. Mubarak
- Operational Research Center in Healthcare, Near East University, Mersin 10, Turkey
| | - Chadi Altrjman
- Waterloo University, 200 University Avenue West. Waterloo, ON, Canada
| |
Collapse
|
12
|
Hughes-Cano JA, Quiroz-Mercado H, Hernández-Zimbrón LF, García-Franco R, Rubio Mijangos JF, López-Star E, García-Roa M, Lansingh VC, Olivares-Pinto U, Thébault SC. Improved predictive diagnosis of diabetic macular edema based on hybrid models: An observational study. Comput Biol Med 2024; 170:107979. [PMID: 38219645 DOI: 10.1016/j.compbiomed.2024.107979] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/09/2023] [Revised: 12/11/2023] [Accepted: 01/08/2024] [Indexed: 01/16/2024]
Abstract
Diabetic Macular Edema (DME) is the most common sight-threatening complication of type 2 diabetes. Optical Coherence Tomography (OCT) is the most useful imaging technique to diagnose, follow up, and evaluate treatments for DME. However, OCT exam and devices are expensive and unavailable in all clinics in low- and middle-income countries. Our primary goal was therefore to develop an alternative method to OCT for DME diagnosis by introducing spectral information derived from spontaneous electroretinogram (ERG) signals as a single input or combined with fundus that is much more widespread. Baseline ERGs were recorded in 233 patients and transformed into scalograms and spectrograms via Wavelet and Fourier transforms, respectively. Using transfer learning, distinct Convolutional Neural Networks (CNN) were trained as classifiers for DME using OCT, scalogram, spectrogram, and eye fundus images. Input data were randomly split into training and test sets with a proportion of 80 %-20 %, respectively. The top performers for each input type were selected, OpticNet-71 for OCT, DenseNet-201 for eye fundus, and non-evoked ERG-derived scalograms, to generate a combined model by assigning different weights for each of the selected models. Model validation was performed using a dataset alien to the training phase of the models. None of the models powered by mock ERG-derived input performed well. In contrast, hybrid models showed better results, in particular, the model powered by eye fundus combined with mock ERG-derived information with a 91 % AUC and 86 % F1-score, and the model powered by OCT and mock ERG-derived scalogram images with a 93 % AUC and 89 % F1-score. These data show that the spontaneous ERG-derived input adds predictive value to the fundus- and OCT-based models to diagnose DME, except for the sensitivity of the OCT model which remains the same. The inclusion of mock ERG signals, which have recently been shown to take only 5 min to record in daylight conditions, therefore represents a potential improvement over existing OCT-based models, as well as a reliable and cost-effective alternative when combined with the fundus, especially in underserved areas, to predict DME.
Collapse
Affiliation(s)
- J A Hughes-Cano
- Laboratorio de Investigación Traslacional en Salud Visual, Instituto de Neurobiología, Universidad Nacional Autónoma de México (UNAM), Campus Juriquilla, Querétaro, Mexico
| | - H Quiroz-Mercado
- Research Department, Asociación Para Evitar la Ceguera, Mexico City, Mexico
| | | | - R García-Franco
- Instituto Mexicano de Oftalmología (IMO), I.A.P., Circuito Exterior Estadio Corregidora Sn, Centro Sur, 76090 Santiago de Querétaro, Querétaro, Mexico
| | - J F Rubio Mijangos
- Instituto Mexicano de Oftalmología (IMO), I.A.P., Circuito Exterior Estadio Corregidora Sn, Centro Sur, 76090 Santiago de Querétaro, Querétaro, Mexico
| | - E López-Star
- Instituto Mexicano de Oftalmología (IMO), I.A.P., Circuito Exterior Estadio Corregidora Sn, Centro Sur, 76090 Santiago de Querétaro, Querétaro, Mexico
| | - M García-Roa
- Instituto Mexicano de Oftalmología (IMO), I.A.P., Circuito Exterior Estadio Corregidora Sn, Centro Sur, 76090 Santiago de Querétaro, Querétaro, Mexico
| | - V C Lansingh
- Instituto Mexicano de Oftalmología (IMO), I.A.P., Circuito Exterior Estadio Corregidora Sn, Centro Sur, 76090 Santiago de Querétaro, Querétaro, Mexico; HelpMeSee, Inc., 20 West 36th Street, Floor 4, New York, NY, 10018-8005, USA
| | - U Olivares-Pinto
- Escuela Nacional de Estudios Superiores Unidad Juriquilla, Universidad Nacional Autónoma de México (UNAM), Campus Juriquilla, Querétaro, Mexico
| | - S C Thébault
- Laboratorio de Investigación Traslacional en Salud Visual, Instituto de Neurobiología, Universidad Nacional Autónoma de México (UNAM), Campus Juriquilla, Querétaro, Mexico.
| |
Collapse
|
13
|
Song A, Lusk JB, Roh KM, Hsu ST, Valikodath NG, Lad EM, Muir KW, Engelhard MM, Limkakeng AT, Izatt JA, McNabb RP, Kuo AN. RobOCTNet: Robotics and Deep Learning for Referable Posterior Segment Pathology Detection in an Emergency Department Population. Transl Vis Sci Technol 2024; 13:12. [PMID: 38488431 PMCID: PMC10946693 DOI: 10.1167/tvst.13.3.12] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/26/2023] [Accepted: 01/31/2024] [Indexed: 03/19/2024] Open
Abstract
Purpose To evaluate the diagnostic performance of a robotically aligned optical coherence tomography (RAOCT) system coupled with a deep learning model in detecting referable posterior segment pathology in OCT images of emergency department patients. Methods A deep learning model, RobOCTNet, was trained and internally tested to classify OCT images as referable versus non-referable for ophthalmology consultation. For external testing, emergency department patients with signs or symptoms warranting evaluation of the posterior segment were imaged with RAOCT. RobOCTNet was used to classify the images. Model performance was evaluated against a reference standard based on clinical diagnosis and retina specialist OCT review. Results We included 90,250 OCT images for training and 1489 images for internal testing. RobOCTNet achieved an area under the curve (AUC) of 1.00 (95% confidence interval [CI], 0.99-1.00) for detection of referable posterior segment pathology in the internal test set. For external testing, RAOCT was used to image 72 eyes of 38 emergency department patients. In this set, RobOCTNet had an AUC of 0.91 (95% CI, 0.82-0.97), a sensitivity of 95% (95% CI, 87%-100%), and a specificity of 76% (95% CI, 62%-91%). The model's performance was comparable to two human experts' performance. Conclusions A robotically aligned OCT coupled with a deep learning model demonstrated high diagnostic performance in detecting referable posterior segment pathology in a cohort of emergency department patients. Translational Relevance Robotically aligned OCT coupled with a deep learning model may have the potential to improve emergency department patient triage for ophthalmology referral.
Collapse
Affiliation(s)
- Ailin Song
- Duke University School of Medicine, Durham, NC, USA
- Department of Ophthalmology, Duke University, Durham, NC, USA
| | - Jay B. Lusk
- Duke University School of Medicine, Durham, NC, USA
| | - Kyung-Min Roh
- Department of Ophthalmology, Duke University, Durham, NC, USA
| | - S. Tammy Hsu
- Department of Ophthalmology, Duke University, Durham, NC, USA
| | | | - Eleonora M. Lad
- Department of Ophthalmology, Duke University, Durham, NC, USA
| | - Kelly W. Muir
- Department of Ophthalmology, Duke University, Durham, NC, USA
| | - Matthew M. Engelhard
- Department of Biostatistics and Bioinformatics, Duke University, Durham, NC, USA
| | | | - Joseph A. Izatt
- Department of Biomedical Engineering, Duke University, Durham, NC, USA
| | - Ryan P. McNabb
- Department of Ophthalmology, Duke University, Durham, NC, USA
| | - Anthony N. Kuo
- Department of Ophthalmology, Duke University, Durham, NC, USA
- Department of Biomedical Engineering, Duke University, Durham, NC, USA
| |
Collapse
|
14
|
Alizadeh Eghtedar R, Vard A, Malekahmadi M, Peyman A. A new computer-aided diagnosis tool based on deep learning methods for automatic detection of retinal disorders from OCT images. Int Ophthalmol 2024; 44:110. [PMID: 38396074 DOI: 10.1007/s10792-024-03033-9] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/10/2023] [Accepted: 01/11/2024] [Indexed: 02/25/2024]
Abstract
PURPOSE Early detection of retinal disorders using optical coherence tomography (OCT) images can prevent vision loss. Since manual screening can be time-consuming, tedious, and fallible, we present a reliable computer-aided diagnosis (CAD) software based on deep learning. Also, we made efforts to increase the interpretability of the deep learning methods, overcome their vague and black box nature, and also understand their behavior in the diagnosis. METHODS We propose a novel method to improve the interpretability of the used deep neural network by embedding the rich semantic information of abnormal areas based on the ophthalmologists' interpretations and medical descriptions in the OCT images. Finally, we trained the classification network on a small subset of the online publicly available University of California San Diego (UCSD) dataset with an overall of 29,800 OCT images. RESULTS The experimental results on the 1000 test OCT images show that the proposed method achieves the overall precision, accuracy, sensitivity, and f1-score of 97.6%, 97.6%, 97.6%, and 97.59%, respectively. Also, the heat map images provide a clear region of interest which indicates that the interpretability of the proposed method is increased dramatically. CONCLUSION The proposed software can help ophthalmologists in providing a second opinion to make a decision, and primitive automated diagnoses of retinal diseases and even it can be used as a screening tool, in eye clinics. Also, the improvement of the interpretability of the proposed method causes to increase in the model generalization, and therefore, it will work properly on a wide range of other OCT datasets.
Collapse
Affiliation(s)
- Reza Alizadeh Eghtedar
- Department of Bioelectrics and Biomedical Engineering, School of Advanced Technologies in Medicine, Isfahan University of Medical Sciences, Isfahan, Iran
- Student Research Committee, Isfahan University of Medical Sciences, Isfahan, Iran
| | - Alireza Vard
- Department of Bioelectrics and Biomedical Engineering, School of Advanced Technologies in Medicine, Isfahan University of Medical Sciences, Isfahan, Iran.
- Medical Image & Signal Processing Research Center, School of Advanced Technologies in Medicine, Isfahan University of Medical Sciences, Isfahan, Iran.
| | - Mohammad Malekahmadi
- Department of Ophthalmology, School of Medicine, Isfahan University of Medical Sciences, Isfahan, Iran
- Isfahan Eye Research Center, Department of Ophthalmology, Isfahan University of Medical Sciences, Isfahan, Iran
| | - Alireza Peyman
- Department of Ophthalmology, School of Medicine, Isfahan University of Medical Sciences, Isfahan, Iran
- Isfahan Eye Research Center, Department of Ophthalmology, Isfahan University of Medical Sciences, Isfahan, Iran
| |
Collapse
|
15
|
Xiao Z, Zhang X, Zheng B, Guo Y, Higashita R, Liu J. Multi-style spatial attention module for cortical cataract classification in AS-OCT image with supervised contrastive learning. COMPUTER METHODS AND PROGRAMS IN BIOMEDICINE 2024; 244:107958. [PMID: 38070390 DOI: 10.1016/j.cmpb.2023.107958] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 08/30/2023] [Revised: 10/30/2023] [Accepted: 11/27/2023] [Indexed: 01/26/2024]
Abstract
BACKGROUND AND OBJECTIVE Precise cortical cataract (CC) classification plays a significant role in early cataract intervention and surgery. Anterior segment optical coherence tomography (AS-OCT) images have shown excellent potential in cataract diagnosis. However, due to the complex opacity distributions of CC, automatic AS-OCT-based CC classification has been rarely studied. In this paper, we aim to explore the opacity distribution characteristics of CC as clinical priori to enhance the representational capability of deep convolutional neural networks (CNNs) in CC classification tasks. METHODS We propose a novel architectural unit, Multi-style Spatial Attention module (MSSA), which recalibrates intermediate feature maps by exploiting diverse clinical contexts. MSSA first extracts the clinical style context features with Group-wise Style Pooling (GSP), then refines the clinical style context features with Local Transform (LT), and finally executes group-wise feature map recalibration via Style Feature Recalibration (SFR). MSSA can be easily integrated into modern CNNs with negligible overhead. RESULTS The extensive experiments on a CASIA2 AS-OCT dataset and two public ophthalmic datasets demonstrate the superiority of MSSA over state-of-the-art attention methods. The visualization analysis and ablation study are conducted to improve the explainability of MSSA in the decision-making process. CONCLUSIONS Our proposed MSSANet utilized the opacity distribution characteristics of CC to enhance the representational power and explainability of deep convolutional neural network (CNN) and improve the CC classification performance. Our proposed method has the potential in the early clinical CC diagnosis.
Collapse
Affiliation(s)
- Zunjie Xiao
- Department of Computer Science and Engineering, Southern University of Science and Technology, Shenzhen, 518055, China.
| | - Xiaoqing Zhang
- Department of Computer Science and Engineering, Southern University of Science and Technology, Shenzhen, 518055, China; Research Institute of Trustworthy Autonomous Systems, Southern University of Science and Technology, Shenzhen, 518055, China.
| | - Bofang Zheng
- Department of Computer Science and Engineering, Southern University of Science and Technology, Shenzhen, 518055, China.
| | - Yitong Guo
- Department of Computer Science and Engineering, Southern University of Science and Technology, Shenzhen, 518055, China.
| | - Risa Higashita
- Department of Computer Science and Engineering, Southern University of Science and Technology, Shenzhen, 518055, China; TOMEY Corporation, Nagoya, 4510051, Japan.
| | - Jiang Liu
- Department of Computer Science and Engineering, Southern University of Science and Technology, Shenzhen, 518055, China; Research Institute of Trustworthy Autonomous Systems, Southern University of Science and Technology, Shenzhen, 518055, China; Guangdong Provincial Key Laboratory of Brain-inspired Intelligent Computation, Southern University of Science and Technology, Shenzhen, 518055, China; Singapore Eye Research Institute, 169856, Singapore.
| |
Collapse
|
16
|
Peng J, Lu J, Zhuo J, Li P. Multi-Scale-Denoising Residual Convolutional Network for Retinal Disease Classification Using OCT. SENSORS (BASEL, SWITZERLAND) 2023; 24:150. [PMID: 38203011 PMCID: PMC10781341 DOI: 10.3390/s24010150] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 11/21/2023] [Revised: 12/13/2023] [Accepted: 12/15/2023] [Indexed: 01/12/2024]
Abstract
Macular pathologies can cause significant vision loss. Optical coherence tomography (OCT) images of the retina can assist ophthalmologists in diagnosing macular diseases. Traditional deep learning networks for retinal disease classification cannot extract discriminative features under strong noise conditions in OCT images. To address this issue, we propose a multi-scale-denoising residual convolutional network (MS-DRCN) for classifying retinal diseases. Specifically, the MS-DRCN includes a soft-denoising block (SDB), a multi-scale context block (MCB), and a feature fusion block (FFB). The SDB can determine the threshold for soft thresholding automatically, which removes speckle noise features efficiently. The MCB is designed to capture multi-scale context information and strengthen extracted features. The FFB is dedicated to integrating high-resolution and low-resolution features to precisely identify variable lesion areas. Our approach achieved classification accuracies of 96.4% and 96.5% on the OCT2017 and OCT-C4 public datasets, respectively, outperforming other classification methods. To evaluate the robustness of our method, we introduced Gaussian noise and speckle noise with varying PSNRs into the test set of the OCT2017 dataset. The results of our anti-noise experiments demonstrate that our approach exhibits superior robustness compared with other methods, yielding accuracy improvements ranging from 0.6% to 2.9% when compared with ResNet under various PSNR noise conditions.
Collapse
Affiliation(s)
- Jinbo Peng
- State Key Laboratory of Digital Medical Engineering, School of Biomedical Engineering, Hainan University, Haiko 570228, China; (J.P.); (J.L.)
- Key Laboratory of Biomedical Engineering of Hainan Province, One Health Institute, Hainan University, Haiko 570228, China
- Research Unit of Multimodal Cross Scale Neural Signal Detection and Imaging, Chinese Academy of Medical Science, HUST-Suzhou Institute for Brainsmatics, Jiangsu Industrial Technology Research Institute (JITRI), Suzhou 215100, China
| | - Jinling Lu
- State Key Laboratory of Digital Medical Engineering, School of Biomedical Engineering, Hainan University, Haiko 570228, China; (J.P.); (J.L.)
- Key Laboratory of Biomedical Engineering of Hainan Province, One Health Institute, Hainan University, Haiko 570228, China
- Research Unit of Multimodal Cross Scale Neural Signal Detection and Imaging, Chinese Academy of Medical Science, HUST-Suzhou Institute for Brainsmatics, Jiangsu Industrial Technology Research Institute (JITRI), Suzhou 215100, China
- Britton Chance Center for Biomedical Photonics and MoE Key Laboratory for Biomedical Photonics, Wuhan National Laboratory for Optoelectronics, Huazhong University of Science and Technology, Wuhan 430074, China
| | - Junjie Zhuo
- State Key Laboratory of Digital Medical Engineering, School of Biomedical Engineering, Hainan University, Haiko 570228, China; (J.P.); (J.L.)
- Key Laboratory of Biomedical Engineering of Hainan Province, One Health Institute, Hainan University, Haiko 570228, China
- Research Unit of Multimodal Cross Scale Neural Signal Detection and Imaging, Chinese Academy of Medical Science, HUST-Suzhou Institute for Brainsmatics, Jiangsu Industrial Technology Research Institute (JITRI), Suzhou 215100, China
| | - Pengcheng Li
- State Key Laboratory of Digital Medical Engineering, School of Biomedical Engineering, Hainan University, Haiko 570228, China; (J.P.); (J.L.)
- Key Laboratory of Biomedical Engineering of Hainan Province, One Health Institute, Hainan University, Haiko 570228, China
- Research Unit of Multimodal Cross Scale Neural Signal Detection and Imaging, Chinese Academy of Medical Science, HUST-Suzhou Institute for Brainsmatics, Jiangsu Industrial Technology Research Institute (JITRI), Suzhou 215100, China
- Britton Chance Center for Biomedical Photonics and MoE Key Laboratory for Biomedical Photonics, Wuhan National Laboratory for Optoelectronics, Huazhong University of Science and Technology, Wuhan 430074, China
| |
Collapse
|
17
|
Opoku M, Weyori BA, Adekoya AF, Adu K. CLAHE-CapsNet: Efficient retina optical coherence tomography classification using capsule networks with contrast limited adaptive histogram equalization. PLoS One 2023; 18:e0288663. [PMID: 38032915 PMCID: PMC10688733 DOI: 10.1371/journal.pone.0288663] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/03/2023] [Accepted: 07/01/2023] [Indexed: 12/02/2023] Open
Abstract
Manual detection of eye diseases using retina Optical Coherence Tomography (OCT) images by Ophthalmologists is time consuming, prone to errors and tedious. Previous researchers have developed a computer aided system using deep learning-based convolutional neural networks (CNNs) to aid in faster detection of the retina diseases. However, these methods find it difficult to achieve better classification performance due to noise in the OCT image. Moreover, the pooling operations in CNN reduce resolution of the image that limits the performance of the model. The contributions of the paper are in two folds. Firstly, this paper makes a comprehensive literature review to establish current-state-of-act methods successfully implemented in retina OCT image classifications. Additionally, this paper proposes a capsule network coupled with contrast limited adaptive histogram equalization (CLAHE-CapsNet) for retina OCT image classification. The CLAHE was implemented as layers to minimize the noise in the retina image for better performance of the model. A three-layer convolutional capsule network was designed with carefully chosen hyperparameters. The dataset used for this study was presented by University of California San Diego (UCSD). The dataset consists of 84,495 X-Ray images (JPEG) and 4 categories (NORMAL, CNV, DME, and DRUSEN). The images went through a grading system consisting of multiple layers of trained graders of expertise for verification and correction of image labels. Evaluation experiments were conducted and comparison of results was done with state-of-the-art models to find out the best performing model. The evaluation metrics; accuracy, sensitivity, precision, specificity, and AUC are used to determine the performance of the models. The evaluation results show that the proposed model achieves the best performing model of accuracies of 97.7%, 99.5%, and 99.3% on overall accuracy (OA), overall sensitivity (OS), and overall precision (OP), respectively. The results obtained indicate that the proposed model can be adopted and implemented to help ophthalmologists in detecting retina OCT diseases.
Collapse
Affiliation(s)
- Michael Opoku
- Department of Computer Science and Informatics, University of Energy and Natural Resource, Sunyani, Ghana
| | - Benjamin Asubam Weyori
- Department of Computer Science and Informatics, University of Energy and Natural Resource, Sunyani, Ghana
| | - Adebayo Felix Adekoya
- Department of Computer Science and Informatics, University of Energy and Natural Resource, Sunyani, Ghana
| | - Kwabena Adu
- Department of Computer Science and Informatics, University of Energy and Natural Resource, Sunyani, Ghana
| |
Collapse
|
18
|
Saidi L, Jomaa H, Zainab H, Zgolli H, Mabrouk S, Sidibé D, Tabia H, Khlifa N. Automatic Detection of AMD and DME Retinal Pathologies Using Deep Learning. Int J Biomed Imaging 2023; 2023:9966107. [PMID: 38046618 PMCID: PMC10691890 DOI: 10.1155/2023/9966107] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/12/2023] [Revised: 10/09/2023] [Accepted: 11/03/2023] [Indexed: 12/05/2023] Open
Abstract
Diabetic macular edema (DME) and age-related macular degeneration (AMD) are two common eye diseases. They are often undiagnosed or diagnosed late. This can result in permanent and irreversible vision loss. Therefore, early detection and treatment of these diseases can prevent vision loss, save money, and provide a better quality of life for individuals. Optical coherence tomography (OCT) imaging is widely applied to identify eye diseases, including DME and AMD. In this work, we developed automatic deep learning-based methods to detect these pathologies using SD-OCT scans. The convolutional neural network (CNN) from scratch we developed gave the best classification score with an accuracy higher than 99% on Duke dataset of OCT images.
Collapse
Affiliation(s)
- Latifa Saidi
- Laboratory of Biophysics and Medical Technologies, Higher Institute of Medical Technologies of Tunis, University of Tunis El Manar, Tunis, Tunisia
| | - Hajer Jomaa
- Laboratory of Biophysics and Medical Technologies, Higher Institute of Medical Technologies of Tunis, University of Tunis El Manar, Tunis, Tunisia
| | - Haddad Zainab
- Laboratory of Biophysics and Medical Technologies, National Engineering School Tunis, University of Tunis El Manar, Tunis, Tunisia
| | - Hsouna Zgolli
- Department A, Hedi Raies of Ophthalmology Institute, Tunis, Tunisia
| | - Sonia Mabrouk
- Department A, Hedi Raies of Ophthalmology Institute, Tunis, Tunisia
| | - Désiré Sidibé
- IBISC, University of Paris-Saclay, Univ Evry, Evry, France
| | - Hedi Tabia
- IBISC, University of Paris-Saclay, Univ Evry, Evry, France
| | - Nawres Khlifa
- Laboratory of Biophysics and Medical Technologies, Higher Institute of Medical Technologies of Tunis, University of Tunis El Manar, Tunis, Tunisia
| |
Collapse
|
19
|
Ye X, He S, Zhong X, Yu J, Yang S, Shen Y, Chen Y, Wang Y, Huang X, Shen L. OIMHS: An Optical Coherence Tomography Image Dataset Based on Macular Hole Manual Segmentation. Sci Data 2023; 10:769. [PMID: 37932307 PMCID: PMC10628143 DOI: 10.1038/s41597-023-02675-1] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/15/2023] [Accepted: 10/24/2023] [Indexed: 11/08/2023] Open
Abstract
Macular holes, one of the most common macular diseases, require timely treatment. The morphological changes on optical coherence tomography (OCT) images provided an opportunity for direct observation of the disease, and accurate segmentation was needed to identify and quantify the lesions. Developments of such algorithms had been obstructed by a lack of high-quality datasets (the OCT images and the corresponding gold standard macular hole segmentation labels), especially for supervised learning-based segmentation algorithms. In such context, we established a large OCT image macular hole segmentation (OIMHS) dataset with 3859 B-scan images of 119 patients, and each image provided four segmentation labels: retina, macular hole, intraretinal cysts, and choroid. This dataset offered an excellent opportunity for investigating the accuracy and reliability of different segmentation algorithms for macular holes and a new research insight into the further development of clinical research for macular diseases, which included the retina, lesions, and choroid in quantitative analyses.
Collapse
Affiliation(s)
- Xin Ye
- Center for Rehabilitation Medicine, Department of Ophthalmology, Zhejiang Provincial People's Hospital (Affiliated People's Hospital, Hangzhou Medical College), Hangzhou, Zhejiang, China
| | - Shucheng He
- Center for Rehabilitation Medicine, Department of Ophthalmology, Zhejiang Provincial People's Hospital (Affiliated People's Hospital, Hangzhou Medical College), Hangzhou, Zhejiang, China
| | - Xiaxing Zhong
- Wenzhou Medical University, Wenzhou, Zhejiang, China
| | - Jiafeng Yu
- Center for Rehabilitation Medicine, Department of Ophthalmology, Zhejiang Provincial People's Hospital (Affiliated People's Hospital, Hangzhou Medical College), Hangzhou, Zhejiang, China
| | | | - Yingjiao Shen
- Center for Rehabilitation Medicine, Department of Ophthalmology, Zhejiang Provincial People's Hospital (Affiliated People's Hospital, Hangzhou Medical College), Hangzhou, Zhejiang, China
| | - Yiqi Chen
- Center for Rehabilitation Medicine, Department of Ophthalmology, Zhejiang Provincial People's Hospital (Affiliated People's Hospital, Hangzhou Medical College), Hangzhou, Zhejiang, China
| | - Yaqi Wang
- College of Media Engineering, Communication University of Zhejiang, Hangzhou, China
| | - Xingru Huang
- School of Electronic Engineering and Computer Science, Queen Mary University of London, London, UK.
| | - Lijun Shen
- Center for Rehabilitation Medicine, Department of Ophthalmology, Zhejiang Provincial People's Hospital (Affiliated People's Hospital, Hangzhou Medical College), Hangzhou, Zhejiang, China.
| |
Collapse
|
20
|
Baharlouei Z, Rabbani H, Plonka G. Wavelet scattering transform application in classification of retinal abnormalities using OCT images. Sci Rep 2023; 13:19013. [PMID: 37923770 PMCID: PMC10624695 DOI: 10.1038/s41598-023-46200-1] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/10/2022] [Accepted: 10/29/2023] [Indexed: 11/06/2023] Open
Abstract
To assist ophthalmologists in diagnosing retinal abnormalities, Computer Aided Diagnosis has played a significant role. In this paper, a particular Convolutional Neural Network based on Wavelet Scattering Transform (WST) is used to detect one to four retinal abnormalities from Optical Coherence Tomography (OCT) images. Predefined wavelet filters in this network decrease the computation complexity and processing time compared to deep learning methods. We use two layers of the WST network to obtain a direct and efficient model. WST generates a sparse representation of the images which is translation-invariant and stable concerning local deformations. Next, a Principal Component Analysis classifies the extracted features. We evaluate the model using four publicly available datasets to have a comprehensive comparison with the literature. The accuracies of classifying the OCT images of the OCTID dataset into two and five classes were [Formula: see text] and [Formula: see text], respectively. We achieved an accuracy of [Formula: see text] in detecting Diabetic Macular Edema from Normal ones using the TOPCON device-based dataset. Heidelberg and Duke datasets contain DME, Age-related Macular Degeneration, and Normal classes, in which we achieved accuracy of [Formula: see text] and [Formula: see text], respectively. A comparison of our results with the state-of-the-art models shows that our model outperforms these models for some assessments or achieves nearly the best results reported so far while having a much smaller computational complexity.
Collapse
Affiliation(s)
- Zahra Baharlouei
- Medical Image and Signal Processing Research Center, School of Advanced Technologies in Medicine, Isfahan University of Medical Sciences, Isfahan, Iran
| | - Hossein Rabbani
- Medical Image and Signal Processing Research Center, School of Advanced Technologies in Medicine, Isfahan University of Medical Sciences, Isfahan, Iran.
| | - Gerlind Plonka
- Institute for Numerical and Applied Mathematics, Georg-August-University of Goettingen, Göttingen, Germany
| |
Collapse
|
21
|
Daich Varela M, Sen S, De Guimaraes TAC, Kabiri N, Pontikos N, Balaskas K, Michaelides M. Artificial intelligence in retinal disease: clinical application, challenges, and future directions. Graefes Arch Clin Exp Ophthalmol 2023; 261:3283-3297. [PMID: 37160501 PMCID: PMC10169139 DOI: 10.1007/s00417-023-06052-x] [Citation(s) in RCA: 13] [Impact Index Per Article: 13.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/26/2023] [Revised: 03/20/2023] [Accepted: 03/24/2023] [Indexed: 05/11/2023] Open
Abstract
Retinal diseases are a leading cause of blindness in developed countries, accounting for the largest share of visually impaired children, working-age adults (inherited retinal disease), and elderly individuals (age-related macular degeneration). These conditions need specialised clinicians to interpret multimodal retinal imaging, with diagnosis and intervention potentially delayed. With an increasing and ageing population, this is becoming a global health priority. One solution is the development of artificial intelligence (AI) software to facilitate rapid data processing. Herein, we review research offering decision support for the diagnosis, classification, monitoring, and treatment of retinal disease using AI. We have prioritised diabetic retinopathy, age-related macular degeneration, inherited retinal disease, and retinopathy of prematurity. There is cautious optimism that these algorithms will be integrated into routine clinical practice to facilitate access to vision-saving treatments, improve efficiency of healthcare systems, and assist clinicians in processing the ever-increasing volume of multimodal data, thereby also liberating time for doctor-patient interaction and co-development of personalised management plans.
Collapse
Affiliation(s)
- Malena Daich Varela
- UCL Institute of Ophthalmology, London, UK
- Moorfields Eye Hospital, London, UK
| | | | | | | | - Nikolas Pontikos
- UCL Institute of Ophthalmology, London, UK
- Moorfields Eye Hospital, London, UK
| | | | - Michel Michaelides
- UCL Institute of Ophthalmology, London, UK.
- Moorfields Eye Hospital, London, UK.
| |
Collapse
|
22
|
Bui PN, Le DT, Bum J, Kim S, Song SJ, Choo H. Multi-Scale Learning with Sparse Residual Network for Explainable Multi-Disease Diagnosis in OCT Images. Bioengineering (Basel) 2023; 10:1249. [PMID: 38002373 PMCID: PMC10669434 DOI: 10.3390/bioengineering10111249] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/02/2023] [Revised: 10/19/2023] [Accepted: 10/23/2023] [Indexed: 11/26/2023] Open
Abstract
In recent decades, medical imaging techniques have revolutionized the field of disease diagnosis, enabling healthcare professionals to noninvasively observe the internal structures of the human body. Among these techniques, optical coherence tomography (OCT) has emerged as a powerful and versatile tool that allows high-resolution, non-invasive, and real-time imaging of biological tissues. Deep learning algorithms have been successfully employed to detect and classify various retinal diseases in OCT images, enabling early diagnosis and treatment planning. However, existing deep learning algorithms are primarily designed for single-disease diagnosis, which limits their practical application in clinical settings where OCT images often contain symptoms of multiple diseases. In this paper, we propose an effective approach for multi-disease diagnosis in OCT images using a multi-scale learning (MSL) method and a sparse residual network (SRN). Specifically, the MSL method extracts and fuses useful features from images of different sizes to enhance the discriminative capability of a classifier and make the disease predictions interpretable. The SRN is a minimal residual network, where convolutional layers with large kernel sizes are replaced with multiple convolutional layers that have smaller kernel sizes, thereby reducing model complexity while achieving a performance similar to that of existing convolutional neural networks. The proposed multi-scale sparse residual network significantly outperforms existing methods, exhibiting 97.40% accuracy, 95.38% sensitivity, and 98.25% specificity. Experimental results show the potential of our method to improve explainable diagnosis systems for various eye diseases via visual discrimination.
Collapse
Affiliation(s)
- Phuoc-Nguyen Bui
- Department of AI Systems Engineering, Sungkyunkwan University, Suwon 16419, Republic of Korea;
| | - Duc-Tai Le
- College of Computing and Informatics, Sungkyunkwan University, Suwon 16419, Republic of Korea;
| | - Junghyun Bum
- Sungkyun AI Research Institute, Sungkyunkwan University, Suwon 16419, Republic of Korea;
| | - Seongho Kim
- Department of Ophthalmology, Kangbuk Samsung Hospital, School of Medicine, Sungkyunkwan University, Seoul 03181, Republic of Korea;
| | - Su Jeong Song
- Department of Ophthalmology, Kangbuk Samsung Hospital, School of Medicine, Sungkyunkwan University, Seoul 03181, Republic of Korea;
- Biomedical Institute for Convergence, Sungkyunkwan University, Suwon 16419, Republic of Korea
| | - Hyunseung Choo
- Department of AI Systems Engineering, Sungkyunkwan University, Suwon 16419, Republic of Korea;
- College of Computing and Informatics, Sungkyunkwan University, Suwon 16419, Republic of Korea;
- Department of Electrical and Computer Engineering, Sungkyunkwan University, Suwon 16419, Republic of Korea
| |
Collapse
|
23
|
Gholami S, Lim JI, Leng T, Ong SSY, Thompson AC, Alam MN. Federated learning for diagnosis of age-related macular degeneration. Front Med (Lausanne) 2023; 10:1259017. [PMID: 37901412 PMCID: PMC10613107 DOI: 10.3389/fmed.2023.1259017] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/14/2023] [Accepted: 09/25/2023] [Indexed: 10/31/2023] Open
Abstract
This paper presents a federated learning (FL) approach to train deep learning models for classifying age-related macular degeneration (AMD) using optical coherence tomography image data. We employ the use of residual network and vision transformer encoders for the normal vs. AMD binary classification, integrating four unique domain adaptation techniques to address domain shift issues caused by heterogeneous data distribution in different institutions. Experimental results indicate that FL strategies can achieve competitive performance similar to centralized models even though each local model has access to a portion of the training data. Notably, the Adaptive Personalization FL strategy stood out in our FL evaluations, consistently delivering high performance across all tests due to its additional local model. Furthermore, the study provides valuable insights into the efficacy of simpler architectures in image classification tasks, particularly in scenarios where data privacy and decentralization are critical using both encoders. It suggests future exploration into deeper models and other FL strategies for a more nuanced understanding of these models' performance. Data and code are available at https://github.com/QIAIUNCC/FL_UNCC_QIAI.
Collapse
Affiliation(s)
- Sina Gholami
- Department of Electrical Engineering, University of North Carolina at Charlotte, Charlotte, NC, United States
| | - Jennifer I. Lim
- Department of Ophthalmology and Visual Science, University of Illinois at Chicago, Chicago, IL, United States
| | - Theodore Leng
- Department of Ophthalmology, School of Medicine, Stanford University, Stanford, CA, United States
| | - Sally Shin Yee Ong
- Department of Surgical Ophthalmology, Atrium-Health Wake Forest Baptist, Winston-Salem, NC, United States
| | - Atalie Carina Thompson
- Department of Surgical Ophthalmology, Atrium-Health Wake Forest Baptist, Winston-Salem, NC, United States
| | - Minhaj Nur Alam
- Department of Electrical Engineering, University of North Carolina at Charlotte, Charlotte, NC, United States
| |
Collapse
|
24
|
Bhandari M, Shahi TB, Neupane A. Evaluating Retinal Disease Diagnosis with an Interpretable Lightweight CNN Model Resistant to Adversarial Attacks. J Imaging 2023; 9:219. [PMID: 37888326 PMCID: PMC10607865 DOI: 10.3390/jimaging9100219] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/01/2023] [Revised: 09/29/2023] [Accepted: 10/09/2023] [Indexed: 10/28/2023] Open
Abstract
Optical Coherence Tomography (OCT) is an imperative symptomatic tool empowering the diagnosis of retinal diseases and anomalies. The manual decision towards those anomalies by specialists is the norm, but its labor-intensive nature calls for more proficient strategies. Consequently, the study recommends employing a Convolutional Neural Network (CNN) for the classification of OCT images derived from the OCT dataset into distinct categories, including Choroidal NeoVascularization (CNV), Diabetic Macular Edema (DME), Drusen, and Normal. The average k-fold (k = 10) training accuracy, test accuracy, validation accuracy, training loss, test loss, and validation loss values of the proposed model are 96.33%, 94.29%, 94.12%, 0.1073, 0.2002, and 0.1927, respectively. Fast Gradient Sign Method (FGSM) is employed to introduce non-random noise aligned with the cost function's data gradient, with varying epsilon values scaling the noise, and the model correctly handles all noise levels below 0.1 epsilon. Explainable AI algorithms: Local Interpretable Model-Agnostic Explanations (LIME) and SHapley Additive exPlanations (SHAP) are utilized to provide human interpretable explanations approximating the behaviour of the model within the region of a particular retinal image. Additionally, two supplementary datasets, namely, COVID-19 and Kidney Stone, are assimilated to enhance the model's robustness and versatility, resulting in a level of precision comparable to state-of-the-art methodologies. Incorporating a lightweight CNN model with 983,716 parameters, 2.37×108 floating point operations per second (FLOPs) and leveraging explainable AI strategies, this study contributes to efficient OCT-based diagnosis, underscores its potential in advancing medical diagnostics, and offers assistance in the Internet-of-Medical-Things.
Collapse
Affiliation(s)
- Mohan Bhandari
- Department of Science and Technology, Samriddhi College, Bhaktapur 44800, Nepal;
| | - Tej Bahadur Shahi
- School of Engineering and Technology, Central Queensland University, Norman Gardens, Rockhampton, QLD 4701, Australia;
- Central Department of Computer Science and IT, Tribhuvan University, Kathmandu 44600, Nepal
| | - Arjun Neupane
- School of Engineering and Technology, Central Queensland University, Norman Gardens, Rockhampton, QLD 4701, Australia;
| |
Collapse
|
25
|
Nawaz M, Uvaliyev A, Bibi K, Wei H, Abaxi SMD, Masood A, Shi P, Ho HP, Yuan W. Unraveling the complexity of Optical Coherence Tomography image segmentation using machine and deep learning techniques: A review. Comput Med Imaging Graph 2023; 108:102269. [PMID: 37487362 DOI: 10.1016/j.compmedimag.2023.102269] [Citation(s) in RCA: 2] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/03/2023] [Revised: 06/30/2023] [Accepted: 07/03/2023] [Indexed: 07/26/2023]
Abstract
Optical Coherence Tomography (OCT) is an emerging technology that provides three-dimensional images of the microanatomy of biological tissue in-vivo and at micrometer-scale resolution. OCT imaging has been widely used to diagnose and manage various medical diseases, such as macular degeneration, glaucoma, and coronary artery disease. Despite its wide range of applications, the segmentation of OCT images remains difficult due to the complexity of tissue structures and the presence of artifacts. In recent years, different approaches have been used for OCT image segmentation, such as intensity-based, region-based, and deep learning-based methods. This paper reviews the major advances in state-of-the-art OCT image segmentation techniques. It provides an overview of the advantages and limitations of each method and presents the most relevant research works related to OCT image segmentation. It also provides an overview of existing datasets and discusses potential clinical applications. Additionally, this review gives an in-depth analysis of machine learning and deep learning approaches for OCT image segmentation. It outlines challenges and opportunities for further research in this field.
Collapse
Affiliation(s)
- Mehmood Nawaz
- Department of Biomedical Engineering, The Chinese University of Hong Kong, Hong Kong Special Administrative Region of China
| | - Adilet Uvaliyev
- Department of Biomedical Engineering, The Chinese University of Hong Kong, Hong Kong Special Administrative Region of China
| | - Khadija Bibi
- Department of Biomedical Engineering, The Chinese University of Hong Kong, Hong Kong Special Administrative Region of China
| | - Hao Wei
- Department of Biomedical Engineering, The Chinese University of Hong Kong, Hong Kong Special Administrative Region of China
| | - Sai Mu Dalike Abaxi
- Department of Biomedical Engineering, The Chinese University of Hong Kong, Hong Kong Special Administrative Region of China
| | - Anum Masood
- Department of Circulation and Medical Imaging, Norwegian University of Science and Technology, Trondheim, Norway
| | - Peilun Shi
- Department of Biomedical Engineering, The Chinese University of Hong Kong, Hong Kong Special Administrative Region of China
| | - Ho-Pui Ho
- Department of Biomedical Engineering, The Chinese University of Hong Kong, Hong Kong Special Administrative Region of China
| | - Wu Yuan
- Department of Biomedical Engineering, The Chinese University of Hong Kong, Hong Kong Special Administrative Region of China.
| |
Collapse
|
26
|
Wang J, Zong Y, He Y, Shi G, Jiang C. Domain Adaptation-Based Automated Detection of Retinal Diseases from Optical Coherence Tomography Images. Curr Eye Res 2023; 48:836-842. [PMID: 37203787 DOI: 10.1080/02713683.2023.2212878] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/09/2022] [Revised: 05/03/2023] [Accepted: 05/05/2023] [Indexed: 05/20/2023]
Abstract
PURPOSE To verify the effectiveness of domain adaptation in generalizing a deep learning-based anomaly detection model to unseen optical coherence tomography (OCT) images. METHODS Two datasets (source and target, where labelled training data was only available for the source) captured by two different OCT facilities were collected to train the model. We defined the model containing a feature extractor and a classifier as Model One and trained it with only labeled source data. The proposed domain adaptation model was defined as Model Two, which has the same feature extractor and classifier as Model One but has an additional domain critic in the training phase. We trained the Model Two with both the source and target datasets; the feature extractor was trained to extract domain-invariant features while the domain critic learned to capture the domain discrepancy. Finally, a well-trained feature extractor was used to extract domain-invariant features and a classifier was used to detect images with retinal pathologies in the two domains. RESULTS The target data consisted of 3,058 OCT B-scans captured from 163 participants. Model One achieved an area under the curve (AUC) of 0.912 [95% confidence interval (CI), 0.895-0.962], while Model Two achieved an overall AUC of 0.989 [95% CI, 0.982-0.993] for detecting pathological retinas from healthy samples. Moreover, Model Two achieved an average retinopathies detection accuracy of 94.52%. Heat maps showed that the algorithm focused on the area with pathological changes during processing, similar to manual grading in daily clinical work. CONCLUSIONS The proposed domain adaptation model showed a strong ability in reducing the domain distance between different OCT datasets.
Collapse
Affiliation(s)
- Jing Wang
- Suzhou Institute of Biomedical Engineering and Technology, Chinese Academy of Science, Suzhou, People's Republic of China
- School of Biomedical Engineering (Suzhou), Division of Life Sciences and Medicine, University of Science and Technology of China, Suzhou, People's Republic of China
| | - Yuan Zong
- Department of Ophthalmology and Vision Science, Eye and ENT Hospital, Fudan University, Shanghai, People's Republic of China
- Key Laboratory of Myopia of State Health Ministry and Key Laboratory of Visual Impairment and Restoration, Shanghai, People's Republic of China
| | - Yi He
- Suzhou Institute of Biomedical Engineering and Technology, Chinese Academy of Science, Suzhou, People's Republic of China
- School of Biomedical Engineering (Suzhou), Division of Life Sciences and Medicine, University of Science and Technology of China, Suzhou, People's Republic of China
| | - Guohua Shi
- Suzhou Institute of Biomedical Engineering and Technology, Chinese Academy of Science, Suzhou, People's Republic of China
- School of Biomedical Engineering (Suzhou), Division of Life Sciences and Medicine, University of Science and Technology of China, Suzhou, People's Republic of China
| | - Chunhui Jiang
- Department of Ophthalmology and Vision Science, Eye and ENT Hospital, Fudan University, Shanghai, People's Republic of China
- Key Laboratory of Myopia of State Health Ministry and Key Laboratory of Visual Impairment and Restoration, Shanghai, People's Republic of China
| |
Collapse
|
27
|
Tripathi A, Kumar P, Mayya V, Tulsani A. Generating OCT B-Scan DME images using optimized Generative Adversarial Networks (GANs). Heliyon 2023; 9:e18773. [PMID: 37609420 PMCID: PMC10440457 DOI: 10.1016/j.heliyon.2023.e18773] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/03/2023] [Revised: 07/26/2023] [Accepted: 07/27/2023] [Indexed: 08/24/2023] Open
Abstract
Diabetic Macular Edema (DME) represents a significant visual impairment among individuals with diabetes, leading to a dramatic reduction in visual acuity and potentially resulting in irreversible vision loss. Optical Coherence Tomography (OCT), a technique that produces high-resolution retinal images, plays a vital role in the clinical assessment of this condition. Physicians typically rely on OCT B-Scan images to evaluate DME severity. However, manual interpretation of these images is susceptible to errors, which can lead to detrimental consequences, such as misdiagnosis and improper treatment strategies. Hence, there is a critical need for more reliable diagnostic methods. This study aims to address this gap by proposing an automated model based on Generative Adversarial Networks (GANs) to generate OCT B-Scan images of DME. The model synthesizes images from patients' baseline OCT B-Scan images, which could potentially enhance the robustness of DME detection systems. We employ five distinct GANs in this study: Deep Convolutional GAN, Conditional GAN, CycleGAN, StyleGAN2, and StyleGAN3, drawing comparisons across their performance. Subsequently, the hyperparameters of the best-performing GAN are fine-tuned using Particle Swarm Optimization (PSO) to produce more realistic OCT images. This comparative analysis not only serves to improve the detection of DME severity using OCT images but also provides insights into the appropriate choice of GANs for the effective generation of realistic OCT images from the baseline OCT datasets.
Collapse
Affiliation(s)
- Aditya Tripathi
- Department of Information & Communication Technology, Manipal Institute of Technology, Manipal, Manipal Academy of Higher Education, Manipal, Karnataka, 576104, India
| | - Preetham Kumar
- Department of Information & Communication Technology, Manipal Institute of Technology, Manipal, Manipal Academy of Higher Education, Manipal, Karnataka, 576104, India
| | - Veena Mayya
- Department of Information & Communication Technology, Manipal Institute of Technology, Manipal, Manipal Academy of Higher Education, Manipal, Karnataka, 576104, India
| | - Akshat Tulsani
- Department of Information & Communication Technology, Manipal Institute of Technology, Manipal, Manipal Academy of Higher Education, Manipal, Karnataka, 576104, India
| |
Collapse
|
28
|
Akinniyi O, Rahman MM, Sandhu HS, El-Baz A, Khalifa F. Multi-Stage Classification of Retinal OCT Using Multi-Scale Ensemble Deep Architecture. Bioengineering (Basel) 2023; 10:823. [PMID: 37508850 PMCID: PMC10376573 DOI: 10.3390/bioengineering10070823] [Citation(s) in RCA: 2] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/30/2023] [Revised: 07/01/2023] [Accepted: 07/07/2023] [Indexed: 07/30/2023] Open
Abstract
Accurate noninvasive diagnosis of retinal disorders is required for appropriate treatment or precision medicine. This work proposes a multi-stage classification network built on a multi-scale (pyramidal) feature ensemble architecture for retinal image classification using optical coherence tomography (OCT) images. First, a scale-adaptive neural network is developed to produce multi-scale inputs for feature extraction and ensemble learning. The larger input sizes yield more global information, while the smaller input sizes focus on local details. Then, a feature-rich pyramidal architecture is designed to extract multi-scale features as inputs using DenseNet as the backbone. The advantage of the hierarchical structure is that it allows the system to extract multi-scale, information-rich features for the accurate classification of retinal disorders. Evaluation on two public OCT datasets containing normal and abnormal retinas (e.g., diabetic macular edema (DME), choroidal neovascularization (CNV), age-related macular degeneration (AMD), and Drusen) and comparison against recent networks demonstrates the advantages of the proposed architecture's ability to produce feature-rich classification with average accuracy of 97.78%, 96.83%, and 94.26% for the first (binary) stage, second (three-class) stage, and all-at-once (four-class) classification, respectively, using cross-validation experiments using the first dataset. In the second dataset, our system showed an overall accuracy, sensitivity, and specificity of 99.69%, 99.71%, and 99.87%, respectively. Overall, the tangible advantages of the proposed network for enhanced feature learning might be used in various medical image classification tasks where scale-invariant features are crucial for precise diagnosis.
Collapse
Affiliation(s)
- Oluwatunmise Akinniyi
- Department of Computer Science, School of Computer, Mathematical and Natural Sciences, Morgan State University, Baltimore, MD 21251, USA
| | - Md Mahmudur Rahman
- Department of Computer Science, School of Computer, Mathematical and Natural Sciences, Morgan State University, Baltimore, MD 21251, USA
| | - Harpal Singh Sandhu
- Bioengineering Department, University of Louisville, Louisville, KY 20292, USA
| | - Ayman El-Baz
- Bioengineering Department, University of Louisville, Louisville, KY 20292, USA
| | - Fahmi Khalifa
- Electronics and Communications Engineering Department, Mansoura University, Mansoura 35516, Egypt
- Electrical and Computer Engineering Department, Morgan State University, Baltimore MD 21251, USA
| |
Collapse
|
29
|
Diao S, Su J, Yang C, Zhu W, Xiang D, Chen X, Peng Q, Shi F. Classification and segmentation of OCT images for age-related macular degeneration based on dual guidance networks. Biomed Signal Process Control 2023. [DOI: 10.1016/j.bspc.2023.104810] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 03/13/2023]
|
30
|
Huang X, Ai Z, Wang H, She C, Feng J, Wei Q, Hao B, Tao Y, Lu Y, Zeng F. GABNet: global attention block for retinal OCT disease classification. Front Neurosci 2023; 17:1143422. [PMID: 37332865 PMCID: PMC10272427 DOI: 10.3389/fnins.2023.1143422] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/13/2023] [Accepted: 04/19/2023] [Indexed: 06/20/2023] Open
Abstract
Introduction The retina represents a critical ocular structure. Of the various ophthalmic afflictions, retinal pathologies have garnered considerable scientific interest, owing to their elevated prevalence and propensity to induce blindness. Among clinical evaluation techniques employed in ophthalmology, optical coherence tomography (OCT) is the most commonly utilized, as it permits non-invasive, rapid acquisition of high-resolution, cross-sectional images of the retina. Timely detection and intervention can significantly abate the risk of blindness and effectively mitigate the national incidence rate of visual impairments. Methods This study introduces a novel, efficient global attention block (GAB) for feed forward convolutional neural networks (CNNs). The GAB generates an attention map along three dimensions (height, width, and channel) for any intermediate feature map, which it then uses to compute adaptive feature weights by multiplying it with the input feature map. This GAB is a versatile module that can seamlessly integrate with any CNN, significantly improving its classification performance. Based on the GAB, we propose a lightweight classification network model, GABNet, which we develop on a UCSD general retinal OCT dataset comprising 108,312 OCT images from 4686 patients, including choroidal neovascularization (CNV), diabetic macular edema (DME), drusen, and normal cases. Results Notably, our approach improves the classification accuracy by 3.7% over the EfficientNetV2B3 network model. We further employ gradient-weighted class activation mapping (Grad-CAM) to highlight regions of interest on retinal OCT images for each class, enabling doctors to easily interpret model predictions and improve their efficiency in evaluating relevant models. Discussion With the increasing use and application of OCT technology in the clinical diagnosis of retinal images, our approach offers an additional diagnostic tool to enhance the diagnostic efficiency of clinical OCT retinal images.
Collapse
Affiliation(s)
- Xuan Huang
- Department of Ophthalmology, Beijing Chaoyang Hospital, Capital Medical University, Beijing, China
- Medical Research Center, Beijing Chaoyang Hospital, Capital Medical University, Beijing, China
| | - Zhuang Ai
- Sinopharm Genomics Technology Co., Ltd., Changzhou, Jiangsu, China
| | - Hui Wang
- Department of Ophthalmology, Beijing Chaoyang Hospital, Capital Medical University, Beijing, China
| | - Chongyang She
- Department of Ophthalmology, Beijing Chaoyang Hospital, Capital Medical University, Beijing, China
| | - Jing Feng
- Department of Ophthalmology, Beijing Chaoyang Hospital, Capital Medical University, Beijing, China
| | - Qihao Wei
- Sinopharm Genomics Technology Co., Ltd., Changzhou, Jiangsu, China
| | - Baohai Hao
- AI-Farm (Nanjing) Big Data Services Co., Ltd., Nanjing, China
| | - Yong Tao
- Department of Ophthalmology, Beijing Chaoyang Hospital, Capital Medical University, Beijing, China
| | - Yaping Lu
- Sinopharm Genomics Technology Co., Ltd., Changzhou, Jiangsu, China
| | - Fanxin Zeng
- Department of Clinical Research Center, Dazhou Central Hospital, Dazhou, Sichuan, China
- Department of Big Data and Biomedical AI, College of Future Technology, Peking University, Beijing, China
| |
Collapse
|
31
|
He X, Wang Y, Poiesi F, Song W, Xu Q, Feng Z, Wan Y. Exploiting multi-granularity visual features for retinal layer segmentation in human eyes. Front Bioeng Biotechnol 2023; 11:1191803. [PMID: 37324431 PMCID: PMC10267414 DOI: 10.3389/fbioe.2023.1191803] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/22/2023] [Accepted: 05/22/2023] [Indexed: 06/17/2023] Open
Abstract
Accurate segmentation of retinal layer boundaries can facilitate the detection of patients with early ophthalmic disease. Typical segmentation algorithms operate at low resolutions without fully exploiting multi-granularity visual features. Moreover, several related studies do not release their datasets that are key for the research on deep learning-based solutions. We propose a novel end-to-end retinal layer segmentation network based on ConvNeXt, which can retain more feature map details by using a new depth-efficient attention module and multi-scale structures. In addition, we provide a semantic segmentation dataset containing 206 retinal images of healthy human eyes (named NR206 dataset), which is easy to use as it does not require any additional transcoding processing. We experimentally show that our segmentation approach outperforms state-of-the-art approaches on this new dataset, achieving, on average, a Dice score of 91.3% and mIoU of 84.4%. Moreover, our approach achieves state-of-the-art performance on a glaucoma dataset and a diabetic macular edema (DME) dataset, showing that our model is also suitable for other applications. We will make our source code and the NR206 dataset publicly available at (https://github.com/Medical-Image-Analysis/Retinal-layer-segmentation).
Collapse
Affiliation(s)
- Xiang He
- School of Mechanical Engineering, Shandong University, Jinan, China
- Joint SDU-NTU Centre for Artificial Intelligence Research (C-FAIR), Shandong University, Jinan, China
| | | | | | - Weiye Song
- School of Mechanical Engineering, Shandong University, Jinan, China
| | - Quanqing Xu
- School of Mechanical Engineering, Shandong University, Jinan, China
| | - Zixuan Feng
- School of Mechanical Engineering, Shandong University, Jinan, China
| | - Yi Wan
- School of Mechanical Engineering, Shandong University, Jinan, China
| |
Collapse
|
32
|
Sahoo M, Mitra M, Pal S. Improved Detection of Dry Age-Related Macular Degeneration from Optical Coherence Tomography Images using Adaptive Window Based Feature Extraction and Weighted Ensemble Based Classification Approach. Photodiagnosis Photodyn Ther 2023:103629. [PMID: 37244451 DOI: 10.1016/j.pdpdt.2023.103629] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/04/2023] [Revised: 05/20/2023] [Accepted: 05/22/2023] [Indexed: 05/29/2023]
Abstract
BACKGROUND Dry Age-related macular degeneration (AMD), which affects the older population, can lead to blindness when left untreated. Preventing vision loss in elderly needs early identification. Dry-AMD diagnosis is still time-consuming and very subjective, depending on the ophthalmologist. Setting up a thorough eye-screening system to find Dry-AMD is a very difficult task. METHODOLOGY This study aims to develop a weighted majority voting (WMV) ensemble-based prediction model to diagnose Dry-AMD. The WMV approach combines the predictions from base-classifiers and chooses the class with greatest vote based on assigned weights to each classifier. A novel feature extraction method is used along the retinal pigment epithelium (RPE) layer, with the number of windows calculated for each picture playing an important part in identifying Dry-AMD/normal images using the WMV methodology. Pre-processing using hybrid-median filter followed by scale-invariant feature transform based segmentation of RPE layer and curvature flattening of retina is employed to measure exact thickness of RPE layer. RESULT The proposed model is trained on 70% of the OCT image database (OCTID) and evaluated on remaining OCTID and SD-OCT Noor dataset. Model has achieved accuracy of 96.15% and 96.94%, respectively. The suggested algorithm's effectiveness in Dry-AMD identification is demonstrated by comparison with alternative approaches. Even though the suggested model is only trained on the OCTID, it has performed well when tested on additional dataset. CONCLUSION The suggested architecture can be used for quick eye-screening for early identification of Dry-AMD. The recommended method may be applied in real-time since it requires fewer complexity and learning-variables.
Collapse
Affiliation(s)
- Moumita Sahoo
- Department of Applied Electronics and Instrumentation Engineering, Haldia Institute of Technology, Haldia, West Bengal, India.
| | - Madhuchhanda Mitra
- Department of Applied Physics, University of Calcutta, Kolkata, West Bengal, India
| | - Saurabh Pal
- Department of Applied Physics, University of Calcutta, Kolkata, West Bengal, India
| |
Collapse
|
33
|
Feng H, Chen J, Zhang Z, Lou Y, Zhang S, Yang W. A bibliometric analysis of artificial intelligence applications in macular edema: exploring research hotspots and Frontiers. Front Cell Dev Biol 2023; 11:1174936. [PMID: 37255600 PMCID: PMC10225517 DOI: 10.3389/fcell.2023.1174936] [Citation(s) in RCA: 3] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/14/2023] [Accepted: 05/02/2023] [Indexed: 06/01/2023] Open
Abstract
Background: Artificial intelligence (AI) is used in ophthalmological disease screening and diagnostics, medical image diagnostics, and predicting late-disease progression rates. We reviewed all AI publications associated with macular edema (ME) research Between 2011 and 2022 and performed modeling, quantitative, and qualitative investigations. Methods: On 1st February 2023, we screened the Web of Science Core Collection for AI applications related to ME, from which 297 studies were identified and analyzed (2011-2022). We collected information on: publications, institutions, country/region, keywords, journal name, references, and research hotspots. Literature clustering networks and Frontier knowledge bases were investigated using bibliometrix-BiblioShiny, VOSviewer, and CiteSpace bibliometric platforms. We used the R "bibliometrix" package to synopsize our observations, enumerate keywords, visualize collaboration networks between countries/regions, and generate a topic trends plot. VOSviewer was used to examine cooperation between institutions and identify citation relationships between journals. We used CiteSpace to identify clustering keywords over the timeline and identify keywords with the strongest citation bursts. Results: In total, 47 countries published AI studies related to ME; the United States had the highest H-index, thus the greatest influence. China and the United States cooperated most closely between all countries. Also, 613 institutions generated publications - the Medical University of Vienna had the highest number of studies. This publication record and H-index meant the university was the most influential in the ME field. Reference clusters were also categorized into 10 headings: retinal Optical Coherence Tomography (OCT) fluid detection, convolutional network models, deep learning (DL)-based single-shot predictions, retinal vascular disease, diabetic retinopathy (DR), convolutional neural networks (CNNs), automated macular pathology diagnosis, dry age-related macular degeneration (DARMD), class weight, and advanced DL architecture systems. Frontier keywords were represented by diabetic macular edema (DME) (2021-2022). Conclusion: Our review of the AI-related ME literature was comprehensive, systematic, and objective, and identified future trends and current hotspots. With increased DL outputs, the ME research focus has gradually shifted from manual ME examinations to automatic ME detection and associated symptoms. In this review, we present a comprehensive and dynamic overview of AI in ME and identify future research areas.
Collapse
Affiliation(s)
- Haiwen Feng
- Department of Software Engineering, School of Software, Shenyang University of Technology, Shenyang, Liaoning, China
| | - Jiaqi Chen
- Department of Software Engineering, School of Software, Shenyang University of Technology, Shenyang, Liaoning, China
| | - Zhichang Zhang
- Department of Computer, School of Intelligent Medicine, China Medical University, Shenyang, Liaoning, China
| | - Yan Lou
- Department of Computer, School of Intelligent Medicine, China Medical University, Shenyang, Liaoning, China
| | - Shaochong Zhang
- Shenzhen Eye Institute, Shenzhen Eye Hospital, Jinan University, Shenzhen, China
| | - Weihua Yang
- Shenzhen Eye Institute, Shenzhen Eye Hospital, Jinan University, Shenzhen, China
| |
Collapse
|
34
|
Rasti R, Biglari A, Rezapourian M, Yang Z, Farsiu S. RetiFluidNet: A Self-Adaptive and Multi-Attention Deep Convolutional Network for Retinal OCT Fluid Segmentation. IEEE TRANSACTIONS ON MEDICAL IMAGING 2023; 42:1413-1423. [PMID: 37015695 DOI: 10.1109/tmi.2022.3228285] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 05/04/2023]
Abstract
Optical coherence tomography (OCT) helps ophthalmologists assess macular edema, accumulation of fluids, and lesions at microscopic resolution. Quantification of retinal fluids is necessary for OCT-guided treatment management, which relies on a precise image segmentation step. As manual analysis of retinal fluids is a time-consuming, subjective, and error-prone task, there is increasing demand for fast and robust automatic solutions. In this study, a new convolutional neural architecture named RetiFluidNet is proposed for multi-class retinal fluid segmentation. The model benefits from hierarchical representation learning of textural, contextual, and edge features using a new self-adaptive dual-attention (SDA) module, multiple self-adaptive attention-based skip connections (SASC), and a novel multi-scale deep self-supervision learning (DSL) scheme. The attention mechanism in the proposed SDA module enables the model to automatically extract deformation-aware representations at different levels, and the introduced SASC paths further consider spatial-channel interdependencies for concatenation of counterpart encoder and decoder units, which improve representational capability. RetiFluidNet is also optimized using a joint loss function comprising a weighted version of dice overlap and edge-preserved connectivity-based losses, where several hierarchical stages of multi-scale local losses are integrated into the optimization process. The model is validated based on three publicly available datasets: RETOUCH, OPTIMA, and DUKE, with comparisons against several baselines. Experimental results on the datasets prove the effectiveness of the proposed model in retinal OCT fluid segmentation and reveal that the suggested method is more effective than existing state-of-the-art fluid segmentation algorithms in adapting to retinal OCT scans recorded by various image scanning instruments.
Collapse
|
35
|
Zhang Y, Li Z, Nan N, Wang X. TranSegNet: Hybrid CNN-Vision Transformers Encoder for Retina Segmentation of Optical Coherence Tomography. Life (Basel) 2023; 13:life13040976. [PMID: 37109505 PMCID: PMC10146870 DOI: 10.3390/life13040976] [Citation(s) in RCA: 4] [Impact Index Per Article: 4.0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/15/2023] [Revised: 04/04/2023] [Accepted: 04/06/2023] [Indexed: 04/29/2023] Open
Abstract
Optical coherence tomography (OCT) provides unique advantages in ophthalmic examinations owing to its noncontact, high-resolution, and noninvasive features, which have evolved into one of the most crucial modalities for identifying and evaluating retinal abnormalities. Segmentation of laminar structures and lesion tissues in retinal OCT images can provide quantitative information on retinal morphology and reliable guidance for clinical diagnosis and treatment. Convolutional neural networks (CNNs) have achieved success in various medical image segmentation tasks. However, the receptive field of convolution has inherent locality constraints, resulting in limitations of mainstream frameworks based on CNNs, which is still evident in recognizing the morphological changes of retina OCT. In this study, we proposed an end-to-end network, TranSegNet, which incorporates a hybrid encoder that combines the advantages of a lightweight vision transformer (ViT) and the U-shaped network. The CNN features under multiscale resolution are extracted based on the improved U-net backbone, and a ViT with the multi-head convolutional attention is introduced to capture the feature information in a global view, realizing accurate localization and segmentation of retinal layers and lesion tissues. The experimental results illustrate that hybrid CNN-ViT is a strong encoder for retinal OCT image segmentation tasks and the lightweight design reduces its parameter size and computational complexity while maintaining its outstanding performance. By applying TranSegNet to healthy and diseased retinal OCT datasets separately, TranSegNet demonstrated superior efficiency, accuracy, and robustness in the segmentation results of retinal layers and accumulated fluid than the four advanced segmentation methods, such as FCN, SegNet, Unet and TransUnet.
Collapse
Affiliation(s)
- Yiheng Zhang
- Laboratory of Information Optics and Opto-Electronic Technology, Shanghai Institute of Optics and Fine Mechanics, Chinese Academy of Sciences, Shanghai 201800, China
- Center of Materials Science and Optoelectronics Engineering, University of Chinese Academy of Sciences, Beijing 100049, China
| | - Zhongliang Li
- Laboratory of Information Optics and Opto-Electronic Technology, Shanghai Institute of Optics and Fine Mechanics, Chinese Academy of Sciences, Shanghai 201800, China
- Center of Materials Science and Optoelectronics Engineering, University of Chinese Academy of Sciences, Beijing 100049, China
| | - Nan Nan
- Laboratory of Information Optics and Opto-Electronic Technology, Shanghai Institute of Optics and Fine Mechanics, Chinese Academy of Sciences, Shanghai 201800, China
| | - Xiangzhao Wang
- Laboratory of Information Optics and Opto-Electronic Technology, Shanghai Institute of Optics and Fine Mechanics, Chinese Academy of Sciences, Shanghai 201800, China
- Center of Materials Science and Optoelectronics Engineering, University of Chinese Academy of Sciences, Beijing 100049, China
| |
Collapse
|
36
|
Lake SR, Bottema MJ, Lange T, Williams KA, Reynolds KJ. Swept-Source OCT Mid-Peripheral Retinal Irregularity in Retinal Detachment and Posterior Vitreous Detachment Eyes. Bioengineering (Basel) 2023; 10:bioengineering10030377. [PMID: 36978768 PMCID: PMC10044997 DOI: 10.3390/bioengineering10030377] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/10/2023] [Revised: 03/10/2023] [Accepted: 03/16/2023] [Indexed: 03/30/2023] Open
Abstract
Irregularities in retinal shape have been shown to correlate with axial length, a major risk factor for retinal detachment. To further investigate this association, a comparison was performed of the swept-source optical coherence tomography (SS OCT) peripheral retinal shape of eyes that had either a posterior vitreous detachment (PVD) or vitrectomy for retinal detachment. The objective was to identify a biomarker that can be tested as a predictor for retinal detachment. Eyes with a PVD (N = 88), treated retinal detachment (N = 67), or retinal tear (N = 53) were recruited between July 2020 and January 2022 from hospital retinal clinics in South Australia. The mid-peripheral retina was imaged in four quadrants with SS OCT. The features explored were patient age, eye axial length, and retinal shape irregularity quantified in the frequency domain. A discriminant analysis classifier to identify retinal detachment eyes was trained with two-thirds and tested with one-third of the sample. Retinal detachment eyes had greater irregularity than PVD eyes. A classifier trained using shape features from the superior and temporal retina had a specificity of 84% and a sensitivity of 48%. Models incorporating axial length were less successful, suggesting peripheral retinal irregularity is a better biomarker for retinal detachment than axial length. Mid-peripheral retinal irregularity can identify eyes that have experienced a retinal detachment.
Collapse
Affiliation(s)
- Stewart R Lake
- Flinders Institute for Health and Medical Research, GPO Box 2100, Adelaide 5001, Australia
- Medical Device Research Institute, College of Science and Engineering, Flinders University, GPO Box 2100, Adelaide 5001, Australia
| | - Murk J Bottema
- Medical Device Research Institute, College of Science and Engineering, Flinders University, GPO Box 2100, Adelaide 5001, Australia
| | - Tyra Lange
- Medical Device Research Institute, College of Science and Engineering, Flinders University, GPO Box 2100, Adelaide 5001, Australia
| | - Keryn A Williams
- Flinders Institute for Health and Medical Research, GPO Box 2100, Adelaide 5001, Australia
| | - Karen J Reynolds
- Medical Device Research Institute, College of Science and Engineering, Flinders University, GPO Box 2100, Adelaide 5001, Australia
| |
Collapse
|
37
|
Kayadibi İ, Güraksın GE. An Explainable Fully Dense Fusion Neural Network with Deep Support Vector Machine for Retinal Disease Determination. INT J COMPUT INT SYS 2023. [DOI: 10.1007/s44196-023-00210-z] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 03/11/2023] Open
Abstract
AbstractRetinal issues are crucial because they result in visual loss. Early diagnosis can aid physicians in initiating treatment and preventing visual loss. Optical coherence tomography (OCT), which portrays retinal morphology cross-sectionally and noninvasively, is used to identify retinal abnormalities. The process of analyzing OCT images, on the other hand, takes time. This study has proposed a hybrid approach based on a fully dense fusion neural network (FD-CNN) and dual preprocessing to identify retinal diseases, such as choroidal neovascularization, diabetic macular edema, drusen from OCT images. A dual preprocessing methodology, in other words, a hybrid speckle reduction filter was initially used to diminish speckle noise present in OCT images. Secondly, the FD-CNN architecture was trained, and the features obtained from this architecture were extracted. Then Deep Support Vector Machine (D-SVM) and Deep K-Nearest Neighbor (D-KNN) classifiers were proposed to reclassify those features and tested on University of California San Diego (UCSD) and Duke OCT datasets. D-SVM demonstrated the best performance in both datasets. D-SVM achieved 99.60% accuracy, 99.60% sensitivity, 99.87% specificity, 99.60% precision and 99.60% F1 score in the UCSD dataset. It achieved 97.50% accuracy, 97.64% sensitivity, 98.91% specificity, 96.61% precision, and 97.03% F1 score in Duke dataset. Additionally, the results were compared to state-of-the-art works on the both datasets. The D-SVM was demonstrated to be an efficient and productive strategy for improving the robustness of automatic retinal disease classification. Also, in this study, it is shown that the unboxing of how AI systems' black-box choices is made by generating heat maps using the local interpretable model-agnostic explanation method, which is an explainable artificial intelligence (XAI) technique. Heat maps, in particular, may contribute to the development of more stable deep learning-based systems, as well as enhancing the confidence in the diagnosis of retinal disease in the analysis of OCT image for ophthalmologists.
Collapse
|
38
|
Liew A, Agaian S, Benbelkacem S. Distinctions between Choroidal Neovascularization and Age Macular Degeneration in Ocular Disease Predictions via Multi-Size Kernels ξcho-Weighted Median Patterns. Diagnostics (Basel) 2023; 13:diagnostics13040729. [PMID: 36832215 PMCID: PMC9956029 DOI: 10.3390/diagnostics13040729] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/28/2022] [Revised: 12/31/2022] [Accepted: 01/05/2023] [Indexed: 02/17/2023] Open
Abstract
Age-related macular degeneration is a visual disorder caused by abnormalities in a part of the eye's retina and is a leading source of blindness. The correct detection, precise location, classification, and diagnosis of choroidal neovascularization (CNV) may be challenging if the lesion is small or if Optical Coherence Tomography (OCT) images are degraded by projection and motion. This paper aims to develop an automated quantification and classification system for CNV in neovascular age-related macular degeneration using OCT angiography images. OCT angiography is a non-invasive imaging tool that visualizes retinal and choroidal physiological and pathological vascularization. The presented system is based on new retinal layers in the OCT image-specific macular diseases feature extractor, including Multi-Size Kernels ξcho-Weighted Median Patterns (MSKξMP). Computer simulations show that the proposed method: (i) outperforms current state-of-the-art methods, including deep learning techniques; and (ii) achieves an overall accuracy of 99% using ten-fold cross-validation on the Duke University dataset and over 96% on the noisy Noor Eye Hospital dataset. In addition, MSKξMP performs well in binary eye disease classifications and is more accurate than recent works in image texture descriptors.
Collapse
Affiliation(s)
- Alex Liew
- Department of Computer Science, Graduate Center of City University New York, 365 5th Ave., New York, NY 10016, USA
- Correspondence:
| | - Sos Agaian
- Department of Computer Science, Graduate Center of City University New York, 365 5th Ave., New York, NY 10016, USA
| | - Samir Benbelkacem
- Robotics and Industrial Automation Division, Centre de Développement des Technologies Avancées (CDTA), Algiers 16081, Algeria
| |
Collapse
|
39
|
Gan F, Wu FP, Zhong YL. Artificial intelligence method based on multi-feature fusion for automatic macular edema (ME) classification on spectral-domain optical coherence tomography (SD-OCT) images. Front Neurosci 2023; 17:1097291. [PMID: 36793539 PMCID: PMC9922866 DOI: 10.3389/fnins.2023.1097291] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/13/2022] [Accepted: 01/09/2023] [Indexed: 01/31/2023] Open
Abstract
Purpose A common ocular manifestation, macular edema (ME) is the primary cause of visual deterioration. In this study, an artificial intelligence method based on multi-feature fusion was introduced to enable automatic ME classification on spectral-domain optical coherence tomography (SD-OCT) images, to provide a convenient method of clinical diagnosis. Methods First, 1,213 two-dimensional (2D) cross-sectional OCT images of ME were collected from the Jiangxi Provincial People's Hospital between 2016 and 2021. According to OCT reports of senior ophthalmologists, there were 300 images with diabetic (DME), 303 images with age-related macular degeneration (AMD), 304 images with retinal-vein occlusion (RVO), and 306 images with central serous chorioretinopathy (CSC). Then, traditional omics features of the images were extracted based on the first-order statistics, shape, size, and texture. After extraction by the alexnet, inception_v3, resnet34, and vgg13 models and selected by dimensionality reduction using principal components analysis (PCA), the deep-learning features were fused. Next, the gradient-weighted class-activation map (Grad-CAM) was used to visualize the-deep-learning process. Finally, the fusion features set, which was fused from the traditional omics features and the deep-fusion features, was used to establish the final classification models. The performance of the final models was evaluated by accuracy, confusion matrix, and the receiver operating characteristic (ROC) curve. Results Compared with other classification models, the performance of the support vector machine (SVM) model was best, with an accuracy of 93.8%. The area under curves AUC of micro- and macro-averages were 99%, and the AUC of the AMD, DME, RVO, and CSC groups were 100, 99, 98, and 100%, respectively. Conclusion The artificial intelligence model in this study could be used to classify DME, AME, RVO, and CSC accurately from SD-OCT images.
Collapse
Affiliation(s)
- Fan Gan
- Medical College of Nanchang University, Nanchang, China,Department of Ophthalmology, Jiangxi Provincial People’s Hospital, The First Affiliated Hospital of Nanchang Medical College, Nanchang, China
| | - Fei-Peng Wu
- Department of Ophthalmology, Jiangxi Provincial People’s Hospital, The First Affiliated Hospital of Nanchang Medical College, Nanchang, China
| | - Yu-Lin Zhong
- Department of Ophthalmology, Jiangxi Provincial People’s Hospital, The First Affiliated Hospital of Nanchang Medical College, Nanchang, China,*Correspondence: Yu-Lin Zhong,
| |
Collapse
|
40
|
Classification of Retinal Diseases in Optical Coherence Tomography Images Using Artificial Intelligence and Firefly Algorithm. Diagnostics (Basel) 2023; 13:diagnostics13030433. [PMID: 36766537 PMCID: PMC9914873 DOI: 10.3390/diagnostics13030433] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/30/2022] [Revised: 01/20/2023] [Accepted: 01/24/2023] [Indexed: 01/27/2023] Open
Abstract
In recent years, the number of studies for the automatic diagnosis of biomedical diseases has increased. Many of these studies have used Deep Learning, which gives extremely good results but requires a vast amount of data and computing load. If the processor is of insufficient quality, this takes time and places an excessive load on the processor. On the other hand, Machine Learning is faster than Deep Learning and does not have a much-needed computing load, but it does not provide as high an accuracy value as Deep Learning. Therefore, our goal is to develop a hybrid system that provides a high accuracy value, while requiring a smaller computing load and less time to diagnose biomedical diseases such as the retinal diseases we chose for this study. For this purpose, first, retinal layer extraction was conducted through image preprocessing. Then, traditional feature extractors were combined with pre-trained Deep Learning feature extractors. To select the best features, we used the Firefly algorithm. In the end, multiple binary classifications were conducted instead of multiclass classification with Machine Learning classifiers. Two public datasets were used in this study. The first dataset had a mean accuracy of 0.957, and the second dataset had a mean accuracy of 0.954.
Collapse
|
41
|
Choudhary A, Ahlawat S, Urooj S, Pathak N, Lay-Ekuakille A, Sharma N. A Deep Learning-Based Framework for Retinal Disease Classification. Healthcare (Basel) 2023; 11:healthcare11020212. [PMID: 36673578 PMCID: PMC9859538 DOI: 10.3390/healthcare11020212] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/21/2022] [Revised: 12/23/2022] [Accepted: 12/29/2022] [Indexed: 01/12/2023] Open
Abstract
This study addresses the problem of the automatic detection of disease states of the retina. In order to solve the abovementioned problem, this study develops an artificially intelligent model. The model is based on a customized 19-layer deep convolutional neural network called VGG-19 architecture. The model (VGG-19 architecture) is empowered by transfer learning. The model is designed so that it can learn from a large set of images taken with optical coherence tomography (OCT) and classify them into four conditions of the retina: (1) choroidal neovascularization, (2) drusen, (3) diabetic macular edema, and (4) normal form. The training datasets (taken from publicly available sources) consist of 84,568 instances of OCT retinal images. The datasets exhibit all four classes of retinal disease mentioned above. The proposed model achieved a 99.17% classification accuracy with 0.995 specificities and 0.99 sensitivity, making it better than the existing models. In addition, the proper statistical evaluation is done on the predictions using such performance measures as (1) area under the receiver operating characteristic curve, (2) Cohen's kappa parameter, and (3) confusion matrix. Experimental results show that the proposed VGG-19 architecture coupled with transfer learning is an effective technique for automatically detecting the disease state of a retina.
Collapse
Affiliation(s)
- Amit Choudhary
- University School of Automation and Robotics, G.G.S. Indraprastha University, New Delhi 110092, India
| | - Savita Ahlawat
- Maharaja Surajmal Institute of Technology, G.G.S. Indraprastha University, New Delhi 110058, India
- Correspondence: (S.A.); (S.U.)
| | - Shabana Urooj
- Department of Electrical Engineering, College of Engineering, Princess Nourah Bint Abdulrahman University, P.O. Box 84428, Riyadh 11671, Saudi Arabia
- Correspondence: (S.A.); (S.U.)
| | - Nitish Pathak
- Department of Information Technology, Bhagwan Parshuram Institute of Technology (BPIT), G.G.S. Indraprastha University, New Delhi 110078, India
| | - Aimé Lay-Ekuakille
- Department of Innovation Engineering, University of Salento, 73100 Lecce, Italy
| | - Neelam Sharma
- Department of Artificial Intelligence and Machine Learning, Maharaja Agrasen Institute of Technology (MAIT), G.G.S. Indraprastha University, New Delhi 110086, India
| |
Collapse
|
42
|
Shen J, Hu Y, Zhang X, Gong Y, Kawasaki R, Liu J. Structure-Oriented Transformer for retinal diseases grading from OCT images. Comput Biol Med 2023; 152:106445. [PMID: 36549031 DOI: 10.1016/j.compbiomed.2022.106445] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/28/2022] [Revised: 11/23/2022] [Accepted: 12/15/2022] [Indexed: 12/23/2022]
Abstract
Retinal diseases are the leading causes of vision temporary or permanent loss. Precise retinal disease grading is a prerequisite for early intervention or specific therapeutic schedules. Existing works based on Convolutional Neural Networks (CNN) focus on typical locality structures and cannot capture long-range dependencies. But retinal disease grading relies more on the relationship between the local lesion and the whole retina, which is consistent with the self-attention mechanism. Therefore, the paper proposes a novel Structure-Oriented Transformer (SoT) framework to further construct the relationship between lesions and retina on clinical datasets. To reduce the dependence on the amount of data, we design structure guidance as a model-oriented filter to emphasize the whole retina structure and guide relation construction. Then, we adopt the pre-trained vision transformer that efficiently models all feature patches' relationships via transfer learning. Besides, to make the best of all output tokens, a Token vote classifier is proposed to obtain the final grading results. We conduct extensive experiments on one clinical neovascular Age-related Macular Degeneration (nAMD) dataset. The experiments demonstrate the effectiveness of SoT components and improve the ability of relation construction between lesion and retina, which outperforms the state-of-the-art methods for nAMD grading. Furthermore, we evaluate our SoT on one publicly available retinal diseases dataset, which proves our algorithm has classification superiority and good generality.
Collapse
Affiliation(s)
- Junyong Shen
- Research Institute of Trustworthy Autonomous Systems and Department of Computer Science and Engineering, Southern University of Science and Technology, Shenzhen, 51805, Guangdong, China
| | - Yan Hu
- Research Institute of Trustworthy Autonomous Systems and Department of Computer Science and Engineering, Southern University of Science and Technology, Shenzhen, 51805, Guangdong, China.
| | - Xiaoqing Zhang
- Research Institute of Trustworthy Autonomous Systems and Department of Computer Science and Engineering, Southern University of Science and Technology, Shenzhen, 51805, Guangdong, China
| | - Yan Gong
- Ningbo Eye hospital, Ningbo, 315000, Zhenjiang, China
| | - Ryo Kawasaki
- Osaka University Graduate School of Medicine, Suita, Osaka, Japan
| | - Jiang Liu
- Research Institute of Trustworthy Autonomous Systems and Department of Computer Science and Engineering, Southern University of Science and Technology, Shenzhen, 51805, Guangdong, China; Guangdong Provincial Key Laboratory of Brain-inspired Intelligent Computation, Department of Computer Science and Engineering, Southern University of Science and Technology, Shenzhen, 51805, Guangdong, China.
| |
Collapse
|
43
|
Feng D, Chen X, Wang X, Mou X, Bai L, Zhang S, Zhou Z. Predicting effectiveness of anti-VEGF injection through self-supervised learning in OCT images. MATHEMATICAL BIOSCIENCES AND ENGINEERING : MBE 2023; 20:2439-2458. [PMID: 36899541 DOI: 10.3934/mbe.2023114] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/18/2023]
Abstract
Anti-vascular endothelial growth factor (Anti-VEGF) therapy has become a standard way for choroidal neovascularization (CNV) and cystoid macular edema (CME) treatment. However, anti-VEGF injection is a long-term therapy with expensive cost and may be not effective for some patients. Therefore, predicting the effectiveness of anti-VEGF injection before the therapy is necessary. In this study, a new optical coherence tomography (OCT) images based self-supervised learning (OCT-SSL) model for predicting the effectiveness of anti-VEGF injection is developed. In OCT-SSL, we pre-train a deep encoder-decoder network through self-supervised learning to learn the general features using a public OCT image dataset. Then, model fine-tuning is performed on our own OCT dataset to learn the discriminative features to predict the effectiveness of anti-VEGF. Finally, classifier trained by the features from fine-tuned encoder as a feature extractor is built to predict the response. Experimental results on our private OCT dataset demonstrated that the proposed OCT-SSL can achieve an average accuracy, area under the curve (AUC), sensitivity and specificity of 0.93, 0.98, 0.94 and 0.91, respectively. Meanwhile, it is found that not only the lesion region but also the normal region in OCT image is related to the effectiveness of anti-VEGF.
Collapse
Affiliation(s)
- Dehua Feng
- School of Information and Communications Engineering, Xi'an Jiaotong University, Shaanxi 710049, China
| | - Xi Chen
- School of Information and Communications Engineering, Xi'an Jiaotong University, Shaanxi 710049, China
| | - Xiaoyu Wang
- School of Information and Communications Engineering, Xi'an Jiaotong University, Shaanxi 710049, China
| | - Xuanqin Mou
- School of Information and Communications Engineering, Xi'an Jiaotong University, Shaanxi 710049, China
| | - Ling Bai
- Department of Ophthalmology, the Second Affiliated Hospital of Xi'an Jiaotong University, Shaanxi 710004, China
| | - Shu Zhang
- Department of Geriatric Surgery, the Second Affiliated Hospital of Xi'an Jiaotong University, Shaanxi 710004, China
| | - Zhiguo Zhou
- Department of Biostatistics and Data Science, University of Kansas Medical Center, KS 66160, USA
| |
Collapse
|
44
|
Wang X, Tang F, Chen H, Cheung CY, Heng PA. Deep semi-supervised multiple instance learning with self-correction for DME classification from OCT images. Med Image Anal 2023; 83:102673. [PMID: 36403310 DOI: 10.1016/j.media.2022.102673] [Citation(s) in RCA: 3] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/06/2021] [Revised: 07/03/2022] [Accepted: 10/20/2022] [Indexed: 11/18/2022]
Abstract
Supervised deep learning has achieved prominent success in various diabetic macular edema (DME) recognition tasks from optical coherence tomography (OCT) volumetric images. A common problematic issue that frequently occurs in this field is the shortage of labeled data due to the expensive fine-grained annotations, which increases substantial difficulty in accurate analysis by supervised learning. The morphological changes in the retina caused by DME might be distributed sparsely in B-scan images of the OCT volume, and OCT data is often coarsely labeled at the volume level. Hence, the DME identification task can be formulated as a multiple instance classification problem that could be addressed by multiple instance learning (MIL) techniques. Nevertheless, none of previous studies utilize unlabeled data simultaneously to promote the classification accuracy, which is particularly significant for a high quality of analysis at the minimum annotation cost. To this end, we present a novel deep semi-supervised multiple instance learning framework to explore the feasibility of leveraging a small amount of coarsely labeled data and a large amount of unlabeled data to tackle this problem. Specifically, we come up with several modules to further improve the performance according to the availability and granularity of their labels. To warm up the training, we propagate the bag labels to the corresponding instances as the supervision of training, and propose a self-correction strategy to handle the label noise in the positive bags. This strategy is based on confidence-based pseudo-labeling with consistency regularization. The model uses its prediction to generate the pseudo-label for each weakly augmented input only if it is highly confident about the prediction, which is subsequently used to supervise the same input in a strongly augmented version. This learning scheme is also applicable to unlabeled data. To enhance the discrimination capability of the model, we introduce the Student-Teacher architecture and impose consistency constraints between two models. For demonstration, the proposed approach was evaluated on two large-scale DME OCT image datasets. Extensive results indicate that the proposed method improves DME classification with the incorporation of unlabeled data and outperforms competing MIL methods significantly, which confirm the feasibility of deep semi-supervised multiple instance learning at a low annotation cost.
Collapse
Affiliation(s)
- Xi Wang
- Zhejiang Lab, Hangzhou, China; Department of Computer Science and Engineering, The Chinese University of Hong Kong, Hong Kong, China; Department of Radiation Oncology, Stanford University School of Medicine, Palo Alto, CA, USA
| | - Fangyao Tang
- Department of Ophthalmology and Visual Sciences, The Chinese University of Hong Kong, Hong Kong, China
| | - Hao Chen
- Department of Computer Science and Engineering, The Hong Kong University of Science and Technology, Hong Kong, China.
| | - Carol Y Cheung
- Department of Ophthalmology and Visual Sciences, The Chinese University of Hong Kong, Hong Kong, China
| | - Pheng-Ann Heng
- Department of Computer Science and Engineering, The Chinese University of Hong Kong, Hong Kong, China; Guangdong-Hong Kong-Macao Joint Laboratory of Human-Machine Intelligence-Synergy Systems, Shenzhen Institutes of Advanced Technology, Chinese Academy of Sciences, China
| |
Collapse
|
45
|
Pavithra K, Kumar P, Geetha M, Bhandary SV. Computer aided diagnosis of diabetic macular edema in retinal fundus and OCT images: A review. Biocybern Biomed Eng 2023. [DOI: 10.1016/j.bbe.2022.12.005] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/04/2023]
|
46
|
Selvachandran G, Quek SG, Paramesran R, Ding W, Son LH. Developments in the detection of diabetic retinopathy: a state-of-the-art review of computer-aided diagnosis and machine learning methods. Artif Intell Rev 2023; 56:915-964. [PMID: 35498558 PMCID: PMC9038999 DOI: 10.1007/s10462-022-10185-6] [Citation(s) in RCA: 7] [Impact Index Per Article: 7.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Accepted: 04/04/2022] [Indexed: 02/02/2023]
Abstract
The exponential increase in the number of diabetics around the world has led to an equally large increase in the number of diabetic retinopathy (DR) cases which is one of the major complications caused by diabetes. Left unattended, DR worsens the vision and would lead to partial or complete blindness. As the number of diabetics continue to increase exponentially in the coming years, the number of qualified ophthalmologists need to increase in tandem in order to meet the demand for screening of the growing number of diabetic patients. This makes it pertinent to develop ways to automate the detection process of DR. A computer aided diagnosis system has the potential to significantly reduce the burden currently placed on the ophthalmologists. Hence, this review paper is presented with the aim of summarizing, classifying, and analyzing all the recent development on automated DR detection using fundus images from 2015 up to this date. Such work offers an unprecedentedly thorough review of all the recent works on DR, which will potentially increase the understanding of all the recent studies on automated DR detection, particularly on those that deploys machine learning algorithms. Firstly, in this paper, a comprehensive state-of-the-art review of the methods that have been introduced in the detection of DR is presented, with a focus on machine learning models such as convolutional neural networks (CNN) and artificial neural networks (ANN) and various hybrid models. Each AI will then be classified according to its type (e.g. CNN, ANN, SVM), its specific task(s) in performing DR detection. In particular, the models that deploy CNN will be further analyzed and classified according to some important properties of the respective CNN architectures of each model. A total of 150 research articles related to the aforementioned areas that were published in the recent 5 years have been utilized in this review to provide a comprehensive overview of the latest developments in the detection of DR. Supplementary Information The online version contains supplementary material available at 10.1007/s10462-022-10185-6.
Collapse
Affiliation(s)
- Ganeshsree Selvachandran
- Department of Actuarial Science and Applied Statistics, Faculty of Business & Management, UCSI University, Jalan Menara Gading, Cheras, 56000 Kuala Lumpur, Malaysia
| | - Shio Gai Quek
- Department of Actuarial Science and Applied Statistics, Faculty of Business & Management, UCSI University, Jalan Menara Gading, Cheras, 56000 Kuala Lumpur, Malaysia
| | - Raveendran Paramesran
- Institute of Computer Science and Digital Innovation, UCSI University, Jalan Menara Gading, Cheras, 56000 Kuala Lumpur, Malaysia
| | - Weiping Ding
- School of Information Science and Technology, Nantong University, Nantong, 226019 People’s Republic of China
| | - Le Hoang Son
- VNU Information Technology Institute, Vietnam National University, Hanoi, Vietnam
| |
Collapse
|
47
|
Medhi JP, S.R. N, Choudhury S, Dandapat S. Improved detection and analysis of Macular Edema using modified guided image filtering with modified level set spatial fuzzy clustering on Optical Coherence Tomography images. Biomed Signal Process Control 2023. [DOI: 10.1016/j.bspc.2022.104149] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/14/2022]
|
48
|
Celebi ARC, Bulut E, Sezer A. Artificial intelligence based detection of age-related macular degeneration using optical coherence tomography with unique image preprocessing. Eur J Ophthalmol 2023; 33:65-73. [PMID: 35469472 DOI: 10.1177/11206721221096294] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/11/2023]
Abstract
PURPOSE The aim of the study is to improve the accuracy of age related macular degeneration (AMD) disease in its earlier phases with proposed Capsule Network (CapsNet) architecture trained on speckle noise reduced spectral domain optical coherence tomography (SD-OCT) images based on an optimized Bayesian non-local mean (OBNLM) filter augmentation techniques. METHODS A total of 726 local SD-OCT images were collected and labelled as 159 drusen, 145 dry AMD, 156 wet AMD and 266 normal. Region of interest (ROI) was identified. Speckle noise in SD-OCT images were reduced based on OBNLM filter. The processed images were fed to proposed CapsNet architecture to clasify SD-OCT images. Accuracy rates were calculated in both public and local dataset. RESULTS Accuracy rate of local SD-OCT image dataset classification was achieved to a value of 96.39% after performing data augmentation and speckle noise reduction with OBNLM. The performance of proposed CapsNet was also evaluated on the public Kaggle dataset under the same processing procedures and the accuracy rate was calculated as 98.07%. The sensitivity and specificity rates were 96.72% and 99.98%, respectively. CONCLUSIONS The classification success of proposed CapsNet may be improved with robust pre-processing steps like; determination of ROI and denoised SD-OCT images based on OBNLM. These impactful image preprocessing steps yielded higher accuracy rates for determining different types of AMD including its precursor lesion on the both local and public dataset with proposed CapsNet architecture.
Collapse
Affiliation(s)
- Ali Riza Cenk Celebi
- Department of Ophthalmology, Acibadem University School of Medicine, Istanbul, Turkey
| | - Erkan Bulut
- Department of Ophthalmology, Beylikduzu Public Hospital, Istanbul, Turkey
| | - Aysun Sezer
- United'Informatique et d'Ingenierie des Systemes, 52849ENSTA-ParisTech, Universite de Paris-Saclay, Villefranche Sur Mer, Provence-Alpes-Côte d'azur, France
| |
Collapse
|
49
|
Anton N, Doroftei B, Curteanu S, Catãlin L, Ilie OD, Târcoveanu F, Bogdănici CM. Comprehensive Review on the Use of Artificial Intelligence in Ophthalmology and Future Research Directions. Diagnostics (Basel) 2022; 13:100. [PMID: 36611392 PMCID: PMC9818832 DOI: 10.3390/diagnostics13010100] [Citation(s) in RCA: 8] [Impact Index Per Article: 4.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/03/2022] [Revised: 12/12/2022] [Accepted: 12/26/2022] [Indexed: 12/31/2022] Open
Abstract
BACKGROUND Having several applications in medicine, and in ophthalmology in particular, artificial intelligence (AI) tools have been used to detect visual function deficits, thus playing a key role in diagnosing eye diseases and in predicting the evolution of these common and disabling diseases. AI tools, i.e., artificial neural networks (ANNs), are progressively involved in detecting and customized control of ophthalmic diseases. The studies that refer to the efficiency of AI in medicine and especially in ophthalmology were analyzed in this review. MATERIALS AND METHODS We conducted a comprehensive review in order to collect all accounts published between 2015 and 2022 that refer to these applications of AI in medicine and especially in ophthalmology. Neural networks have a major role in establishing the demand to initiate preliminary anti-glaucoma therapy to stop the advance of the disease. RESULTS Different surveys in the literature review show the remarkable benefit of these AI tools in ophthalmology in evaluating the visual field, optic nerve, and retinal nerve fiber layer, thus ensuring a higher precision in detecting advances in glaucoma and retinal shifts in diabetes. We thus identified 1762 applications of artificial intelligence in ophthalmology: review articles and research articles (301 pub med, 144 scopus, 445 web of science, 872 science direct). Of these, we analyzed 70 articles and review papers (diabetic retinopathy (N = 24), glaucoma (N = 24), DMLV (N = 15), other pathologies (N = 7)) after applying the inclusion and exclusion criteria. CONCLUSION In medicine, AI tools are used in surgery, radiology, gynecology, oncology, etc., in making a diagnosis, predicting the evolution of a disease, and assessing the prognosis in patients with oncological pathologies. In ophthalmology, AI potentially increases the patient's access to screening/clinical diagnosis and decreases healthcare costs, mainly when there is a high risk of disease or communities face financial shortages. AI/DL (deep learning) algorithms using both OCT and FO images will change image analysis techniques and methodologies. Optimizing these (combined) technologies will accelerate progress in this area.
Collapse
Affiliation(s)
- Nicoleta Anton
- Faculty of Medicine, University of Medicine and Pharmacy “Grigore T. Popa”, University Street, No 16, 700115 Iasi, Romania
| | - Bogdan Doroftei
- Faculty of Medicine, University of Medicine and Pharmacy “Grigore T. Popa”, University Street, No 16, 700115 Iasi, Romania
| | - Silvia Curteanu
- Department of Chemical Engineering, Cristofor Simionescu Faculty of Chemical Engineering and Environmental Protection, Gheorghe Asachi Technical University, Prof.dr.doc Dimitrie Mangeron Avenue, No 67, 700050 Iasi, Romania
| | - Lisa Catãlin
- Department of Chemical Engineering, Cristofor Simionescu Faculty of Chemical Engineering and Environmental Protection, Gheorghe Asachi Technical University, Prof.dr.doc Dimitrie Mangeron Avenue, No 67, 700050 Iasi, Romania
| | - Ovidiu-Dumitru Ilie
- Department of Biology, Faculty of Biology, “Alexandru Ioan Cuza” University, Carol I Avenue, No 20A, 700505 Iasi, Romania
| | - Filip Târcoveanu
- Faculty of Medicine, University of Medicine and Pharmacy “Grigore T. Popa”, University Street, No 16, 700115 Iasi, Romania
| | - Camelia Margareta Bogdănici
- Faculty of Medicine, University of Medicine and Pharmacy “Grigore T. Popa”, University Street, No 16, 700115 Iasi, Romania
| |
Collapse
|
50
|
Schwartz R, Khalid H, Liakopoulos S, Ouyang Y, de Vente C, González-Gonzalo C, Lee AY, Guymer R, Chew EY, Egan C, Wu Z, Kumar H, Farrington J, Müller PL, Sánchez CI, Tufail A. A Deep Learning Framework for the Detection and Quantification of Reticular Pseudodrusen and Drusen on Optical Coherence Tomography. Transl Vis Sci Technol 2022; 11:3. [PMID: 36458946 PMCID: PMC9728496 DOI: 10.1167/tvst.11.12.3] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/05/2022] Open
Abstract
Purpose The purpose of this study was to develop and validate a deep learning (DL) framework for the detection and quantification of reticular pseudodrusen (RPD) and drusen on optical coherence tomography (OCT) scans. Methods A DL framework was developed consisting of a classification model and an out-of-distribution (OOD) detection model for the identification of ungradable scans; a classification model to identify scans with drusen or RPD; and an image segmentation model to independently segment lesions as RPD or drusen. Data were obtained from 1284 participants in the UK Biobank (UKBB) with a self-reported diagnosis of age-related macular degeneration (AMD) and 250 UKBB controls. Drusen and RPD were manually delineated by five retina specialists. The main outcome measures were sensitivity, specificity, area under the receiver operating characteristic (ROC) curve (AUC), kappa, accuracy, intraclass correlation coefficient (ICC), and free-response receiver operating characteristic (FROC) curves. Results The classification models performed strongly at their respective tasks (0.95, 0.93, and 0.99 AUC, respectively, for the ungradable scans classifier, the OOD model, and the drusen and RPD classification models). The mean ICC for the drusen and RPD area versus graders was 0.74 and 0.61, respectively, compared with 0.69 and 0.68 for intergrader agreement. FROC curves showed that the model's sensitivity was close to human performance. Conclusions The models achieved high classification and segmentation performance, similar to human performance. Translational Relevance Application of this robust framework will further our understanding of RPD as a separate entity from drusen in both research and clinical settings.
Collapse
Affiliation(s)
- Roy Schwartz
- Moorfields Eye Hospital NHS Foundation Trust, London, UK
- Institute of Health Informatics, University College London, London, UK
- Quantitative Healthcare Analysis (qurAI) Group, Informatics Institute, University of Amsterdam, Amsterdam, The Netherlands
| | - Hagar Khalid
- Moorfields Eye Hospital NHS Foundation Trust, London, UK
- Tanta University Hospital, Tanta, Egypt
| | - Sandra Liakopoulos
- Cologne Image Reading Center, Department of Ophthalmology, Faculty of Medicine and University Hospital Cologne, University of Cologne, Cologne, Germany
- Department of Ophthalmology, Goethe University, Frankfurt, Germany
| | - Yanling Ouyang
- Moorfields Eye Hospital NHS Foundation Trust, London, UK
| | - Coen de Vente
- Quantitative Healthcare Analysis (qurAI) Group, Informatics Institute, University of Amsterdam, Amsterdam, The Netherlands
- Amsterdam UMC location University of Amsterdam, Biomedical Engineering and Physics, Amsterdam, The Netherlands
- Diagnostic Image Analysis Group (DIAG), Department of Radiology and Nuclear Medicine, Radboud UMC, Nijmegen, The Netherlands
| | - Cristina González-Gonzalo
- Quantitative Healthcare Analysis (qurAI) Group, Informatics Institute, University of Amsterdam, Amsterdam, The Netherlands
- Diagnostic Image Analysis Group (DIAG), Department of Radiology and Nuclear Medicine, Radboud UMC, Nijmegen, The Netherlands
| | - Aaron Y. Lee
- Roger and Angie Karalis Johnson Retina Center, University of Washington, Seattle, WA, USA
- Department of Ophthalmology, University of Washington, Seattle, WA, USA
| | - Robyn Guymer
- Centre for Eye Research Australia, Royal Victorian Eye and Ear Hospital, East Melbourne, Australia
| | - Emily Y. Chew
- National Eye Institute (NEI), National Institutes of Health (NIH), Bethesda, MD, USA
| | - Catherine Egan
- Moorfields Eye Hospital NHS Foundation Trust, London, UK
| | - Zhichao Wu
- Centre for Eye Research Australia, Royal Victorian Eye and Ear Hospital, East Melbourne, Australia
| | - Himeesh Kumar
- Centre for Eye Research Australia, Royal Victorian Eye and Ear Hospital, East Melbourne, Australia
- Ophthalmology, Department of Surgery, The University of Melbourne, Melbourne, Australia
| | - Joseph Farrington
- Institute of Health Informatics, University College London, London, UK
| | - Philipp L. Müller
- Moorfields Eye Hospital NHS Foundation Trust, London, UK
- Makula Center, Südblick Eye Centers, Augsburg, Germany
- Department of Ophthalmology, University of Bonn, Bonn, Germany
| | - Clara I. Sánchez
- Quantitative Healthcare Analysis (qurAI) Group, Informatics Institute, University of Amsterdam, Amsterdam, The Netherlands
- Amsterdam UMC location University of Amsterdam, Biomedical Engineering and Physics, Amsterdam, The Netherlands
| | - Adnan Tufail
- Moorfields Eye Hospital NHS Foundation Trust, London, UK
| |
Collapse
|