1
|
Akpinar MH, Sengur A, Faust O, Tong L, Molinari F, Acharya UR. Artificial intelligence in retinal screening using OCT images: A review of the last decade (2013-2023). COMPUTER METHODS AND PROGRAMS IN BIOMEDICINE 2024; 254:108253. [PMID: 38861878 DOI: 10.1016/j.cmpb.2024.108253] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 12/22/2023] [Revised: 04/22/2024] [Accepted: 05/25/2024] [Indexed: 06/13/2024]
Abstract
BACKGROUND AND OBJECTIVES Optical coherence tomography (OCT) has ushered in a transformative era in the domain of ophthalmology, offering non-invasive imaging with high resolution for ocular disease detection. OCT, which is frequently used in diagnosing fundamental ocular pathologies, such as glaucoma and age-related macular degeneration (AMD), plays an important role in the widespread adoption of this technology. Apart from glaucoma and AMD, we will also investigate pertinent pathologies, such as epiretinal membrane (ERM), macular hole (MH), macular dystrophy (MD), vitreomacular traction (VMT), diabetic maculopathy (DMP), cystoid macular edema (CME), central serous chorioretinopathy (CSC), diabetic macular edema (DME), diabetic retinopathy (DR), drusen, glaucomatous optic neuropathy (GON), neovascular AMD (nAMD), myopia macular degeneration (MMD) and choroidal neovascularization (CNV) diseases. This comprehensive review examines the role that OCT-derived images play in detecting, characterizing, and monitoring eye diseases. METHOD The 2020 PRISMA guideline was used to structure a systematic review of research on various eye conditions using machine learning (ML) or deep learning (DL) techniques. A thorough search across IEEE, PubMed, Web of Science, and Scopus databases yielded 1787 publications, of which 1136 remained after removing duplicates. Subsequent exclusion of conference papers, review papers, and non-open-access articles reduced the selection to 511 articles. Further scrutiny led to the exclusion of 435 more articles due to lower-quality indexing or irrelevance, resulting in 76 journal articles for the review. RESULTS During our investigation, we found that a major challenge for ML-based decision support is the abundance of features and the determination of their significance. In contrast, DL-based decision support is characterized by a plug-and-play nature rather than relying on a trial-and-error approach. Furthermore, we observed that pre-trained networks are practical and especially useful when working on complex images such as OCT. Consequently, pre-trained deep networks were frequently utilized for classification tasks. Currently, medical decision support aims to reduce the workload of ophthalmologists and retina specialists during routine tasks. In the future, it might be possible to create continuous learning systems that can predict ocular pathologies by identifying subtle changes in OCT images.
Collapse
Affiliation(s)
- Muhammed Halil Akpinar
- Department of Electronics and Automation, Vocational School of Technical Sciences, Istanbul University-Cerrahpasa, Istanbul, Turkey
| | - Abdulkadir Sengur
- Electrical-Electronics Engineering Department, Technology Faculty, Firat University, Elazig, Turkey.
| | - Oliver Faust
- School of Computing and Information Science, Anglia Ruskin University Cambridge Campus, United Kingdom
| | - Louis Tong
- Singapore Eye Research Institute, Singapore, Singapore
| | - Filippo Molinari
- Biolab, PolitoBIOMedLab, Department of Electronics and Telecommunications, Politecnico di Torino, Turin, Italy
| | - U Rajendra Acharya
- School of Mathematics, Physics and Computing, University of Southern Queensland, Springfield, Australia
| |
Collapse
|
2
|
Kalupahana D, Kahatapitiya NS, Silva BN, Kim J, Jeon M, Wijenayake U, Wijesinghe RE. Dense Convolutional Neural Network-Based Deep Learning Pipeline for Pre-Identification of Circular Leaf Spot Disease of Diospyros kaki Leaves Using Optical Coherence Tomography. SENSORS (BASEL, SWITZERLAND) 2024; 24:5398. [PMID: 39205092 PMCID: PMC11359294 DOI: 10.3390/s24165398] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 06/21/2024] [Revised: 07/30/2024] [Accepted: 08/05/2024] [Indexed: 09/04/2024]
Abstract
Circular leaf spot (CLS) disease poses a significant threat to persimmon cultivation, leading to substantial harvest reductions. Existing visual and destructive inspection methods suffer from subjectivity, limited accuracy, and considerable time consumption. This study presents an automated pre-identification method of the disease through a deep learning (DL) based pipeline integrated with optical coherence tomography (OCT), thereby addressing the highlighted issues with the existing methods. The investigation yielded promising outcomes by employing transfer learning with pre-trained DL models, specifically DenseNet-121 and VGG-16. The DenseNet-121 model excels in differentiating among three stages of CLS disease (healthy (H), apparently healthy (or healthy-infected (HI)), and infected (I)). The model achieved precision values of 0.7823 for class-H, 0.9005 for class-HI, and 0.7027 for class-I, supported by recall values of 0.8953 for class-HI and 0.8387 for class-I. Moreover, the performance of CLS detection was enhanced by a supplemental quality inspection model utilizing VGG-16, which attained an accuracy of 98.99% in discriminating between low-detail and high-detail images. Moreover, this study employed a combination of LAMP and A-scan for the dataset labeling process, significantly enhancing the accuracy of the models. Overall, this study underscores the potential of DL techniques integrated with OCT to enhance disease identification processes in agricultural settings, particularly in persimmon cultivation, by offering efficient and objective pre-identification of CLS and enabling early intervention and management strategies.
Collapse
Affiliation(s)
- Deshan Kalupahana
- Department of Computer Engineering, Faculty of Engineering, University of Sri Jayewardenepura, Nugegoda 10250, Sri Lanka; (D.K.); (N.S.K.)
| | - Nipun Shantha Kahatapitiya
- Department of Computer Engineering, Faculty of Engineering, University of Sri Jayewardenepura, Nugegoda 10250, Sri Lanka; (D.K.); (N.S.K.)
| | - Bhagya Nathali Silva
- Department of Information Technology, Faculty of Computing, Sri Lanka Institute of Information Technology, Malabe 10115, Sri Lanka;
- Center for Excellence in Informatics, Electronics & Transmission (CIET), Sri Lanka Institute of Information Technology, Malabe 10115, Sri Lanka
| | - Jeehyun Kim
- School of Electronic and Electrical Engineering, College of IT Engineering, Kyungpook National University, 80 Daehak-ro, Buk-gu, Daegu 41566, Republic of Korea; (J.K.); (M.J.)
| | - Mansik Jeon
- School of Electronic and Electrical Engineering, College of IT Engineering, Kyungpook National University, 80 Daehak-ro, Buk-gu, Daegu 41566, Republic of Korea; (J.K.); (M.J.)
| | - Udaya Wijenayake
- Department of Computer Engineering, Faculty of Engineering, University of Sri Jayewardenepura, Nugegoda 10250, Sri Lanka; (D.K.); (N.S.K.)
| | - Ruchire Eranga Wijesinghe
- Center for Excellence in Informatics, Electronics & Transmission (CIET), Sri Lanka Institute of Information Technology, Malabe 10115, Sri Lanka
- Department of Electrical and Electronic Engineering, Faculty of Engineering, Sri Lanka Institute of Information Technology, Malabe 10115, Sri Lanka
| |
Collapse
|
3
|
Pang S, Zou B, Xiao X, Peng Q, Yan J, Zhang W, Yue K. A novel approach for automatic classification of macular degeneration OCT images. Sci Rep 2024; 14:19285. [PMID: 39164445 PMCID: PMC11335908 DOI: 10.1038/s41598-024-70175-2] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/14/2024] [Accepted: 08/13/2024] [Indexed: 08/22/2024] Open
Abstract
Age-related macular degeneration (AMD) and diabetic macular edema (DME) are significant causes of blindness worldwide. The prevalence of these diseases is steadily increasing due to population aging. Therefore, early diagnosis and prevention are crucial for effective treatment. Classification of Macular Degeneration OCT Images is a widely used method for assessing retinal lesions. However, there are two main challenges in OCT image classification: incomplete image feature extraction and lack of prominence in important positional features. To address these challenges, we proposed a deep learning neural network model called MSA-Net, which incorporates our proposed multi-scale architecture and spatial attention mechanism. Our multi-scale architecture is based on depthwise separable convolution, which ensures comprehensive feature extraction from multiple scales while minimizing the growth of model parameters. The spatial attention mechanism is aim to highlight the important positional features in the images, which emphasizes the representation of macular region features in OCT images. We test MSA-NET on the NEH dataset and the UCSD dataset, performing three-class (CNV, DURSEN, and NORMAL) and four-class (CNV, DURSEN, DME, and NORMAL) classification tasks. On the NEH dataset, the accuracy, sensitivity, and specificity are 98.1%, 97.9%, and 98.0%, respectively. After fine-tuning on the UCSD dataset, the accuracy, sensitivity, and specificity are 96.7%, 96.7%, and 98.9%, respectively. Experimental results demonstrate the excellent classification performance and generalization ability of our model compared to previous models and recent well-known OCT classification models, establishing it as a highly competitive intelligence classification approach in the field of macular degeneration.
Collapse
Affiliation(s)
- Shilong Pang
- School of Informatics, Hunan University of Chinese Medicine, Changsha, 410208, Hunan, China
| | - Beiji Zou
- School of Informatics, Hunan University of Chinese Medicine, Changsha, 410208, Hunan, China
- School of Computer Science and Engineering, Central South University, Changsha, 410083, Hunan, China
| | - Xiaoxia Xiao
- School of Informatics, Hunan University of Chinese Medicine, Changsha, 410208, Hunan, China.
| | - Qinghua Peng
- School of Traditional Chinese Medicine, Hunan University of Chinese Medicine, Changsha, 410208, Hunan, China
| | - Junfeng Yan
- School of Informatics, Hunan University of Chinese Medicine, Changsha, 410208, Hunan, China
| | - Wensheng Zhang
- School of Informatics, Hunan University of Chinese Medicine, Changsha, 410208, Hunan, China
- University of Chinese Academy of Sciences (UCAS), Beijing, 100049, China
- Research Center of Precision Sensing and Control, Institute of Automation, Chinese Academy of Sciences, Beijing, 100190, China
| | - Kejuan Yue
- School of Computer Science, Hunan First Normal University, Changsha, 410205, Hunan, China
| |
Collapse
|
4
|
Rozhyna A, Somfai GM, Atzori M, DeBuc DC, Saad A, Zoellin J, Müller H. Exploring Publicly Accessible Optical Coherence Tomography Datasets: A Comprehensive Overview. Diagnostics (Basel) 2024; 14:1668. [PMID: 39125544 PMCID: PMC11312046 DOI: 10.3390/diagnostics14151668] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/31/2024] [Revised: 07/15/2024] [Accepted: 07/25/2024] [Indexed: 08/12/2024] Open
Abstract
Artificial intelligence has transformed medical diagnostic capabilities, particularly through medical image analysis. AI algorithms perform well in detecting abnormalities with a strong performance, enabling computer-aided diagnosis by analyzing the extensive amounts of patient data. The data serve as a foundation upon which algorithms learn and make predictions. Thus, the importance of data cannot be underestimated, and clinically corresponding datasets are required. Many researchers face a lack of medical data due to limited access, privacy concerns, or the absence of available annotations. One of the most widely used diagnostic tools in ophthalmology is Optical Coherence Tomography (OCT). Addressing the data availability issue is crucial for enhancing AI applications in the field of OCT diagnostics. This review aims to provide a comprehensive analysis of all publicly accessible retinal OCT datasets. Our main objective is to compile a list of OCT datasets and their properties, which can serve as an accessible reference, facilitating data curation for medical image analysis tasks. For this review, we searched through the Zenodo repository, Mendeley Data repository, MEDLINE database, and Google Dataset search engine. We systematically evaluated all the identified datasets and found 23 open-access datasets containing OCT images, which significantly vary in terms of size, scope, and ground-truth labels. Our findings indicate the need for improvement in data-sharing practices and standardized documentation. Enhancing the availability and quality of OCT datasets will support the development of AI algorithms and ultimately improve diagnostic capabilities in ophthalmology. By providing a comprehensive list of accessible OCT datasets, this review aims to facilitate better utilization and development of AI in medical image analysis.
Collapse
Affiliation(s)
- Anastasiia Rozhyna
- Informatics Institute, University of Applied Sciences Western Switzerland (HES-SO), 3960 Sierre, Switzerland
- Medical Informatics, University of Geneva, 1205 Geneva, Switzerland
| | - Gábor Márk Somfai
- Department of Ophthalmology, Stadtspital Zürich, 8063 Zurich, Switzerland
- Spross Research Institute, 8063 Zurich, Switzerland
| | - Manfredo Atzori
- Informatics Institute, University of Applied Sciences Western Switzerland (HES-SO), 3960 Sierre, Switzerland
- Department of Neuroscience, University of Padua, 35121 Padova, Italy
| | - Delia Cabrera DeBuc
- Bascom Palmer Eye Institute, Miller School of Medicine, University of Miami, Miami, FL 33136, USA
| | - Amr Saad
- Department of Ophthalmology, Stadtspital Zürich, 8063 Zurich, Switzerland
- Spross Research Institute, 8063 Zurich, Switzerland
| | - Jay Zoellin
- Department of Ophthalmology, Stadtspital Zürich, 8063 Zurich, Switzerland
- Spross Research Institute, 8063 Zurich, Switzerland
| | - Henning Müller
- Informatics Institute, University of Applied Sciences Western Switzerland (HES-SO), 3960 Sierre, Switzerland
- Medical Informatics, University of Geneva, 1205 Geneva, Switzerland
- The Sense Research and Innovation Center, 1007 Lausanne, Switzerland
| |
Collapse
|
5
|
Abd El-Khalek AA, Balaha HM, Sewelam A, Ghazal M, Khalil AT, Abo-Elsoud MEA, El-Baz A. A Comprehensive Review of AI Diagnosis Strategies for Age-Related Macular Degeneration (AMD). Bioengineering (Basel) 2024; 11:711. [PMID: 39061793 PMCID: PMC11273790 DOI: 10.3390/bioengineering11070711] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/12/2024] [Revised: 07/02/2024] [Accepted: 07/09/2024] [Indexed: 07/28/2024] Open
Abstract
The rapid advancement of computational infrastructure has led to unprecedented growth in machine learning, deep learning, and computer vision, fundamentally transforming the analysis of retinal images. By utilizing a wide array of visual cues extracted from retinal fundus images, sophisticated artificial intelligence models have been developed to diagnose various retinal disorders. This paper concentrates on the detection of Age-Related Macular Degeneration (AMD), a significant retinal condition, by offering an exhaustive examination of recent machine learning and deep learning methodologies. Additionally, it discusses potential obstacles and constraints associated with implementing this technology in the field of ophthalmology. Through a systematic review, this research aims to assess the efficacy of machine learning and deep learning techniques in discerning AMD from different modalities as they have shown promise in the field of AMD and retinal disorders diagnosis. Organized around prevalent datasets and imaging techniques, the paper initially outlines assessment criteria, image preprocessing methodologies, and learning frameworks before conducting a thorough investigation of diverse approaches for AMD detection. Drawing insights from the analysis of more than 30 selected studies, the conclusion underscores current research trajectories, major challenges, and future prospects in AMD diagnosis, providing a valuable resource for both scholars and practitioners in the domain.
Collapse
Affiliation(s)
- Aya A. Abd El-Khalek
- Communications and Electronics Engineering Department, Nile Higher Institute for Engineering and Technology, Mansoura 35511, Egypt;
| | - Hossam Magdy Balaha
- Department of Bioengineering, J.B. Speed School of Engineering, University of Louisville, Louisville, KY 40292, USA
| | - Ashraf Sewelam
- Ophthalmology Department, Faculty of Medicine, Mansoura University, Mansoura 35511, Egypt;
| | - Mohammed Ghazal
- Electrical, Computer, and Biomedical Engineering Department, Abu Dhabi University, Abu Dhabi 59911, United Arab Emirates;
| | - Abeer T. Khalil
- Communications and Electronics Engineering Department, Faculty of Engineering, Mansoura University, Mansoura 35511, Egypt; (A.T.K.); (M.E.A.A.-E.)
| | - Mohy Eldin A. Abo-Elsoud
- Communications and Electronics Engineering Department, Faculty of Engineering, Mansoura University, Mansoura 35511, Egypt; (A.T.K.); (M.E.A.A.-E.)
| | - Ayman El-Baz
- Department of Bioengineering, J.B. Speed School of Engineering, University of Louisville, Louisville, KY 40292, USA
| |
Collapse
|
6
|
Wong CYT, Antaki F, Woodward-Court P, Ong AY, Keane PA. The role of saliency maps in enhancing ophthalmologists' trust in artificial intelligence models. Asia Pac J Ophthalmol (Phila) 2024; 13:100087. [PMID: 39069106 DOI: 10.1016/j.apjo.2024.100087] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/21/2024] [Revised: 07/17/2024] [Accepted: 07/23/2024] [Indexed: 07/30/2024] Open
Abstract
PURPOSE Saliency maps (SM) allow clinicians to better understand the opaque decision-making process in artificial intelligence (AI) models by visualising the important features responsible for predictions. This ultimately improves interpretability and confidence. In this work, we review the use case for SMs, exploring their impact on clinicians' understanding and trust in AI models. We use the following ophthalmic conditions as examples: (1) glaucoma, (2) myopia, (3) age-related macular degeneration, and (4) diabetic retinopathy. METHOD A multi-field search on MEDLINE, Embase, and Web of Science was conducted using specific keywords. Only studies on the use of SMs in glaucoma, myopia, AMD, or DR were considered for inclusion. RESULTS Findings reveal that SMs are often used to validate AI models and advocate for their adoption, potentially leading to biased claims. Overlooking the technical limitations of SMs, and the conductance of superficial assessments of their quality and relevance, was discerned. Uncertainties persist regarding the role of saliency maps in building trust in AI. It is crucial to enhance understanding of SMs' technical constraints and improve evaluation of their quality, impact, and suitability for specific tasks. Establishing a standardised framework for selecting and assessing SMs, as well as exploring their relationship with other reliability sources (e.g. safety and generalisability), is essential for enhancing clinicians' trust in AI. CONCLUSION We conclude that SMs are not beneficial for interpretability and trust-building purposes in their current forms. Instead, SMs may confer benefits to model debugging, model performance enhancement, and hypothesis testing (e.g. novel biomarkers).
Collapse
Affiliation(s)
| | - Fares Antaki
- Institute of Ophthalmology, University College London, London, United Kingdom
| | | | - Ariel Yuhan Ong
- Institute of Ophthalmology, University College London, London, United Kingdom
| | - Pearse A Keane
- Institute of Ophthalmology, University College London, London, United Kingdom.
| |
Collapse
|
7
|
Azizi MM, Abhari S, Sajedi H. Stitched vision transformer for age-related macular degeneration detection using retinal optical coherence tomography images. PLoS One 2024; 19:e0304943. [PMID: 38837967 DOI: 10.1371/journal.pone.0304943] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/23/2023] [Accepted: 05/21/2024] [Indexed: 06/07/2024] Open
Abstract
Age-related macular degeneration (AMD) is an eye disease that leads to the deterioration of the central vision area of the eye and can gradually result in vision loss in elderly individuals. Early identification of this disease can significantly impact patient treatment outcomes. Furthermore, given the increasing elderly population globally, the importance of automated methods for rapidly monitoring at-risk individuals and accurately diagnosing AMD is growing daily. One standard method for diagnosing AMD is using optical coherence tomography (OCT) images as a non-invasive imaging technology. In recent years, numerous deep neural networks have been proposed for the classification of OCT images. Utilizing pre-trained neural networks can speed up model deployment in related tasks without compromising accuracy. However, most previous methods overlook the feasibility of leveraging pre-existing trained networks to search for an optimal architecture for AMD staging on a new target dataset. In this study, our objective was to achieve an optimal architecture in the efficiency-accuracy trade-off for classifying retinal OCT images. To this end, we employed pre-trained medical vision transformer (MedViT) models. MedViT combines convolutional and transformer neural networks, explicitly designed for medical image classification. Our approach involved pre-training two distinct MedViT models on a source dataset with labels identical to those in the target dataset. This pre-training was conducted in a supervised manner. Subsequently, we evaluated the performance of the pre-trained MedViT models for classifying retinal OCT images from the target Noor Eye Hospital (NEH) dataset into the normal, drusen, and choroidal neovascularization (CNV) classes in zero-shot settings and through five-fold cross-validation. Then, we proposed a stitching approach to search for an optimal model from two MedViT family models. The proposed stitching method is an efficient architecture search algorithm known as stitchable neural networks. Stitchable neural networks create a candidate model in search space for each pair of stitchable layers by inserting a linear layer between them. A pair of stitchable layers consists of layers, each selected from one input model. While stitchable neural networks had previously been tested on more extensive and general datasets, this study demonstrated that stitching networks could also be helpful in smaller medical datasets. The results of this approach indicate that when pre-trained models were available for OCT images from another dataset, it was possible to achieve a model in 100 epochs with an accuracy of over 94.9% in classifying images from the NEH dataset. The results of this study demonstrate the efficacy of stitchable neural networks as a fine-tuning method for OCT image classification. This approach not only leads to higher accuracy but also considers architecture optimization at a reasonable computational cost.
Collapse
Affiliation(s)
- Mohammad Mahdi Azizi
- Department of Mathematics, Statistics and Computer Science, College of Science, University of Tehran, Tehran, Iran
| | - Setareh Abhari
- Department of Mathematics, Statistics and Computer Science, College of Science, University of Tehran, Tehran, Iran
| | - Hedieh Sajedi
- Department of Mathematics, Statistics and Computer Science, College of Science, University of Tehran, Tehran, Iran
| |
Collapse
|
8
|
Liu X, Zhu X, Zhang Y, Wang M. Point based weakly semi-supervised biomarker detection with cross-scale and label assignment in retinal OCT images. COMPUTER METHODS AND PROGRAMS IN BIOMEDICINE 2024; 251:108229. [PMID: 38761413 DOI: 10.1016/j.cmpb.2024.108229] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 10/23/2023] [Revised: 04/09/2024] [Accepted: 05/14/2024] [Indexed: 05/20/2024]
Abstract
BACKGROUND AND OBJECTIVE Optical coherence tomography (OCT) is currently one of the most advanced retinal imaging methods. Retinal biomarkers in OCT images are of clinical significance and can assist ophthalmologists in diagnosing lesions. Compared with fundus images, OCT can provide higher resolution segmentation. However, image annotation at the bounding box level needs to be performed by ophthalmologists carefully and is difficult to obtain. In addition, the large variation in shape of different retinal markers and the inconspicuous appearance of biomarkers make it difficult for existing deep learning-based methods to effectively detect them. To overcome the above challenges, we propose a novel network for the detection of retinal biomarkers in OCT images. METHODS We first address the issue of labeling cost using a novel weakly semi-supervised object detection method with point annotations which can reduce bounding box-level annotation efforts. To extend the method to the detection of biomarkers in OCT images, we propose multiple consistent regularizations for point-to-box regression network to deal with the shortage of supervision, which aims to learn more accurate regression mappings. Furthermore, in the subsequent fully supervised detection, we propose a cross-scale feature enhancement module to alleviate the detection problems caused by the large-scale variation of biomarkers. We also propose a dynamic label assignment strategy to distinguish samples of different importance more flexibly, thereby reducing detection errors due to the indistinguishable appearance of the biomarkers. RESULTS When using our detection network, our regressor also achieves an AP value of 20.83 s when utilizing a 5 % fully labeled dataset partition, surpassing the performance of other comparative methods at 5 % and 10 %. Even coming close to the 20.87 % result achieved by Point DETR under 20 % full labeling conditions. When using Group R-CNN as the point-to-box regressor, our detector achieves 27.21 % AP in the 50 % fully labeled dataset experiment. 7.42 % AP improvement is achieved compared to our detection network baseline Faster R-CNN. CONCLUSIONS The experimental findings not only demonstrate the effectiveness of our approach with minimal bounding box annotations but also highlight the enhanced biomarker detection performance of the proposed module. We have included a detailed algorithmic flow in the supplementary material.
Collapse
Affiliation(s)
- Xiaoming Liu
- School of Computer Science and Technology, Wuhan University of Science and Technology, Wuhan 430065, PR China; Hubei Province Key Laboratory of Intelligent Information Processing and Real-time Industrial System, Wuhan 430065, PR China.
| | - Xin Zhu
- School of Computer Science and Technology, Wuhan University of Science and Technology, Wuhan 430065, PR China; Hubei Province Key Laboratory of Intelligent Information Processing and Real-time Industrial System, Wuhan 430065, PR China
| | - Ying Zhang
- Aier Eye Hospital of Wuhan University, Wuhan 430064, PR China
| | - Man Wang
- Aier Eye Hospital of Wuhan University, Wuhan 430064, PR China
| |
Collapse
|
9
|
Bhulakshmi D, Rajput DS. A systematic review on diabetic retinopathy detection and classification based on deep learning techniques using fundus images. PeerJ Comput Sci 2024; 10:e1947. [PMID: 38699206 PMCID: PMC11065411 DOI: 10.7717/peerj-cs.1947] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/15/2023] [Accepted: 02/28/2024] [Indexed: 05/05/2024]
Abstract
Diabetic retinopathy (DR) is the leading cause of visual impairment globally. It occurs due to long-term diabetes with fluctuating blood glucose levels. It has become a significant concern for people in the working age group as it can lead to vision loss in the future. Manual examination of fundus images is time-consuming and requires much effort and expertise to determine the severity of the retinopathy. To diagnose and evaluate the disease, deep learning-based technologies have been used, which analyze blood vessels, microaneurysms, exudates, macula, optic discs, and hemorrhages also used for initial detection and grading of DR. This study examines the fundamentals of diabetes, its prevalence, complications, and treatment strategies that use artificial intelligence methods such as machine learning (ML), deep learning (DL), and federated learning (FL). The research covers future studies, performance assessments, biomarkers, screening methods, and current datasets. Various neural network designs, including recurrent neural networks (RNNs), generative adversarial networks (GANs), and applications of ML, DL, and FL in the processing of fundus images, such as convolutional neural networks (CNNs) and their variations, are thoroughly examined. The potential research methods, such as developing DL models and incorporating heterogeneous data sources, are also outlined. Finally, the challenges and future directions of this research are discussed.
Collapse
Affiliation(s)
- Dasari Bhulakshmi
- School of Computer Science Engineering and Information Systems, Vellore Institute of Technology, Vellore, Tamil Nadu, India
| | - Dharmendra Singh Rajput
- School of Computer Science Engineering and Information Systems, Vellore Institute of Technology, Vellore, Tamil Nadu, India
| |
Collapse
|
10
|
Yang J, Wang G, Xiao X, Bao M, Tian G. Explainable ensemble learning method for OCT detection with transfer learning. PLoS One 2024; 19:e0296175. [PMID: 38517913 PMCID: PMC10959366 DOI: 10.1371/journal.pone.0296175] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/02/2023] [Accepted: 12/07/2023] [Indexed: 03/24/2024] Open
Abstract
The accuracy and interpretability of artificial intelligence (AI) are crucial for the advancement of optical coherence tomography (OCT) image detection, as it can greatly reduce the manual labor required by clinicians. By prioritizing these aspects during development and application, we can make significant progress towards streamlining the clinical workflow. In this paper, we propose an explainable ensemble approach that utilizes transfer learning to detect fundus lesion diseases through OCT imaging. Our study utilized a publicly available OCT dataset consisting of normal subjects, patients with dry age-related macular degeneration (AMD), and patients with diabetic macular edema (DME), each with 15 samples. The impact of pre-trained weights on the performance of individual networks was first compared, and then these networks were ensemble using majority soft polling. Finally, the features learned by the networks were visualized using Grad-CAM and CAM. The use of pre-trained ImageNet weights improved the performance from 68.17% to 92.89%. The ensemble model consisting of the three CNN models with pre-trained parameters loaded performed best, correctly distinguishing between AMD patients, DME patients and normal subjects 100% of the time. Visualization results showed that Grad-CAM could display the lesion area more accurately. It is demonstrated that the proposed approach could have good performance of both accuracy and interpretability in retinal OCT image detection.
Collapse
Affiliation(s)
- Jiasheng Yang
- Academician Workstation, Changsha Medical University, Changsha, Hunan, China
| | - Guanfang Wang
- School of Mathematics and Statistics, Nanjing University of Information Science and Technology, Nanjing, Jiangsu, China
- Geneis Beijing Co. Ltd., Beijing, China
| | - Xu Xiao
- School of International Education, Anhui University of Technology, Maanshan, Anhui, China
| | - Meihua Bao
- Academician Workstation, Changsha Medical University, Changsha, Hunan, China
| | - Geng Tian
- Geneis Beijing Co. Ltd., Beijing, China
| |
Collapse
|
11
|
Liu J, Chen L, Xiong H, Zhang L. A multi-scale attention residual-based U-Net network for stroke electrical impedance tomography. THE REVIEW OF SCIENTIFIC INSTRUMENTS 2024; 95:033702. [PMID: 38526440 DOI: 10.1063/5.0176494] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Received: 09/14/2023] [Accepted: 03/02/2024] [Indexed: 03/26/2024]
Abstract
Electrical impedance tomography (EIT), a non-invasive, radiation-free, and convenient imaging technique, has been widely used in the diagnosis of stroke. However, due to soft-field nonlinearity and the ill-posed inverse problem, EIT images always suffer from low spatial resolution. Therefore, a multi-scale convolutional attention residual-based U-Net (MARU-Net) network is proposed for stroke reconstruction. Based on the U-Net network, a residual module and a multi-scale convolutional attention module are added to the concatenation layer. The multi-scale module extracts feature information of different sizes, the attention module strengthens the useful information, and the residual module improves the performance of the network. Based on the above advantages, the network is used in the EIT system for stroke imaging. Compared with convolutional neural networks and one-dimensional convolutional neural networks, the MARU-Net network has fewer artifacts, and the reconstructed image is clear. At the same time, the reduction of noisy artifacts in the MARU-Net network is verified. The results show that the image correlation coefficient of the reconstructed image with noise is greater than 0.87. Finally, the practicability of the network is verified by a model physics experiment.
Collapse
Affiliation(s)
- Jinzhen Liu
- The School of Control Science and Engineering, Tiangong University, Tianjin 300387, People's Republic of China
- Tianjin Key Laboratory of Intelligent Control of Electrical Equipment, Tiangong University, Tianjin 300387, People's Republic of China
| | - Liming Chen
- The School of Control Science and Engineering, Tiangong University, Tianjin 300387, People's Republic of China
- Tianjin Key Laboratory of Intelligent Control of Electrical Equipment, Tiangong University, Tianjin 300387, People's Republic of China
| | - Hui Xiong
- The School of Control Science and Engineering, Tiangong University, Tianjin 300387, People's Republic of China
- Tianjin Key Laboratory of Intelligent Control of Electrical Equipment, Tiangong University, Tianjin 300387, People's Republic of China
| | - Liying Zhang
- The School of Control Science and Engineering, Tiangong University, Tianjin 300387, People's Republic of China
- Tianjin Key Laboratory of Intelligent Control of Electrical Equipment, Tiangong University, Tianjin 300387, People's Republic of China
| |
Collapse
|
12
|
Prabha AJ, Venkatesan C, Fathimal MS, Nithiyanantham KK, Kirubha SPA. RD-OCT net: hybrid learning system for automated diagnosis of macular diseases from OCT retinal images. Biomed Phys Eng Express 2024; 10:025033. [PMID: 38335542 DOI: 10.1088/2057-1976/ad27ea] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/17/2023] [Accepted: 02/09/2024] [Indexed: 02/12/2024]
Abstract
Macular Edema is a leading cause of visual impairment and blindness in patients with ocular fundus diseases. Due to its non-invasive and high-resolution characteristics, optical coherence tomography (OCT) has been extensively utilized for the diagnosis of macular diseases. The manual detection of retinal diseases by clinicians is a laborious process, further complicated by the challenging identification of macular diseases. This difficulty arises from the significant pathological alterations occurring within the retinal layers, as well as the accumulation of fluid in the retina. Deep Learning neural networks are utilized for automatic detection of retinal diseases. This paper aims to propose a lightweight hybrid learning Retinal Disease OCT Net with a reduced number of trainable parameters and enable automatic classification of retinal diseases. A Hybrid Learning Retinal Disease OCT Net (RD-OCT) is utilized for the multiclass classification of major retinal diseases, namely neovascular age-related macular degeneration (nAMD), diabetic macular edema (DME), retinal vein occlusion (RVO), and normal retinal conditions. The diagnosis of retinal diseases is facilitated by the use of hybrid learning models and pre-trained deep learning models in the field of artificial intelligence. The Hybrid Learning RD-OCT Net provides better accuracy of 97.6% for nAMD, 98.08% for DME, 98% for RVO, and 97% for the Normal group. The respective area under the curve values were 0.99, 0.97, 1.0, and 0.99. The utilization of the RD-OCT model will be useful for ophthalmologists in the diagnosis of prevalent retinal diseases, due to the simplicity of the system and reduced number of trainable parameters.
Collapse
Affiliation(s)
- A Jeya Prabha
- Department of Biomedical Engineering, College of Engineering and Technology, SRM Institute of Science and Technology, Kattankulathur, Chengalpattu-603203, Tamil Nadu, India
| | - C Venkatesan
- Department of Ophthalmology, SRM Medical College Hospital and Research Centre, Kattankulathur, Chengalpattu-603203, Tamil Nadu, India
| | - M Sameera Fathimal
- Department of Biomedical Engineering, College of Engineering and Technology, SRM Institute of Science and Technology, Kattankulathur, Chengalpattu-603203, Tamil Nadu, India
| | - K K Nithiyanantham
- Department of Aeronautical Engineering, Rajalakshmi Engineering College, Thandalam , Kancheepuram-602105, Tamil Nadu, India
| | - S P Angeline Kirubha
- Department of Biomedical Engineering, College of Engineering and Technology, SRM Institute of Science and Technology, Kattankulathur, Chengalpattu-603203, Tamil Nadu, India
| |
Collapse
|
13
|
Peng J, Lu J, Zhuo J, Li P. Multi-Scale-Denoising Residual Convolutional Network for Retinal Disease Classification Using OCT. SENSORS (BASEL, SWITZERLAND) 2023; 24:150. [PMID: 38203011 PMCID: PMC10781341 DOI: 10.3390/s24010150] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 11/21/2023] [Revised: 12/13/2023] [Accepted: 12/15/2023] [Indexed: 01/12/2024]
Abstract
Macular pathologies can cause significant vision loss. Optical coherence tomography (OCT) images of the retina can assist ophthalmologists in diagnosing macular diseases. Traditional deep learning networks for retinal disease classification cannot extract discriminative features under strong noise conditions in OCT images. To address this issue, we propose a multi-scale-denoising residual convolutional network (MS-DRCN) for classifying retinal diseases. Specifically, the MS-DRCN includes a soft-denoising block (SDB), a multi-scale context block (MCB), and a feature fusion block (FFB). The SDB can determine the threshold for soft thresholding automatically, which removes speckle noise features efficiently. The MCB is designed to capture multi-scale context information and strengthen extracted features. The FFB is dedicated to integrating high-resolution and low-resolution features to precisely identify variable lesion areas. Our approach achieved classification accuracies of 96.4% and 96.5% on the OCT2017 and OCT-C4 public datasets, respectively, outperforming other classification methods. To evaluate the robustness of our method, we introduced Gaussian noise and speckle noise with varying PSNRs into the test set of the OCT2017 dataset. The results of our anti-noise experiments demonstrate that our approach exhibits superior robustness compared with other methods, yielding accuracy improvements ranging from 0.6% to 2.9% when compared with ResNet under various PSNR noise conditions.
Collapse
Affiliation(s)
- Jinbo Peng
- State Key Laboratory of Digital Medical Engineering, School of Biomedical Engineering, Hainan University, Haiko 570228, China; (J.P.); (J.L.)
- Key Laboratory of Biomedical Engineering of Hainan Province, One Health Institute, Hainan University, Haiko 570228, China
- Research Unit of Multimodal Cross Scale Neural Signal Detection and Imaging, Chinese Academy of Medical Science, HUST-Suzhou Institute for Brainsmatics, Jiangsu Industrial Technology Research Institute (JITRI), Suzhou 215100, China
| | - Jinling Lu
- State Key Laboratory of Digital Medical Engineering, School of Biomedical Engineering, Hainan University, Haiko 570228, China; (J.P.); (J.L.)
- Key Laboratory of Biomedical Engineering of Hainan Province, One Health Institute, Hainan University, Haiko 570228, China
- Research Unit of Multimodal Cross Scale Neural Signal Detection and Imaging, Chinese Academy of Medical Science, HUST-Suzhou Institute for Brainsmatics, Jiangsu Industrial Technology Research Institute (JITRI), Suzhou 215100, China
- Britton Chance Center for Biomedical Photonics and MoE Key Laboratory for Biomedical Photonics, Wuhan National Laboratory for Optoelectronics, Huazhong University of Science and Technology, Wuhan 430074, China
| | - Junjie Zhuo
- State Key Laboratory of Digital Medical Engineering, School of Biomedical Engineering, Hainan University, Haiko 570228, China; (J.P.); (J.L.)
- Key Laboratory of Biomedical Engineering of Hainan Province, One Health Institute, Hainan University, Haiko 570228, China
- Research Unit of Multimodal Cross Scale Neural Signal Detection and Imaging, Chinese Academy of Medical Science, HUST-Suzhou Institute for Brainsmatics, Jiangsu Industrial Technology Research Institute (JITRI), Suzhou 215100, China
| | - Pengcheng Li
- State Key Laboratory of Digital Medical Engineering, School of Biomedical Engineering, Hainan University, Haiko 570228, China; (J.P.); (J.L.)
- Key Laboratory of Biomedical Engineering of Hainan Province, One Health Institute, Hainan University, Haiko 570228, China
- Research Unit of Multimodal Cross Scale Neural Signal Detection and Imaging, Chinese Academy of Medical Science, HUST-Suzhou Institute for Brainsmatics, Jiangsu Industrial Technology Research Institute (JITRI), Suzhou 215100, China
- Britton Chance Center for Biomedical Photonics and MoE Key Laboratory for Biomedical Photonics, Wuhan National Laboratory for Optoelectronics, Huazhong University of Science and Technology, Wuhan 430074, China
| |
Collapse
|
14
|
Leingang O, Riedl S, Mai J, Reiter GS, Faustmann G, Fuchs P, Scholl HPN, Sivaprasad S, Rueckert D, Lotery A, Schmidt-Erfurth U, Bogunović H. Automated deep learning-based AMD detection and staging in real-world OCT datasets (PINNACLE study report 5). Sci Rep 2023; 13:19545. [PMID: 37945665 PMCID: PMC10636170 DOI: 10.1038/s41598-023-46626-7] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/08/2023] [Accepted: 11/03/2023] [Indexed: 11/12/2023] Open
Abstract
Real-world retinal optical coherence tomography (OCT) scans are available in abundance in primary and secondary eye care centres. They contain a wealth of information to be analyzed in retrospective studies. The associated electronic health records alone are often not enough to generate a high-quality dataset for clinical, statistical, and machine learning analysis. We have developed a deep learning-based age-related macular degeneration (AMD) stage classifier, to efficiently identify the first onset of early/intermediate (iAMD), atrophic (GA), and neovascular (nAMD) stage of AMD in retrospective data. We trained a two-stage convolutional neural network to classify macula-centered 3D volumes from Topcon OCT images into 4 classes: Normal, iAMD, GA and nAMD. In the first stage, a 2D ResNet50 is trained to identify the disease categories on the individual OCT B-scans while in the second stage, four smaller models (ResNets) use the concatenated B-scan-wise output from the first stage to classify the entire OCT volume. Classification uncertainty estimates are generated with Monte-Carlo dropout at inference time. The model was trained on a real-world OCT dataset, 3765 scans of 1849 eyes, and extensively evaluated, where it reached an average ROC-AUC of 0.94 in a real-world test set.
Collapse
Affiliation(s)
- Oliver Leingang
- Department of Ophthalmology and Optometry, Medical University of Vienna, Vienna, Austria
| | - Sophie Riedl
- Department of Ophthalmology and Optometry, Medical University of Vienna, Vienna, Austria
| | - Julia Mai
- Department of Ophthalmology and Optometry, Medical University of Vienna, Vienna, Austria
| | - Gregor S Reiter
- Department of Ophthalmology and Optometry, Medical University of Vienna, Vienna, Austria
| | - Georg Faustmann
- Department of Ophthalmology and Optometry, Medical University of Vienna, Vienna, Austria
- Christian Doppler Lab for Artificial Intelligence in Retina, Department of Ophthalmology and Optometry, Medical University of Vienna, Vienna, Austria
| | - Philipp Fuchs
- Department of Ophthalmology and Optometry, Medical University of Vienna, Vienna, Austria
| | - Hendrik P N Scholl
- Institute of Molecular and Clinical Ophthalmology Basel, Basel, Switzerland
- Department of Ophthalmology, University of Basel, Basel, Switzerland
| | - Sobha Sivaprasad
- NIHR Moorfields Biomedical Research Centre, Moorfields Eye Hospital NHS Foundation Trust, London, UK
| | - Daniel Rueckert
- BioMedIA, Imperial College London, London, UK
- Institute for AI and Informatics in Medicine, Klinikum rechts der Isar, Technical University Munich, Munich, Germany
| | - Andrew Lotery
- Clinical and Experimental Sciences, Faculty of Medicine, University of Southampton, Southampton, UK
| | - Ursula Schmidt-Erfurth
- Department of Ophthalmology and Optometry, Medical University of Vienna, Vienna, Austria
| | - Hrvoje Bogunović
- Department of Ophthalmology and Optometry, Medical University of Vienna, Vienna, Austria.
- Christian Doppler Lab for Artificial Intelligence in Retina, Department of Ophthalmology and Optometry, Medical University of Vienna, Vienna, Austria.
| |
Collapse
|
15
|
Baharlouei Z, Rabbani H, Plonka G. Wavelet scattering transform application in classification of retinal abnormalities using OCT images. Sci Rep 2023; 13:19013. [PMID: 37923770 PMCID: PMC10624695 DOI: 10.1038/s41598-023-46200-1] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/10/2022] [Accepted: 10/29/2023] [Indexed: 11/06/2023] Open
Abstract
To assist ophthalmologists in diagnosing retinal abnormalities, Computer Aided Diagnosis has played a significant role. In this paper, a particular Convolutional Neural Network based on Wavelet Scattering Transform (WST) is used to detect one to four retinal abnormalities from Optical Coherence Tomography (OCT) images. Predefined wavelet filters in this network decrease the computation complexity and processing time compared to deep learning methods. We use two layers of the WST network to obtain a direct and efficient model. WST generates a sparse representation of the images which is translation-invariant and stable concerning local deformations. Next, a Principal Component Analysis classifies the extracted features. We evaluate the model using four publicly available datasets to have a comprehensive comparison with the literature. The accuracies of classifying the OCT images of the OCTID dataset into two and five classes were [Formula: see text] and [Formula: see text], respectively. We achieved an accuracy of [Formula: see text] in detecting Diabetic Macular Edema from Normal ones using the TOPCON device-based dataset. Heidelberg and Duke datasets contain DME, Age-related Macular Degeneration, and Normal classes, in which we achieved accuracy of [Formula: see text] and [Formula: see text], respectively. A comparison of our results with the state-of-the-art models shows that our model outperforms these models for some assessments or achieves nearly the best results reported so far while having a much smaller computational complexity.
Collapse
Affiliation(s)
- Zahra Baharlouei
- Medical Image and Signal Processing Research Center, School of Advanced Technologies in Medicine, Isfahan University of Medical Sciences, Isfahan, Iran
| | - Hossein Rabbani
- Medical Image and Signal Processing Research Center, School of Advanced Technologies in Medicine, Isfahan University of Medical Sciences, Isfahan, Iran.
| | - Gerlind Plonka
- Institute for Numerical and Applied Mathematics, Georg-August-University of Goettingen, Göttingen, Germany
| |
Collapse
|
16
|
Bui PN, Le DT, Bum J, Kim S, Song SJ, Choo H. Multi-Scale Learning with Sparse Residual Network for Explainable Multi-Disease Diagnosis in OCT Images. Bioengineering (Basel) 2023; 10:1249. [PMID: 38002373 PMCID: PMC10669434 DOI: 10.3390/bioengineering10111249] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/02/2023] [Revised: 10/19/2023] [Accepted: 10/23/2023] [Indexed: 11/26/2023] Open
Abstract
In recent decades, medical imaging techniques have revolutionized the field of disease diagnosis, enabling healthcare professionals to noninvasively observe the internal structures of the human body. Among these techniques, optical coherence tomography (OCT) has emerged as a powerful and versatile tool that allows high-resolution, non-invasive, and real-time imaging of biological tissues. Deep learning algorithms have been successfully employed to detect and classify various retinal diseases in OCT images, enabling early diagnosis and treatment planning. However, existing deep learning algorithms are primarily designed for single-disease diagnosis, which limits their practical application in clinical settings where OCT images often contain symptoms of multiple diseases. In this paper, we propose an effective approach for multi-disease diagnosis in OCT images using a multi-scale learning (MSL) method and a sparse residual network (SRN). Specifically, the MSL method extracts and fuses useful features from images of different sizes to enhance the discriminative capability of a classifier and make the disease predictions interpretable. The SRN is a minimal residual network, where convolutional layers with large kernel sizes are replaced with multiple convolutional layers that have smaller kernel sizes, thereby reducing model complexity while achieving a performance similar to that of existing convolutional neural networks. The proposed multi-scale sparse residual network significantly outperforms existing methods, exhibiting 97.40% accuracy, 95.38% sensitivity, and 98.25% specificity. Experimental results show the potential of our method to improve explainable diagnosis systems for various eye diseases via visual discrimination.
Collapse
Affiliation(s)
- Phuoc-Nguyen Bui
- Department of AI Systems Engineering, Sungkyunkwan University, Suwon 16419, Republic of Korea;
| | - Duc-Tai Le
- College of Computing and Informatics, Sungkyunkwan University, Suwon 16419, Republic of Korea;
| | - Junghyun Bum
- Sungkyun AI Research Institute, Sungkyunkwan University, Suwon 16419, Republic of Korea;
| | - Seongho Kim
- Department of Ophthalmology, Kangbuk Samsung Hospital, School of Medicine, Sungkyunkwan University, Seoul 03181, Republic of Korea;
| | - Su Jeong Song
- Department of Ophthalmology, Kangbuk Samsung Hospital, School of Medicine, Sungkyunkwan University, Seoul 03181, Republic of Korea;
- Biomedical Institute for Convergence, Sungkyunkwan University, Suwon 16419, Republic of Korea
| | - Hyunseung Choo
- Department of AI Systems Engineering, Sungkyunkwan University, Suwon 16419, Republic of Korea;
- College of Computing and Informatics, Sungkyunkwan University, Suwon 16419, Republic of Korea;
- Department of Electrical and Computer Engineering, Sungkyunkwan University, Suwon 16419, Republic of Korea
| |
Collapse
|
17
|
Wei S, Liu Y, Bian Z, Wang Y, Zuo L, Calabresi PA, Saidha S, Prince JL, Carass A. Recurrent Self Fusion: Iterative Denoising for Consistent Retinal OCT Segmentation. OPHTHALMIC MEDICAL IMAGE ANALYSIS : 10TH INTERNATIONAL WORKSHOP, OMIA 2023, HELD IN CONJUNCTION WITH MICCAI 2023, VANCOUVER, BC, CANADA, OCTOBER 12, 2023, PROCEEDINGS. OMIA (WORKSHOP) (10TH : 2023 : VANCOUVER, B.C. ; ONLINE) 2023; 14096:42-51. [PMID: 38318463 PMCID: PMC10840975 DOI: 10.1007/978-3-031-44013-7_5] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 02/07/2024]
Abstract
Optical coherence tomography (OCT) is a valuable imaging technique in ophthalmology, providing high-resolution, cross-sectional images of the retina for early detection and monitoring of various retinal and neurological diseases. However, discrepancies in retinal layer thickness measurements among different OCT devices pose challenges for data comparison and interpretation, particularly in longitudinal analyses. This work introduces the idea of a recurrent self fusion (RSF) algorithm to address this issue. Our RSF algorithm, built upon the self fusion methodology, iteratively denoises retinal OCT images. A deep learning-based retinal OCT segmentation algorithm is employed for downstream analyses. A large dataset of paired OCT scans acquired on both a Spectralis and Cirrus OCT device are used for validation. The results demonstrate that the RSF algorithm effectively reduces speckle contrast and enhances the consistency of retinal OCT segmentation.
Collapse
Affiliation(s)
- Shuwen Wei
- Department of Electrical and Computer Engineering, Johns Hopkins University, Baltimore, MD 21218, USA
| | - Yihao Liu
- Department of Electrical and Computer Engineering, Johns Hopkins University, Baltimore, MD 21218, USA
| | - Zhangxing Bian
- Department of Electrical and Computer Engineering, Johns Hopkins University, Baltimore, MD 21218, USA
| | - Yuli Wang
- Department of Biomedical Engineering, Johns Hopkins University School of Medicine, Baltimore, MD 21287, USA
| | - Lianrui Zuo
- Department of Electrical and Computer Engineering, Johns Hopkins University, Baltimore, MD 21218, USA
- Laboratory of Behavioral Neuroscience, National Institute on Aging, National Institutes of Health, Baltimore, MD 21224, USA
| | - Peter A Calabresi
- Department of Neurology, Johns Hopkins University School of Medicine, Baltimore, MD 21287, USA
| | - Shiv Saidha
- Department of Neurology, Johns Hopkins University School of Medicine, Baltimore, MD 21287, USA
| | - Jerry L Prince
- Department of Electrical and Computer Engineering, Johns Hopkins University, Baltimore, MD 21218, USA
| | - Aaron Carass
- Department of Electrical and Computer Engineering, Johns Hopkins University, Baltimore, MD 21218, USA
| |
Collapse
|
18
|
Koseoglu ND, Grzybowski A, Liu TYA. Deep Learning Applications to Classification and Detection of Age-Related Macular Degeneration on Optical Coherence Tomography Imaging: A Review. Ophthalmol Ther 2023; 12:2347-2359. [PMID: 37493854 PMCID: PMC10441995 DOI: 10.1007/s40123-023-00775-0] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/29/2023] [Accepted: 07/14/2023] [Indexed: 07/27/2023] Open
Abstract
Age-related macular degeneration (AMD) is one of the leading causes of blindness in the elderly, more commonly in developed countries. Optical coherence tomography (OCT) is a non-invasive imaging device widely used for the diagnosis and management of AMD. Deep learning (DL) uses multilayered artificial neural networks (NN) for feature extraction, and is the cutting-edge technique for medical image analysis for diagnostic and prognostication purposes. Application of DL models to OCT image analysis has garnered significant interest in recent years. In this review, we aimed to summarize studies focusing on DL models used in classification and detection of AMD. Additionally, we provide a brief introduction to other DL applications in AMD, such as segmentation, prediction/prognostication, and models trained on multimodal imaging.
Collapse
Affiliation(s)
- Neslihan Dilruba Koseoglu
- Wilmer Eye Institute, Johns Hopkins University, 600 N. Wolfe St., Maumenee 726, Baltimore, MD, 21287, USA
| | - Andrzej Grzybowski
- Institute for Research in Ophthalmology, Foundation for Ophthalmology Development, Poznan, Poland
| | - T Y Alvin Liu
- Wilmer Eye Institute, Johns Hopkins University, 600 N. Wolfe St., Maumenee 726, Baltimore, MD, 21287, USA.
| |
Collapse
|
19
|
Ait Hammou B, Antaki F, Boucher MC, Duval R. MBT: Model-Based Transformer for retinal optical coherence tomography image and video multi-classification. Int J Med Inform 2023; 178:105178. [PMID: 37657204 DOI: 10.1016/j.ijmedinf.2023.105178] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/10/2023] [Revised: 07/13/2023] [Accepted: 08/06/2023] [Indexed: 09/03/2023]
Abstract
BACKGROUND AND OBJECTIVE The detection of retinal diseases using optical coherence tomography (OCT) images and videos is a concrete example of a data classification problem. In recent years, Transformer architectures have been successfully applied to solve a variety of real-world classification problems. Although they have shown impressive discriminative abilities compared to other state-of-the-art models, improving their performance is essential, especially in healthcare-related problems. METHODS This paper presents an effective technique named model-based transformer (MBT). It is based on popular pre-trained transformer models, particularly, vision transformer, swin transformer for OCT image classification, and multiscale vision transformer for OCT video classification. The proposed approach is designed to represent OCT data by taking advantage of an approximate sparse representation technique. Then, it estimates the optimal features, and performs data classification. RESULTS The experiments are carried out using three real-world retinal datasets. The experimental results on OCT image and OCT video datasets show that the proposed method outperforms existing state-of-the-art deep learning approaches in terms of classification accuracy, precision, recall, and f1-score, kappa, AUC-ROC, and AUC-PR. It can also boost the performance of existing transformer models, including Vision transformer and Swin transformer for OCT image classification, and Multiscale Vision Transformers for OCT video classification. CONCLUSIONS This work presents an approach for the automated detection of retinal diseases. Although deep neural networks have proven great potential in ophthalmology applications, our findings demonstrate for the first time a new way to identify retinal pathologies using OCT videos instead of images. Moreover, our proposal can help researchers enhance the discriminative capacity of a variety of powerful deep learning models presented in published papers. This can be valuable for future directions in medical research and clinical practice.
Collapse
Affiliation(s)
- Badr Ait Hammou
- Department of Ophthalmology, Université de Montréal, Montreal, Québec, Canada; Centre Universitaire d'Ophtalmologie (CUO), Hôpital Maisonneuve-Rosemont, CIUSSS de l'Est-de-l'Île-de-Montréal, Montréal, Québec, Canada.
| | - Fares Antaki
- Department of Ophthalmology, Université de Montréal, Montreal, Québec, Canada; Centre Universitaire d'Ophtalmologie (CUO), Hôpital Maisonneuve-Rosemont, CIUSSS de l'Est-de-l'Île-de-Montréal, Montréal, Québec, Canada; Department of Ophthalmology, Centre Hospitalier de l'Université de Montréal (CHUM), Montreal, Quebec, Canada
| | - Marie-Carole Boucher
- Department of Ophthalmology, Université de Montréal, Montreal, Québec, Canada; Centre Universitaire d'Ophtalmologie (CUO), Hôpital Maisonneuve-Rosemont, CIUSSS de l'Est-de-l'Île-de-Montréal, Montréal, Québec, Canada
| | - Renaud Duval
- Department of Ophthalmology, Université de Montréal, Montreal, Québec, Canada; Centre Universitaire d'Ophtalmologie (CUO), Hôpital Maisonneuve-Rosemont, CIUSSS de l'Est-de-l'Île-de-Montréal, Montréal, Québec, Canada
| |
Collapse
|
20
|
Akinniyi O, Rahman MM, Sandhu HS, El-Baz A, Khalifa F. Multi-Stage Classification of Retinal OCT Using Multi-Scale Ensemble Deep Architecture. Bioengineering (Basel) 2023; 10:823. [PMID: 37508850 PMCID: PMC10376573 DOI: 10.3390/bioengineering10070823] [Citation(s) in RCA: 2] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/30/2023] [Revised: 07/01/2023] [Accepted: 07/07/2023] [Indexed: 07/30/2023] Open
Abstract
Accurate noninvasive diagnosis of retinal disorders is required for appropriate treatment or precision medicine. This work proposes a multi-stage classification network built on a multi-scale (pyramidal) feature ensemble architecture for retinal image classification using optical coherence tomography (OCT) images. First, a scale-adaptive neural network is developed to produce multi-scale inputs for feature extraction and ensemble learning. The larger input sizes yield more global information, while the smaller input sizes focus on local details. Then, a feature-rich pyramidal architecture is designed to extract multi-scale features as inputs using DenseNet as the backbone. The advantage of the hierarchical structure is that it allows the system to extract multi-scale, information-rich features for the accurate classification of retinal disorders. Evaluation on two public OCT datasets containing normal and abnormal retinas (e.g., diabetic macular edema (DME), choroidal neovascularization (CNV), age-related macular degeneration (AMD), and Drusen) and comparison against recent networks demonstrates the advantages of the proposed architecture's ability to produce feature-rich classification with average accuracy of 97.78%, 96.83%, and 94.26% for the first (binary) stage, second (three-class) stage, and all-at-once (four-class) classification, respectively, using cross-validation experiments using the first dataset. In the second dataset, our system showed an overall accuracy, sensitivity, and specificity of 99.69%, 99.71%, and 99.87%, respectively. Overall, the tangible advantages of the proposed network for enhanced feature learning might be used in various medical image classification tasks where scale-invariant features are crucial for precise diagnosis.
Collapse
Affiliation(s)
- Oluwatunmise Akinniyi
- Department of Computer Science, School of Computer, Mathematical and Natural Sciences, Morgan State University, Baltimore, MD 21251, USA
| | - Md Mahmudur Rahman
- Department of Computer Science, School of Computer, Mathematical and Natural Sciences, Morgan State University, Baltimore, MD 21251, USA
| | - Harpal Singh Sandhu
- Bioengineering Department, University of Louisville, Louisville, KY 20292, USA
| | - Ayman El-Baz
- Bioengineering Department, University of Louisville, Louisville, KY 20292, USA
| | - Fahmi Khalifa
- Electronics and Communications Engineering Department, Mansoura University, Mansoura 35516, Egypt
- Electrical and Computer Engineering Department, Morgan State University, Baltimore MD 21251, USA
| |
Collapse
|
21
|
Diao S, Su J, Yang C, Zhu W, Xiang D, Chen X, Peng Q, Shi F. Classification and segmentation of OCT images for age-related macular degeneration based on dual guidance networks. Biomed Signal Process Control 2023. [DOI: 10.1016/j.bspc.2023.104810] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 03/13/2023]
|
22
|
Moradi M, Chen Y, Du X, Seddon JM. Deep ensemble learning for automated non-advanced AMD classification using optimized retinal layer segmentation and SD-OCT scans. Comput Biol Med 2023; 154:106512. [PMID: 36701964 DOI: 10.1016/j.compbiomed.2022.106512] [Citation(s) in RCA: 5] [Impact Index Per Article: 5.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/23/2022] [Revised: 11/30/2022] [Accepted: 12/31/2022] [Indexed: 01/11/2023]
Abstract
BACKGROUND Accurate retinal layer segmentation in optical coherence tomography (OCT) images is crucial for quantitatively analyzing age-related macular degeneration (AMD) and monitoring its progression. However, previous retinal segmentation models depend on experienced experts and manually annotating retinal layers is time-consuming. On the other hand, accuracy of AMD diagnosis is directly related to the segmentation model's performance. To address these issues, we aimed to improve AMD detection using optimized retinal layer segmentation and deep ensemble learning. METHOD We integrated a graph-cut algorithm with a cubic spline to automatically annotate 11 retinal boundaries. The refined images were fed into a deep ensemble mechanism that combined a Bagged Tree and end-to-end deep learning classifiers. We tested the developed deep ensemble model on internal and external datasets. RESULTS The total error rates for our segmentation model using the boundary refinement approach was significantly lower than OCT Explorer segmentations (1.7% vs. 7.8%, p-value = 0.03). We utilized the refinement approach to quantify 169 imaging features using Zeiss SD-OCT volume scans. The presence of drusen and thickness of total retina, neurosensory retina, and ellipsoid zone to inner-outer segment (EZ-ISOS) thickness had higher contributions to AMD classification compared to other features. The developed ensemble learning model obtained a higher diagnostic accuracy in a shorter time compared with two human graders. The area under the curve (AUC) for normal vs. early AMD was 99.4%. CONCLUSION Testing results showed that the developed framework is repeatable and effective as a potentially valuable tool in retinal imaging research.
Collapse
Affiliation(s)
- Mousa Moradi
- Department of Biomedical Engineering, University of Massachusetts, Amherst, MA, United States
| | - Yu Chen
- Department of Biomedical Engineering, University of Massachusetts, Amherst, MA, United States.
| | - Xian Du
- Department of Mechanical and Industrial Engineering, University of Massachusetts, Amherst, MA, United States.
| | - Johanna M Seddon
- Department of Ophthalmology & Visual Sciences, University of Massachusetts Chan Medical School, Worcester, MA, United States.
| |
Collapse
|
23
|
Liew A, Agaian S, Benbelkacem S. Distinctions between Choroidal Neovascularization and Age Macular Degeneration in Ocular Disease Predictions via Multi-Size Kernels ξcho-Weighted Median Patterns. Diagnostics (Basel) 2023; 13:diagnostics13040729. [PMID: 36832215 PMCID: PMC9956029 DOI: 10.3390/diagnostics13040729] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/28/2022] [Revised: 12/31/2022] [Accepted: 01/05/2023] [Indexed: 02/17/2023] Open
Abstract
Age-related macular degeneration is a visual disorder caused by abnormalities in a part of the eye's retina and is a leading source of blindness. The correct detection, precise location, classification, and diagnosis of choroidal neovascularization (CNV) may be challenging if the lesion is small or if Optical Coherence Tomography (OCT) images are degraded by projection and motion. This paper aims to develop an automated quantification and classification system for CNV in neovascular age-related macular degeneration using OCT angiography images. OCT angiography is a non-invasive imaging tool that visualizes retinal and choroidal physiological and pathological vascularization. The presented system is based on new retinal layers in the OCT image-specific macular diseases feature extractor, including Multi-Size Kernels ξcho-Weighted Median Patterns (MSKξMP). Computer simulations show that the proposed method: (i) outperforms current state-of-the-art methods, including deep learning techniques; and (ii) achieves an overall accuracy of 99% using ten-fold cross-validation on the Duke University dataset and over 96% on the noisy Noor Eye Hospital dataset. In addition, MSKξMP performs well in binary eye disease classifications and is more accurate than recent works in image texture descriptors.
Collapse
Affiliation(s)
- Alex Liew
- Department of Computer Science, Graduate Center of City University New York, 365 5th Ave., New York, NY 10016, USA
- Correspondence:
| | - Sos Agaian
- Department of Computer Science, Graduate Center of City University New York, 365 5th Ave., New York, NY 10016, USA
| | - Samir Benbelkacem
- Robotics and Industrial Automation Division, Centre de Développement des Technologies Avancées (CDTA), Algiers 16081, Algeria
| |
Collapse
|
24
|
Wang S, Ji Y, Bai W, Ji Y, Li J, Yao Y, Zhang Z, Jiang Q, Li K. Advances in artificial intelligence models and algorithms in the field of optometry. Front Cell Dev Biol 2023; 11:1170068. [PMID: 37187617 PMCID: PMC10175695 DOI: 10.3389/fcell.2023.1170068] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/27/2023] [Accepted: 04/17/2023] [Indexed: 05/17/2023] Open
Abstract
The rapid development of computer science over the past few decades has led to unprecedented progress in the field of artificial intelligence (AI). Its wide application in ophthalmology, especially image processing and data analysis, is particularly extensive and its performance excellent. In recent years, AI has been increasingly applied in optometry with remarkable results. This review is a summary of the application progress of different AI models and algorithms used in optometry (for problems such as myopia, strabismus, amblyopia, keratoconus, and intraocular lens) and includes a discussion of the limitations and challenges associated with its application in this field.
Collapse
Affiliation(s)
- Suyu Wang
- Department of Ophthalmology, The Affiliated Eye Hospital of Nanjing Medical University, Nanjing, China
- The Fourth School of Clinical Medicine, Nanjing Medical University, Nanjing, China
| | - Yuke Ji
- Department of Ophthalmology, The Affiliated Eye Hospital of Nanjing Medical University, Nanjing, China
- The Fourth School of Clinical Medicine, Nanjing Medical University, Nanjing, China
| | - Wen Bai
- Department of Ophthalmology, The Affiliated Eye Hospital of Nanjing Medical University, Nanjing, China
- The Fourth School of Clinical Medicine, Nanjing Medical University, Nanjing, China
| | - Yun Ji
- Affiliated Hospital of Shandong University of Traditional Chinese Medicine, Jinan, Shandong, China
| | - Jiajun Li
- Department of Ophthalmology, The Affiliated Eye Hospital of Nanjing Medical University, Nanjing, China
- The Fourth School of Clinical Medicine, Nanjing Medical University, Nanjing, China
| | - Yujia Yao
- Department of Ophthalmology, The Affiliated Eye Hospital of Nanjing Medical University, Nanjing, China
- The Fourth School of Clinical Medicine, Nanjing Medical University, Nanjing, China
| | - Ziran Zhang
- Department of Ophthalmology, The Affiliated Eye Hospital of Nanjing Medical University, Nanjing, China
- The Fourth School of Clinical Medicine, Nanjing Medical University, Nanjing, China
| | - Qin Jiang
- Department of Ophthalmology, The Affiliated Eye Hospital of Nanjing Medical University, Nanjing, China
- The Fourth School of Clinical Medicine, Nanjing Medical University, Nanjing, China
- *Correspondence: Qin Jiang, ; Keran Li,
| | - Keran Li
- Department of Ophthalmology, The Affiliated Eye Hospital of Nanjing Medical University, Nanjing, China
- The Fourth School of Clinical Medicine, Nanjing Medical University, Nanjing, China
- *Correspondence: Qin Jiang, ; Keran Li,
| |
Collapse
|
25
|
A Multi-centric Evaluation of Deep Learning Models for Segmentation of COVID-19 Lung Lesions on Chest CT Scans. IRANIAN JOURNAL OF RADIOLOGY 2022. [DOI: 10.5812/iranjradiol-117992] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 11/19/2022]
Abstract
Background: Chest computed tomography (CT) scan is one of the most common tools used for the diagnosis of patients with coronavirus disease 2019 (COVID-19). While segmentation of COVID-19 lung lesions by radiologists can be time-consuming, the application of advanced deep learning techniques for automated segmentation can be a promising step toward the management of this infection and similar diseases in the future. Objectives: This study aimed to evaluate the performance and generalizability of deep learning-based models for the automated segmentation of COVID-19 lung lesions. Patients and Methods: Four datasets (2 private and 2 public) were used in this study. The first and second private datasets included 297 (147 healthy and 150 COVID-19 cases) and 82 COVID-19 subjects. The public datasets included the COVID19-P20 (20 COVID-19 cases from 2 centers) and the MosMedData datasets (50 COVID-19 patients from a single center). Model comparisons were made based on the Dice similarity coefficient (DSC), receiver operating characteristic (ROC) curve, and area under the curve (AUC). The predicted CT severity scores by the model were compared with those of radiologists by measuring the Pearson’s correlation coefficients (PCC). Also, DSC was used to compare the inter-rater agreement of the model and expert against that of 2 experts on an unseen dataset. Finally, the generalizability of the model was evaluated, and a simple calibration strategy was proposed. Results: The VGG16-UNet model showed the best performance across both private datasets, with a DSC of 84.23% ± 1.73% on the first private dataset and 56.61% ± 1.48% on the second private dataset. Similar results were obtained on public datasets, with a DSC of 60.10% ± 2.34% on the COVID19-P20 dataset and 66.28% ± 2.80% on a combined dataset of COVID19-P20 and MosMedData. The predicted CT severity scores of the model were compared against those of radiologists and were found to be 0.89 and 0.85 on the first private dataset and 0.77 and 0.74 on the second private dataset for the right and left lungs, respectively. Moreover, the model trained on the first private dataset was examined on the second private dataset and compared against the radiologist, which revealed a performance gap of 5.74% based on DSCs. A calibration strategy was employed to reduce this gap to 0.53%. Conclusion: The results demonstrated the potential of the proposed model in localizing COVID-19 lesions on CT scans across multiple datasets; its accuracy competed with the radiologists and could assist them in diagnostic and treatment procedures. The effect of model calibration on the performance of an unseen dataset was also reported, increasing the DSC by more than 5%.
Collapse
|