1
|
Quan X, Ou X, Gao L, Yin W, Hou G, Zhang H. SCINet: A Segmentation and Classification Interaction CNN Method for Arteriosclerotic Retinopathy Grading. Interdiscip Sci 2024:10.1007/s12539-024-00650-x. [PMID: 39222258 DOI: 10.1007/s12539-024-00650-x] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/21/2024] [Revised: 08/09/2024] [Accepted: 08/12/2024] [Indexed: 09/04/2024]
Abstract
As a common disease, cardiovascular and cerebrovascular diseases pose a great harm threat to human wellness. Even using advanced and comprehensive treatment methods, there is still a high mortality rate. Arteriosclerosis, as an important factor reflecting the severity of cardiovascular and cerebrovascular diseases, is imperative to detect the arteriosclerotic retinopathy. However, the detection of arteriosclerosis retinopathy requires expensive and time-consuming manual evaluation, while end-to-end deep learning detection methods also need interpretable design to high light task-related features. Considering the importance of automatic arteriosclerotic retinopathy grading, we propose a segmentation and classification interaction network (SCINet). We propose a segmentation and classification interaction architecture for grading arteriosclerotic retinopathy. After IterNet is used to segment retinal vessel from original fundus images, the backbone feature extractor roughly extracts features from the segmented and original fundus arteriosclerosis images and further enhances them through the vessel aware module. The last classifier module generates fundus arteriosclerosis grading results. Specifically, the vessel aware module is designed to highlight the important areal vessel features segmented from original images by attention mechanism, thereby achieving information interaction. The attention mechanism selectively learns the vessel features of segmentation region information under the proposed interactive architecture, which leads to reweighting the extracted features and enhances significant feature information. Extensive experiments have confirmed the effect of our model. SCINet has the best performance on the task of arteriosclerotic retinopathy grading. Additionally, the CNN method is scalable to similar tasks by incorporating segmented images as auxiliary information.
Collapse
Affiliation(s)
- Xiongwen Quan
- National Key Laboratory of Intelligent Tracking and Forecasting for Infectious Diseases, Engineering Research Center of Trusted Behavior Intelligence, Ministry of Education, College of Artificial Intelligence, Nankai University, Tianjin, 300000, China
| | - Xingyuan Ou
- National Key Laboratory of Intelligent Tracking and Forecasting for Infectious Diseases, Engineering Research Center of Trusted Behavior Intelligence, Ministry of Education, College of Artificial Intelligence, Nankai University, Tianjin, 300000, China
| | - Li Gao
- Ophthalmology, Tianjin Huanhu Hospital, Tianjin, 300000, China
| | - Wenya Yin
- National Key Laboratory of Intelligent Tracking and Forecasting for Infectious Diseases, Engineering Research Center of Trusted Behavior Intelligence, Ministry of Education, College of Artificial Intelligence, Nankai University, Tianjin, 300000, China
| | - Guangyao Hou
- National Key Laboratory of Intelligent Tracking and Forecasting for Infectious Diseases, Engineering Research Center of Trusted Behavior Intelligence, Ministry of Education, College of Artificial Intelligence, Nankai University, Tianjin, 300000, China
| | - Han Zhang
- National Key Laboratory of Intelligent Tracking and Forecasting for Infectious Diseases, Engineering Research Center of Trusted Behavior Intelligence, Ministry of Education, College of Artificial Intelligence, Nankai University, Tianjin, 300000, China.
| |
Collapse
|
2
|
Kalupahana D, Kahatapitiya NS, Silva BN, Kim J, Jeon M, Wijenayake U, Wijesinghe RE. Dense Convolutional Neural Network-Based Deep Learning Pipeline for Pre-Identification of Circular Leaf Spot Disease of Diospyros kaki Leaves Using Optical Coherence Tomography. SENSORS (BASEL, SWITZERLAND) 2024; 24:5398. [PMID: 39205092 PMCID: PMC11359294 DOI: 10.3390/s24165398] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 06/21/2024] [Revised: 07/30/2024] [Accepted: 08/05/2024] [Indexed: 09/04/2024]
Abstract
Circular leaf spot (CLS) disease poses a significant threat to persimmon cultivation, leading to substantial harvest reductions. Existing visual and destructive inspection methods suffer from subjectivity, limited accuracy, and considerable time consumption. This study presents an automated pre-identification method of the disease through a deep learning (DL) based pipeline integrated with optical coherence tomography (OCT), thereby addressing the highlighted issues with the existing methods. The investigation yielded promising outcomes by employing transfer learning with pre-trained DL models, specifically DenseNet-121 and VGG-16. The DenseNet-121 model excels in differentiating among three stages of CLS disease (healthy (H), apparently healthy (or healthy-infected (HI)), and infected (I)). The model achieved precision values of 0.7823 for class-H, 0.9005 for class-HI, and 0.7027 for class-I, supported by recall values of 0.8953 for class-HI and 0.8387 for class-I. Moreover, the performance of CLS detection was enhanced by a supplemental quality inspection model utilizing VGG-16, which attained an accuracy of 98.99% in discriminating between low-detail and high-detail images. Moreover, this study employed a combination of LAMP and A-scan for the dataset labeling process, significantly enhancing the accuracy of the models. Overall, this study underscores the potential of DL techniques integrated with OCT to enhance disease identification processes in agricultural settings, particularly in persimmon cultivation, by offering efficient and objective pre-identification of CLS and enabling early intervention and management strategies.
Collapse
Affiliation(s)
- Deshan Kalupahana
- Department of Computer Engineering, Faculty of Engineering, University of Sri Jayewardenepura, Nugegoda 10250, Sri Lanka; (D.K.); (N.S.K.)
| | - Nipun Shantha Kahatapitiya
- Department of Computer Engineering, Faculty of Engineering, University of Sri Jayewardenepura, Nugegoda 10250, Sri Lanka; (D.K.); (N.S.K.)
| | - Bhagya Nathali Silva
- Department of Information Technology, Faculty of Computing, Sri Lanka Institute of Information Technology, Malabe 10115, Sri Lanka;
- Center for Excellence in Informatics, Electronics & Transmission (CIET), Sri Lanka Institute of Information Technology, Malabe 10115, Sri Lanka
| | - Jeehyun Kim
- School of Electronic and Electrical Engineering, College of IT Engineering, Kyungpook National University, 80 Daehak-ro, Buk-gu, Daegu 41566, Republic of Korea; (J.K.); (M.J.)
| | - Mansik Jeon
- School of Electronic and Electrical Engineering, College of IT Engineering, Kyungpook National University, 80 Daehak-ro, Buk-gu, Daegu 41566, Republic of Korea; (J.K.); (M.J.)
| | - Udaya Wijenayake
- Department of Computer Engineering, Faculty of Engineering, University of Sri Jayewardenepura, Nugegoda 10250, Sri Lanka; (D.K.); (N.S.K.)
| | - Ruchire Eranga Wijesinghe
- Center for Excellence in Informatics, Electronics & Transmission (CIET), Sri Lanka Institute of Information Technology, Malabe 10115, Sri Lanka
- Department of Electrical and Electronic Engineering, Faculty of Engineering, Sri Lanka Institute of Information Technology, Malabe 10115, Sri Lanka
| |
Collapse
|
3
|
Abd El-Khalek AA, Balaha HM, Sewelam A, Ghazal M, Khalil AT, Abo-Elsoud MEA, El-Baz A. A Comprehensive Review of AI Diagnosis Strategies for Age-Related Macular Degeneration (AMD). Bioengineering (Basel) 2024; 11:711. [PMID: 39061793 PMCID: PMC11273790 DOI: 10.3390/bioengineering11070711] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/12/2024] [Revised: 07/02/2024] [Accepted: 07/09/2024] [Indexed: 07/28/2024] Open
Abstract
The rapid advancement of computational infrastructure has led to unprecedented growth in machine learning, deep learning, and computer vision, fundamentally transforming the analysis of retinal images. By utilizing a wide array of visual cues extracted from retinal fundus images, sophisticated artificial intelligence models have been developed to diagnose various retinal disorders. This paper concentrates on the detection of Age-Related Macular Degeneration (AMD), a significant retinal condition, by offering an exhaustive examination of recent machine learning and deep learning methodologies. Additionally, it discusses potential obstacles and constraints associated with implementing this technology in the field of ophthalmology. Through a systematic review, this research aims to assess the efficacy of machine learning and deep learning techniques in discerning AMD from different modalities as they have shown promise in the field of AMD and retinal disorders diagnosis. Organized around prevalent datasets and imaging techniques, the paper initially outlines assessment criteria, image preprocessing methodologies, and learning frameworks before conducting a thorough investigation of diverse approaches for AMD detection. Drawing insights from the analysis of more than 30 selected studies, the conclusion underscores current research trajectories, major challenges, and future prospects in AMD diagnosis, providing a valuable resource for both scholars and practitioners in the domain.
Collapse
Affiliation(s)
- Aya A. Abd El-Khalek
- Communications and Electronics Engineering Department, Nile Higher Institute for Engineering and Technology, Mansoura 35511, Egypt;
| | - Hossam Magdy Balaha
- Department of Bioengineering, J.B. Speed School of Engineering, University of Louisville, Louisville, KY 40292, USA
| | - Ashraf Sewelam
- Ophthalmology Department, Faculty of Medicine, Mansoura University, Mansoura 35511, Egypt;
| | - Mohammed Ghazal
- Electrical, Computer, and Biomedical Engineering Department, Abu Dhabi University, Abu Dhabi 59911, United Arab Emirates;
| | - Abeer T. Khalil
- Communications and Electronics Engineering Department, Faculty of Engineering, Mansoura University, Mansoura 35511, Egypt; (A.T.K.); (M.E.A.A.-E.)
| | - Mohy Eldin A. Abo-Elsoud
- Communications and Electronics Engineering Department, Faculty of Engineering, Mansoura University, Mansoura 35511, Egypt; (A.T.K.); (M.E.A.A.-E.)
| | - Ayman El-Baz
- Department of Bioengineering, J.B. Speed School of Engineering, University of Louisville, Louisville, KY 40292, USA
| |
Collapse
|
4
|
Gueddena Y, Aboudi N, Zgolli H, Mabrouk S, Sidibe D, Tabia H, Khlifa N. A new intelligent system based deep learning to detect DME and AMD in OCT images. Int Ophthalmol 2024; 44:191. [PMID: 38653842 DOI: 10.1007/s10792-024-03115-8] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/06/2023] [Accepted: 03/24/2024] [Indexed: 04/25/2024]
Abstract
Optical Coherence Tomography (OCT) is widely recognized as the leading modality for assessing ocular retinal diseases, playing a crucial role in diagnosing retinopathy while maintaining a non-invasive modality. The increasing volume of OCT images underscores the growing importance of automating image analysis. Age-related diabetic Macular Degeneration (AMD) and Diabetic Macular Edema (DME) are the most common cause of visual impairment. Early detection and timely intervention for diabetes-related conditions are essential for preventing optical complications and reducing the risk of blindness. This study introduces a novel Computer-Aided Diagnosis (CAD) system based on a Convolutional Neural Network (CNN) model, aiming to identify and classify OCT retinal images into AMD, DME, and Normal classes. Leveraging CNN efficiency, including feature learning and classification, various CNN, including pre-trained VGG16, VGG19, Inception_V3, a custom from scratch model, BCNN (VGG16)2 , BCNN (VGG19)2 , and BCNN (Inception_V3)2 , are developed for the classification of AMD, DME, and Normal OCT images. The proposed approach has been evaluated on two datasets, including a DUKE public dataset and a Tunisian private dataset. The combination of the Inception_V3 model and the extracted feature from the proposed custom CNN achieved the highest accuracy value of 99.53% in the DUKE dataset. The obtained results on DUKE public and Tunisian datasets demonstrate the proposed approach as a significant tool for efficient and automatic retinal OCT image classification.
Collapse
Affiliation(s)
- Yassmine Gueddena
- Laboratory of Biophysics and Medical Technologies, Higher Institute of Medical Technologies of Tunis, University of Tuins El Manar, 1006, Tunis, Tunisia
| | - Noura Aboudi
- Laboratory of Biophysics and Medical Technologies, National Engineering School of Carthage, 2035, Tunis, Tunisia.
| | - Hsouna Zgolli
- Department A, Hedi Raies of Ophthalmology Institute, Tunis, Tunisia
| | - Sonia Mabrouk
- Department A, Hedi Raies of Ophthalmology Institute, Tunis, Tunisia
| | - Désiré Sidibe
- IBISC Laboratory, University of Paris-Saclay, Evry, France
| | - Hedi Tabia
- IBISC Laboratory, University of Paris-Saclay, Evry, France
| | - Nawres Khlifa
- Laboratory of Biophysics and Medical Technologies, Higher Institute of Medical Technologies of Tunis, University of Tuins El Manar, 1006, Tunis, Tunisia
| |
Collapse
|
5
|
Hou W, Zou L, Wang D. Tumor Segmentation in Intraoperative Fluorescence Images Based on Transfer Learning and Convolutional Neural Networks. Surg Innov 2024:15533506241246576. [PMID: 38619039 DOI: 10.1177/15533506241246576] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 04/16/2024]
Abstract
OBJECTIVE To propose a transfer learning based method of tumor segmentation in intraoperative fluorescence images, which will assist surgeons to efficiently and accurately identify the boundary of tumors of interest. METHODS We employed transfer learning and deep convolutional neural networks (DCNNs) for tumor segmentation. Specifically, we first pre-trained four networks on the ImageNet dataset to extract low-level features. Subsequently, we fine-tuned these networks on two fluorescence image datasets (ABFM and DTHP) separately to enhance the segmentation performance of fluorescence images. Finally, we tested the trained models on the DTHL dataset. The performance of this approach was compared and evaluated against DCNNs trained end-to-end and the traditional level-set method. RESULTS The transfer learning-based UNet++ model achieved high segmentation accuracies of 82.17% on the ABFM dataset, 95.61% on the DTHP dataset, and 85.49% on the DTHL test set. For the DTHP dataset, the pre-trained Deeplab v3 + network performed exceptionally well, with a segmentation accuracy of 96.48%. Furthermore, all models achieved segmentation accuracies of over 90% when dealing with the DTHP dataset. CONCLUSION To the best of our knowledge, this study explores tumor segmentation on intraoperative fluorescent images for the first time. The results show that compared to traditional methods, deep learning has significant advantages in improving segmentation performance. Transfer learning enables deep learning models to perform better on small-sample fluorescence image data compared to end-to-end training. This discovery provides strong support for surgeons to obtain more reliable and accurate image segmentation results during surgery.
Collapse
Affiliation(s)
- Weijia Hou
- College of Science, Nanjing Forestry University, Nanjing, China
| | - Liwen Zou
- Department of Mathematics, Nanjing University, Nanjing, China
| | - Dong Wang
- Group A: Large-Scale Scientific Computing and Media Imaging, Nanjing Center for Applied Mathematics, Nanjing, China
| |
Collapse
|
6
|
Yang J, Wang G, Xiao X, Bao M, Tian G. Explainable ensemble learning method for OCT detection with transfer learning. PLoS One 2024; 19:e0296175. [PMID: 38517913 PMCID: PMC10959366 DOI: 10.1371/journal.pone.0296175] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/02/2023] [Accepted: 12/07/2023] [Indexed: 03/24/2024] Open
Abstract
The accuracy and interpretability of artificial intelligence (AI) are crucial for the advancement of optical coherence tomography (OCT) image detection, as it can greatly reduce the manual labor required by clinicians. By prioritizing these aspects during development and application, we can make significant progress towards streamlining the clinical workflow. In this paper, we propose an explainable ensemble approach that utilizes transfer learning to detect fundus lesion diseases through OCT imaging. Our study utilized a publicly available OCT dataset consisting of normal subjects, patients with dry age-related macular degeneration (AMD), and patients with diabetic macular edema (DME), each with 15 samples. The impact of pre-trained weights on the performance of individual networks was first compared, and then these networks were ensemble using majority soft polling. Finally, the features learned by the networks were visualized using Grad-CAM and CAM. The use of pre-trained ImageNet weights improved the performance from 68.17% to 92.89%. The ensemble model consisting of the three CNN models with pre-trained parameters loaded performed best, correctly distinguishing between AMD patients, DME patients and normal subjects 100% of the time. Visualization results showed that Grad-CAM could display the lesion area more accurately. It is demonstrated that the proposed approach could have good performance of both accuracy and interpretability in retinal OCT image detection.
Collapse
Affiliation(s)
- Jiasheng Yang
- Academician Workstation, Changsha Medical University, Changsha, Hunan, China
| | - Guanfang Wang
- School of Mathematics and Statistics, Nanjing University of Information Science and Technology, Nanjing, Jiangsu, China
- Geneis Beijing Co. Ltd., Beijing, China
| | - Xu Xiao
- School of International Education, Anhui University of Technology, Maanshan, Anhui, China
| | - Meihua Bao
- Academician Workstation, Changsha Medical University, Changsha, Hunan, China
| | - Geng Tian
- Geneis Beijing Co. Ltd., Beijing, China
| |
Collapse
|
7
|
Prabha AJ, Venkatesan C, Fathimal MS, Nithiyanantham KK, Kirubha SPA. RD-OCT net: hybrid learning system for automated diagnosis of macular diseases from OCT retinal images. Biomed Phys Eng Express 2024; 10:025033. [PMID: 38335542 DOI: 10.1088/2057-1976/ad27ea] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/17/2023] [Accepted: 02/09/2024] [Indexed: 02/12/2024]
Abstract
Macular Edema is a leading cause of visual impairment and blindness in patients with ocular fundus diseases. Due to its non-invasive and high-resolution characteristics, optical coherence tomography (OCT) has been extensively utilized for the diagnosis of macular diseases. The manual detection of retinal diseases by clinicians is a laborious process, further complicated by the challenging identification of macular diseases. This difficulty arises from the significant pathological alterations occurring within the retinal layers, as well as the accumulation of fluid in the retina. Deep Learning neural networks are utilized for automatic detection of retinal diseases. This paper aims to propose a lightweight hybrid learning Retinal Disease OCT Net with a reduced number of trainable parameters and enable automatic classification of retinal diseases. A Hybrid Learning Retinal Disease OCT Net (RD-OCT) is utilized for the multiclass classification of major retinal diseases, namely neovascular age-related macular degeneration (nAMD), diabetic macular edema (DME), retinal vein occlusion (RVO), and normal retinal conditions. The diagnosis of retinal diseases is facilitated by the use of hybrid learning models and pre-trained deep learning models in the field of artificial intelligence. The Hybrid Learning RD-OCT Net provides better accuracy of 97.6% for nAMD, 98.08% for DME, 98% for RVO, and 97% for the Normal group. The respective area under the curve values were 0.99, 0.97, 1.0, and 0.99. The utilization of the RD-OCT model will be useful for ophthalmologists in the diagnosis of prevalent retinal diseases, due to the simplicity of the system and reduced number of trainable parameters.
Collapse
Affiliation(s)
- A Jeya Prabha
- Department of Biomedical Engineering, College of Engineering and Technology, SRM Institute of Science and Technology, Kattankulathur, Chengalpattu-603203, Tamil Nadu, India
| | - C Venkatesan
- Department of Ophthalmology, SRM Medical College Hospital and Research Centre, Kattankulathur, Chengalpattu-603203, Tamil Nadu, India
| | - M Sameera Fathimal
- Department of Biomedical Engineering, College of Engineering and Technology, SRM Institute of Science and Technology, Kattankulathur, Chengalpattu-603203, Tamil Nadu, India
| | - K K Nithiyanantham
- Department of Aeronautical Engineering, Rajalakshmi Engineering College, Thandalam , Kancheepuram-602105, Tamil Nadu, India
| | - S P Angeline Kirubha
- Department of Biomedical Engineering, College of Engineering and Technology, SRM Institute of Science and Technology, Kattankulathur, Chengalpattu-603203, Tamil Nadu, India
| |
Collapse
|
8
|
Li J, Jiang P, An Q, Wang GG, Kong HF. Medical image identification methods: A review. Comput Biol Med 2024; 169:107777. [PMID: 38104516 DOI: 10.1016/j.compbiomed.2023.107777] [Citation(s) in RCA: 3] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/24/2023] [Revised: 10/30/2023] [Accepted: 11/28/2023] [Indexed: 12/19/2023]
Abstract
The identification of medical images is an essential task in computer-aided diagnosis, medical image retrieval and mining. Medical image data mainly include electronic health record data and gene information data, etc. Although intelligent imaging provided a good scheme for medical image analysis over traditional methods that rely on the handcrafted features, it remains challenging due to the diversity of imaging modalities and clinical pathologies. Many medical image identification methods provide a good scheme for medical image analysis. The concepts pertinent of methods, such as the machine learning, deep learning, convolutional neural networks, transfer learning, and other image processing technologies for medical image are analyzed and summarized in this paper. We reviewed these recent studies to provide a comprehensive overview of applying these methods in various medical image analysis tasks, such as object detection, image classification, image registration, segmentation, and other tasks. Especially, we emphasized the latest progress and contributions of different methods in medical image analysis, which are summarized base on different application scenarios, including classification, segmentation, detection, and image registration. In addition, the applications of different methods are summarized in different application area, such as pulmonary, brain, digital pathology, brain, skin, lung, renal, breast, neuromyelitis, vertebrae, and musculoskeletal, etc. Critical discussion of open challenges and directions for future research are finally summarized. Especially, excellent algorithms in computer vision, natural language processing, and unmanned driving will be applied to medical image recognition in the future.
Collapse
Affiliation(s)
- Juan Li
- School of Information Engineering, Wuhan Business University, Wuhan, 430056, China; School of Artificial Intelligence, Wuchang University of Technology, Wuhan, 430223, China; Key Laboratory of Symbolic Computation and Knowledge Engineering of Ministry of Education, Jilin University, Changchun, 130012, China
| | - Pan Jiang
- School of Information Engineering, Wuhan Business University, Wuhan, 430056, China
| | - Qing An
- School of Artificial Intelligence, Wuchang University of Technology, Wuhan, 430223, China
| | - Gai-Ge Wang
- School of Computer Science and Technology, Ocean University of China, Qingdao, 266100, China.
| | - Hua-Feng Kong
- School of Information Engineering, Wuhan Business University, Wuhan, 430056, China.
| |
Collapse
|
9
|
Peng J, Lu J, Zhuo J, Li P. Multi-Scale-Denoising Residual Convolutional Network for Retinal Disease Classification Using OCT. SENSORS (BASEL, SWITZERLAND) 2023; 24:150. [PMID: 38203011 PMCID: PMC10781341 DOI: 10.3390/s24010150] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 11/21/2023] [Revised: 12/13/2023] [Accepted: 12/15/2023] [Indexed: 01/12/2024]
Abstract
Macular pathologies can cause significant vision loss. Optical coherence tomography (OCT) images of the retina can assist ophthalmologists in diagnosing macular diseases. Traditional deep learning networks for retinal disease classification cannot extract discriminative features under strong noise conditions in OCT images. To address this issue, we propose a multi-scale-denoising residual convolutional network (MS-DRCN) for classifying retinal diseases. Specifically, the MS-DRCN includes a soft-denoising block (SDB), a multi-scale context block (MCB), and a feature fusion block (FFB). The SDB can determine the threshold for soft thresholding automatically, which removes speckle noise features efficiently. The MCB is designed to capture multi-scale context information and strengthen extracted features. The FFB is dedicated to integrating high-resolution and low-resolution features to precisely identify variable lesion areas. Our approach achieved classification accuracies of 96.4% and 96.5% on the OCT2017 and OCT-C4 public datasets, respectively, outperforming other classification methods. To evaluate the robustness of our method, we introduced Gaussian noise and speckle noise with varying PSNRs into the test set of the OCT2017 dataset. The results of our anti-noise experiments demonstrate that our approach exhibits superior robustness compared with other methods, yielding accuracy improvements ranging from 0.6% to 2.9% when compared with ResNet under various PSNR noise conditions.
Collapse
Affiliation(s)
- Jinbo Peng
- State Key Laboratory of Digital Medical Engineering, School of Biomedical Engineering, Hainan University, Haiko 570228, China; (J.P.); (J.L.)
- Key Laboratory of Biomedical Engineering of Hainan Province, One Health Institute, Hainan University, Haiko 570228, China
- Research Unit of Multimodal Cross Scale Neural Signal Detection and Imaging, Chinese Academy of Medical Science, HUST-Suzhou Institute for Brainsmatics, Jiangsu Industrial Technology Research Institute (JITRI), Suzhou 215100, China
| | - Jinling Lu
- State Key Laboratory of Digital Medical Engineering, School of Biomedical Engineering, Hainan University, Haiko 570228, China; (J.P.); (J.L.)
- Key Laboratory of Biomedical Engineering of Hainan Province, One Health Institute, Hainan University, Haiko 570228, China
- Research Unit of Multimodal Cross Scale Neural Signal Detection and Imaging, Chinese Academy of Medical Science, HUST-Suzhou Institute for Brainsmatics, Jiangsu Industrial Technology Research Institute (JITRI), Suzhou 215100, China
- Britton Chance Center for Biomedical Photonics and MoE Key Laboratory for Biomedical Photonics, Wuhan National Laboratory for Optoelectronics, Huazhong University of Science and Technology, Wuhan 430074, China
| | - Junjie Zhuo
- State Key Laboratory of Digital Medical Engineering, School of Biomedical Engineering, Hainan University, Haiko 570228, China; (J.P.); (J.L.)
- Key Laboratory of Biomedical Engineering of Hainan Province, One Health Institute, Hainan University, Haiko 570228, China
- Research Unit of Multimodal Cross Scale Neural Signal Detection and Imaging, Chinese Academy of Medical Science, HUST-Suzhou Institute for Brainsmatics, Jiangsu Industrial Technology Research Institute (JITRI), Suzhou 215100, China
| | - Pengcheng Li
- State Key Laboratory of Digital Medical Engineering, School of Biomedical Engineering, Hainan University, Haiko 570228, China; (J.P.); (J.L.)
- Key Laboratory of Biomedical Engineering of Hainan Province, One Health Institute, Hainan University, Haiko 570228, China
- Research Unit of Multimodal Cross Scale Neural Signal Detection and Imaging, Chinese Academy of Medical Science, HUST-Suzhou Institute for Brainsmatics, Jiangsu Industrial Technology Research Institute (JITRI), Suzhou 215100, China
- Britton Chance Center for Biomedical Photonics and MoE Key Laboratory for Biomedical Photonics, Wuhan National Laboratory for Optoelectronics, Huazhong University of Science and Technology, Wuhan 430074, China
| |
Collapse
|
10
|
Opoku M, Weyori BA, Adekoya AF, Adu K. CLAHE-CapsNet: Efficient retina optical coherence tomography classification using capsule networks with contrast limited adaptive histogram equalization. PLoS One 2023; 18:e0288663. [PMID: 38032915 PMCID: PMC10688733 DOI: 10.1371/journal.pone.0288663] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/03/2023] [Accepted: 07/01/2023] [Indexed: 12/02/2023] Open
Abstract
Manual detection of eye diseases using retina Optical Coherence Tomography (OCT) images by Ophthalmologists is time consuming, prone to errors and tedious. Previous researchers have developed a computer aided system using deep learning-based convolutional neural networks (CNNs) to aid in faster detection of the retina diseases. However, these methods find it difficult to achieve better classification performance due to noise in the OCT image. Moreover, the pooling operations in CNN reduce resolution of the image that limits the performance of the model. The contributions of the paper are in two folds. Firstly, this paper makes a comprehensive literature review to establish current-state-of-act methods successfully implemented in retina OCT image classifications. Additionally, this paper proposes a capsule network coupled with contrast limited adaptive histogram equalization (CLAHE-CapsNet) for retina OCT image classification. The CLAHE was implemented as layers to minimize the noise in the retina image for better performance of the model. A three-layer convolutional capsule network was designed with carefully chosen hyperparameters. The dataset used for this study was presented by University of California San Diego (UCSD). The dataset consists of 84,495 X-Ray images (JPEG) and 4 categories (NORMAL, CNV, DME, and DRUSEN). The images went through a grading system consisting of multiple layers of trained graders of expertise for verification and correction of image labels. Evaluation experiments were conducted and comparison of results was done with state-of-the-art models to find out the best performing model. The evaluation metrics; accuracy, sensitivity, precision, specificity, and AUC are used to determine the performance of the models. The evaluation results show that the proposed model achieves the best performing model of accuracies of 97.7%, 99.5%, and 99.3% on overall accuracy (OA), overall sensitivity (OS), and overall precision (OP), respectively. The results obtained indicate that the proposed model can be adopted and implemented to help ophthalmologists in detecting retina OCT diseases.
Collapse
Affiliation(s)
- Michael Opoku
- Department of Computer Science and Informatics, University of Energy and Natural Resource, Sunyani, Ghana
| | - Benjamin Asubam Weyori
- Department of Computer Science and Informatics, University of Energy and Natural Resource, Sunyani, Ghana
| | - Adebayo Felix Adekoya
- Department of Computer Science and Informatics, University of Energy and Natural Resource, Sunyani, Ghana
| | - Kwabena Adu
- Department of Computer Science and Informatics, University of Energy and Natural Resource, Sunyani, Ghana
| |
Collapse
|
11
|
Saidi L, Jomaa H, Zainab H, Zgolli H, Mabrouk S, Sidibé D, Tabia H, Khlifa N. Automatic Detection of AMD and DME Retinal Pathologies Using Deep Learning. Int J Biomed Imaging 2023; 2023:9966107. [PMID: 38046618 PMCID: PMC10691890 DOI: 10.1155/2023/9966107] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/12/2023] [Revised: 10/09/2023] [Accepted: 11/03/2023] [Indexed: 12/05/2023] Open
Abstract
Diabetic macular edema (DME) and age-related macular degeneration (AMD) are two common eye diseases. They are often undiagnosed or diagnosed late. This can result in permanent and irreversible vision loss. Therefore, early detection and treatment of these diseases can prevent vision loss, save money, and provide a better quality of life for individuals. Optical coherence tomography (OCT) imaging is widely applied to identify eye diseases, including DME and AMD. In this work, we developed automatic deep learning-based methods to detect these pathologies using SD-OCT scans. The convolutional neural network (CNN) from scratch we developed gave the best classification score with an accuracy higher than 99% on Duke dataset of OCT images.
Collapse
Affiliation(s)
- Latifa Saidi
- Laboratory of Biophysics and Medical Technologies, Higher Institute of Medical Technologies of Tunis, University of Tunis El Manar, Tunis, Tunisia
| | - Hajer Jomaa
- Laboratory of Biophysics and Medical Technologies, Higher Institute of Medical Technologies of Tunis, University of Tunis El Manar, Tunis, Tunisia
| | - Haddad Zainab
- Laboratory of Biophysics and Medical Technologies, National Engineering School Tunis, University of Tunis El Manar, Tunis, Tunisia
| | - Hsouna Zgolli
- Department A, Hedi Raies of Ophthalmology Institute, Tunis, Tunisia
| | - Sonia Mabrouk
- Department A, Hedi Raies of Ophthalmology Institute, Tunis, Tunisia
| | - Désiré Sidibé
- IBISC, University of Paris-Saclay, Univ Evry, Evry, France
| | - Hedi Tabia
- IBISC, University of Paris-Saclay, Univ Evry, Evry, France
| | - Nawres Khlifa
- Laboratory of Biophysics and Medical Technologies, Higher Institute of Medical Technologies of Tunis, University of Tunis El Manar, Tunis, Tunisia
| |
Collapse
|
12
|
Abbas Q, Albathan M, Altameem A, Almakki RS, Hussain A. Deep-Ocular: Improved Transfer Learning Architecture Using Self-Attention and Dense Layers for Recognition of Ocular Diseases. Diagnostics (Basel) 2023; 13:3165. [PMID: 37891986 PMCID: PMC10605427 DOI: 10.3390/diagnostics13203165] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/28/2023] [Revised: 10/07/2023] [Accepted: 10/09/2023] [Indexed: 10/29/2023] Open
Abstract
It is difficult for clinicians or less-experienced ophthalmologists to detect early eye-related diseases. By hand, eye disease diagnosis is labor-intensive, prone to mistakes, and challenging because of the variety of ocular diseases such as glaucoma (GA), diabetic retinopathy (DR), cataract (CT), and normal eye-related diseases (NL). An automated ocular disease detection system with computer-aided diagnosis (CAD) tools is required to recognize eye-related diseases. Nowadays, deep learning (DL) algorithms enhance the classification results of retinograph images. To address these issues, we developed an intelligent detection system based on retinal fundus images. To create this system, we used ODIR and RFMiD datasets, which included various retinographics of distinct classes of the fundus, using cutting-edge image classification algorithms like ensemble-based transfer learning. In this paper, we suggest a three-step hybrid ensemble model that combines a classifier, a feature extractor, and a feature selector. The original image features are first extracted using a pre-trained AlexNet model with an enhanced structure. The improved AlexNet (iAlexNet) architecture with attention and dense layers offers enhanced feature extraction, task adaptability, interpretability, and potential accuracy benefits compared to other transfer learning architectures, making it particularly suited for tasks like retinograph classification. The extracted features are then selected using the ReliefF method, and then the most crucial elements are chosen to minimize the feature dimension. Finally, an XgBoost classifier offers classification outcomes based on the desired features. These classifications represent different ocular illnesses. We utilized data augmentation techniques to control class imbalance issues. The deep-ocular model, based mainly on the AlexNet-ReliefF-XgBoost model, achieves an accuracy of 95.13%. The results indicate the proposed ensemble model can assist dermatologists in making early decisions for the diagnosing and screening of eye-related diseases.
Collapse
Affiliation(s)
- Qaisar Abbas
- College of Computer and Information Sciences, Imam Mohammad Ibn Saud Islamic University (IMSIU), Riyadh 11432, Saudi Arabia; (M.A.); (A.A.); (R.S.A.)
| | - Mubarak Albathan
- College of Computer and Information Sciences, Imam Mohammad Ibn Saud Islamic University (IMSIU), Riyadh 11432, Saudi Arabia; (M.A.); (A.A.); (R.S.A.)
| | - Abdullah Altameem
- College of Computer and Information Sciences, Imam Mohammad Ibn Saud Islamic University (IMSIU), Riyadh 11432, Saudi Arabia; (M.A.); (A.A.); (R.S.A.)
| | - Riyad Saleh Almakki
- College of Computer and Information Sciences, Imam Mohammad Ibn Saud Islamic University (IMSIU), Riyadh 11432, Saudi Arabia; (M.A.); (A.A.); (R.S.A.)
| | - Ayyaz Hussain
- Department of Computer Science, Quaid-i-Azam University, Islamabad 44000, Pakistan;
| |
Collapse
|
13
|
Park HJ, Kim SH, Choi JY, Cha D. Human-machine cooperation meta-model for clinical diagnosis by adaptation to human expert's diagnostic characteristics. Sci Rep 2023; 13:16204. [PMID: 37758800 PMCID: PMC10533492 DOI: 10.1038/s41598-023-43291-8] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/15/2023] [Accepted: 09/21/2023] [Indexed: 09/29/2023] Open
Abstract
Artificial intelligence (AI) using deep learning approaches the capabilities of human experts in medical image diagnosis. However, due to liability issues in medical decisions, AI is often relegated to an assistant role. Based on this responsibility constraint, the effective use of AI to assist human intelligence in real-world clinics remains a challenge. Given the significant inter-individual variations in clinical decisions among physicians based on their expertise, AI needs to adapt to individual experts, complementing weaknesses and enhancing strengths. For this adaptation, AI should not only acquire domain knowledge but also understand the specific human experts it assists. This study introduces a meta-model for human-machine cooperation that first evaluates each expert's class-specific diagnostic tendencies using conditional probability, based on which the meta-model adjusts the AI's predictions. This meta-model was applied to ear disease diagnosis using otoendoscopy, highlighting improved performance when incorporating individual diagnostic characteristics, even with limited evaluation data. The highest accuracy was achieved by combining each expert's conditional probabilities with machine classification probability, using optimal weights specific to each individual's overall classification accuracy. This tailored model aims to mitigate potential misjudgments due to psychological effects caused by machine suggestions and to capitalize on the unique expertise of individual clinicians.
Collapse
Affiliation(s)
- Hae-Jeong Park
- Department of Nuclear Medicine, Department of Psychiatry, Graduate School of Medical Science, Brain Korea 21 Project, Yonsei University College of Medicine, Seoul, South Korea.
- Department of Cognitive Science, Yonsei University, Seoul, Republic of Korea.
- Center for Systems and Translational Brain Sciences, Institute of Human Complexity and Systems Science, Yonsei University, 50-1, Yonsei-ro, Sinchon-dong, Seodaemun-gu, Seoul, 03722, Republic of Korea.
| | - Sung Huhn Kim
- Department of Otorhinolaryngology, Yonsei University College of Medicine, Seoul, South Korea
| | - Jae Young Choi
- Department of Otorhinolaryngology, Yonsei University College of Medicine, Seoul, South Korea
| | - Dongchul Cha
- Department of Otorhinolaryngology, Yonsei University College of Medicine, Seoul, South Korea.
- Center for Innovative Medicine, Healthcare Lab, NAVER Corporation, 95, Jeongjail-ro, Bundang-gu, Seongnam-si, Gyeonggi-do, 13561, Republic of Korea.
- Healthcare Lab, Naver Cloud Corporation, Seongnam-si, Republic of Korea.
| |
Collapse
|
14
|
Chen S, Wu Z, Li M, Zhu Y, Xie H, Yang P, Zhao C, Zhang Y, Zhang S, Zhao X, Lu L, Zhang G, Lei B. FIT-Net: Feature Interaction Transformer Network for Pathologic Myopia Diagnosis. IEEE TRANSACTIONS ON MEDICAL IMAGING 2023; 42:2524-2538. [PMID: 37030824 DOI: 10.1109/tmi.2023.3260990] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/19/2023]
Abstract
Automatic and accurate classification of retinal optical coherence tomography (OCT) images is essential to assist physicians in diagnosing and grading pathological changes in pathologic myopia (PM). Clinically, due to the obvious differences in the position, shape, and size of the lesion structure in different scanning directions, ophthalmologists usually need to combine the lesion structure in the OCT images in the horizontal and vertical scanning directions to diagnose the type of pathological changes in PM. To address these challenges, we propose a novel feature interaction Transformer network (FIT-Net) to diagnose PM using OCT images, which consists of two dual-scale Transformer (DST) blocks and an interactive attention (IA) unit. Specifically, FIT-Net divides image features of different scales into a series of feature block sequences. In order to enrich the feature representation, we propose an IA unit to realize the interactive learning of class token in feature sequences of different scales. The interaction between feature sequences of different scales can effectively integrate different scale image features, and hence FIT-Net can focus on meaningful lesion regions to improve the PM classification performance. Finally, by fusing the dual-view image features in the horizontal and vertical scanning directions, we propose six dual-view feature fusion methods for PM diagnosis. The extensive experimental results based on the clinically obtained datasets and three publicly available datasets demonstrate the effectiveness and superiority of the proposed method. Our code is avaiable at: https://github.com/chenshaobin/FITNet.
Collapse
|
15
|
Yu YW, Lin CH, Lu CK, Wang JK, Huang TL. Automated Age-Related Macular Degeneration Detector on Optical Coherence Tomography Images Using Slice-Sum Local Binary Patterns and Support Vector Machine. SENSORS (BASEL, SWITZERLAND) 2023; 23:7315. [PMID: 37687770 PMCID: PMC10489965 DOI: 10.3390/s23177315] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 07/19/2023] [Revised: 08/15/2023] [Accepted: 08/19/2023] [Indexed: 09/10/2023]
Abstract
Artificial intelligence has revolutionised smart medicine, resulting in enhanced medical care. This study presents an automated detector chip for age-related macular degeneration (AMD) using a support vector machine (SVM) and three-dimensional (3D) optical coherence tomography (OCT) volume. The aim is to assist ophthalmologists by reducing the time-consuming AMD medical examination. Using the property of 3D OCT volume, a modified feature vector connected method called slice-sum is proposed, reducing computational complexity while maintaining high detection accuracy. Compared to previous methods, this method significantly reduces computational complexity by at least a hundredfold. Image adjustment and noise removal steps are excluded for classification accuracy, and the feature extraction algorithm of local binary patterns is determined based on hardware consumption considerations. Through optimisation of the feature vector connection method after feature extraction, the computational complexity of SVM detection is significantly reduced, making it applicable to similar 3D datasets. Additionally, the design supports model replacement, allowing users to train and update classification models as needed. Using TSMC 40 nm CMOS technology, the proposed detector achieves a core area of 0.12 mm2 while demonstrating a classification throughput of 8.87 decisions/s at a maximum operating frequency of 454.54 MHz. The detector achieves a final testing classification accuracy of 92.31%.
Collapse
Affiliation(s)
- Yao-Wen Yu
- Department of Electrical Engineering, Yuan Ze University, Taoyuan City 320, Taiwan
| | - Cheng-Hung Lin
- Department of Electrical Engineering, Yuan Ze University, Taoyuan City 320, Taiwan
| | - Cheng-Kai Lu
- Department of Electrical Engineering, National Taiwan Normal University, Taipei City 106, Taiwan;
| | - Jia-Kang Wang
- Department of Electrical Engineering, Yuan Ze University, Taoyuan City 320, Taiwan
- Department of Ophthalmology, Far Eastern Memorial Hospital, New Taipei City 220, Taiwan
| | - Tzu-Lun Huang
- Department of Electrical Engineering, Yuan Ze University, Taoyuan City 320, Taiwan
- Department of Ophthalmology, Far Eastern Memorial Hospital, New Taipei City 220, Taiwan
| |
Collapse
|
16
|
Khan A, Pin K, Aziz A, Han JW, Nam Y. Optical Coherence Tomography Image Classification Using Hybrid Deep Learning and Ant Colony Optimization. SENSORS (BASEL, SWITZERLAND) 2023; 23:6706. [PMID: 37571490 PMCID: PMC10422382 DOI: 10.3390/s23156706] [Citation(s) in RCA: 7] [Impact Index Per Article: 7.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 06/13/2023] [Revised: 07/11/2023] [Accepted: 07/24/2023] [Indexed: 08/13/2023]
Abstract
Optical coherence tomography (OCT) is widely used to detect and classify retinal diseases. However, OCT-image-based manual detection by ophthalmologists is prone to errors and subjectivity. Thus, various automation methods have been proposed; however, improvements in detection accuracy are required. Particularly, automated techniques using deep learning on OCT images are being developed to detect various retinal disorders at an early stage. Here, we propose a deep learning-based automatic method for detecting and classifying retinal diseases using OCT images. The diseases include age-related macular degeneration, branch retinal vein occlusion, central retinal vein occlusion, central serous chorioretinopathy, and diabetic macular edema. The proposed method comprises four main steps: three pretrained models, DenseNet-201, InceptionV3, and ResNet-50, are first modified according to the nature of the dataset, after which the features are extracted via transfer learning. The extracted features are improved, and the best features are selected using ant colony optimization. Finally, the best features are passed to the k-nearest neighbors and support vector machine algorithms for final classification. The proposed method, evaluated using OCT retinal images collected from Soonchunhyang University Bucheon Hospital, demonstrates an accuracy of 99.1% with the incorporation of ACO. Without ACO, the accuracy achieved is 97.4%. Furthermore, the proposed method exhibits state-of-the-art performance and outperforms existing techniques in terms of accuracy.
Collapse
Affiliation(s)
- Awais Khan
- Department of ICT Convergence, Soonchunhyang University, Asan 31538, Republic of Korea; (A.K.); (K.P.); (A.A.)
| | - Kuntha Pin
- Department of ICT Convergence, Soonchunhyang University, Asan 31538, Republic of Korea; (A.K.); (K.P.); (A.A.)
| | - Ahsan Aziz
- Department of ICT Convergence, Soonchunhyang University, Asan 31538, Republic of Korea; (A.K.); (K.P.); (A.A.)
| | - Jung Woo Han
- Department of Ophthalmology, Bucheon Hospital, Soonchunhyang University College of Medicine, Bucheon 14584, Republic of Korea
| | - Yunyoung Nam
- Department of Computer Science and Engineering, Soonchunhyang University, Asan 31538, Republic of Korea
| |
Collapse
|
17
|
Feng H, Chen J, Zhang Z, Lou Y, Zhang S, Yang W. A bibliometric analysis of artificial intelligence applications in macular edema: exploring research hotspots and Frontiers. Front Cell Dev Biol 2023; 11:1174936. [PMID: 37255600 PMCID: PMC10225517 DOI: 10.3389/fcell.2023.1174936] [Citation(s) in RCA: 3] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/14/2023] [Accepted: 05/02/2023] [Indexed: 06/01/2023] Open
Abstract
Background: Artificial intelligence (AI) is used in ophthalmological disease screening and diagnostics, medical image diagnostics, and predicting late-disease progression rates. We reviewed all AI publications associated with macular edema (ME) research Between 2011 and 2022 and performed modeling, quantitative, and qualitative investigations. Methods: On 1st February 2023, we screened the Web of Science Core Collection for AI applications related to ME, from which 297 studies were identified and analyzed (2011-2022). We collected information on: publications, institutions, country/region, keywords, journal name, references, and research hotspots. Literature clustering networks and Frontier knowledge bases were investigated using bibliometrix-BiblioShiny, VOSviewer, and CiteSpace bibliometric platforms. We used the R "bibliometrix" package to synopsize our observations, enumerate keywords, visualize collaboration networks between countries/regions, and generate a topic trends plot. VOSviewer was used to examine cooperation between institutions and identify citation relationships between journals. We used CiteSpace to identify clustering keywords over the timeline and identify keywords with the strongest citation bursts. Results: In total, 47 countries published AI studies related to ME; the United States had the highest H-index, thus the greatest influence. China and the United States cooperated most closely between all countries. Also, 613 institutions generated publications - the Medical University of Vienna had the highest number of studies. This publication record and H-index meant the university was the most influential in the ME field. Reference clusters were also categorized into 10 headings: retinal Optical Coherence Tomography (OCT) fluid detection, convolutional network models, deep learning (DL)-based single-shot predictions, retinal vascular disease, diabetic retinopathy (DR), convolutional neural networks (CNNs), automated macular pathology diagnosis, dry age-related macular degeneration (DARMD), class weight, and advanced DL architecture systems. Frontier keywords were represented by diabetic macular edema (DME) (2021-2022). Conclusion: Our review of the AI-related ME literature was comprehensive, systematic, and objective, and identified future trends and current hotspots. With increased DL outputs, the ME research focus has gradually shifted from manual ME examinations to automatic ME detection and associated symptoms. In this review, we present a comprehensive and dynamic overview of AI in ME and identify future research areas.
Collapse
Affiliation(s)
- Haiwen Feng
- Department of Software Engineering, School of Software, Shenyang University of Technology, Shenyang, Liaoning, China
| | - Jiaqi Chen
- Department of Software Engineering, School of Software, Shenyang University of Technology, Shenyang, Liaoning, China
| | - Zhichang Zhang
- Department of Computer, School of Intelligent Medicine, China Medical University, Shenyang, Liaoning, China
| | - Yan Lou
- Department of Computer, School of Intelligent Medicine, China Medical University, Shenyang, Liaoning, China
| | - Shaochong Zhang
- Shenzhen Eye Institute, Shenzhen Eye Hospital, Jinan University, Shenzhen, China
| | - Weihua Yang
- Shenzhen Eye Institute, Shenzhen Eye Hospital, Jinan University, Shenzhen, China
| |
Collapse
|
18
|
Manikandan S, Raman R, Rajalakshmi R, Tamilselvi S, Surya RJ. Deep learning-based detection of diabetic macular edema using optical coherence tomography and fundus images: A meta-analysis. Indian J Ophthalmol 2023; 71:1783-1796. [PMID: 37203031 PMCID: PMC10391382 DOI: 10.4103/ijo.ijo_2614_22] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 05/20/2023] Open
Abstract
Diabetic macular edema (DME) is an important cause of visual impairment in the working-age group. Deep learning methods have been developed to detect DME from two-dimensional retinal images and also from optical coherence tomography (OCT) images. The performances of these algorithms vary and often create doubt regarding their clinical utility. In resource-constrained health-care systems, these algorithms may play an important role in determining referral and treatment. The survey provides a diversified overview of macular edema detection methods, including cutting-edge research, with the objective of providing pertinent information to research groups, health-care professionals, and diabetic patients about the applications of deep learning in retinal image detection and classification process. Electronic databases such as PubMed, IEEE Explore, BioMed, and Google Scholar were searched from inception to March 31, 2022, and the reference lists of published papers were also searched. The study followed the preferred reporting items for systematic review and meta-analysis (PRISMA) reporting guidelines. Examination of various deep learning models and their exhibition regarding precision, epochs, their capacity to detect anomalies for less training data, concepts, and challenges that go deep into the applications were analyzed. A total of 53 studies were included that evaluated the performance of deep learning models in a total of 1,414,169°CT volumes, B-scans, patients, and 472,328 fundus images. The overall area under the receiver operating characteristic curve (AUROC) was 0.9727. The overall sensitivity for detecting DME using OCT images was 96% (95% confidence interval [CI]: 0.94-0.98). The overall sensitivity for detecting DME using fundus images was 94% (95% CI: 0.90-0.96).
Collapse
Affiliation(s)
- Suchetha Manikandan
- Professor & Deputy Director, Centre for Healthcare Advancement, Innovation ! Research, Vellore Institute of Technology, Chennai, Tamil Nadu, India
| | - Rajiv Raman
- Senior Consultant, Shri Bhagwan Mahavir Vitreoretinal Services, Sankara Nethralaya, Chennai, Tamil Nadu, India
| | - Ramachandran Rajalakshmi
- Head Medical Retina, Dr. Mohan's Diabetes Specialties Centre and Madras Diabetes Research Foundation, Chennai, Tamil Nadu, India
| | - S Tamilselvi
- Junior Research Fellow, Centre for Healthcare Advancement, Innovation & Research, Vellore Institute of Technology, Chennai, Tamil Nadu, India
| | - R Janani Surya
- Research Associate, Vision Research Foundation, Chennai, Tamil Nadu, India
| |
Collapse
|
19
|
Vali M, Nazari B, Sadri S, Pour EK, Riazi-Esfahani H, Faghihi H, Ebrahimiadib N, Azizkhani M, Innes W, Steel DH, Hurlbert A, Read JCA, Kafieh R. CNV-Net: Segmentation, Classification and Activity Score Measurement of Choroidal Neovascularization (CNV) Using Optical Coherence Tomography Angiography (OCTA). Diagnostics (Basel) 2023; 13:diagnostics13071309. [PMID: 37046527 PMCID: PMC10093691 DOI: 10.3390/diagnostics13071309] [Citation(s) in RCA: 2] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/29/2023] [Revised: 03/23/2023] [Accepted: 03/24/2023] [Indexed: 04/03/2023] Open
Abstract
This paper aims to present an artificial intelligence-based algorithm for the automated segmentation of Choroidal Neovascularization (CNV) areas and to identify the presence or absence of CNV activity criteria (branching, peripheral arcade, dark halo, shape, loop and anastomoses) in OCTA images. Methods: This retrospective and cross-sectional study includes 130 OCTA images from 101 patients with treatment-naïve CNV. At baseline, OCTA volumes of 6 × 6 mm2 were obtained to develop an AI-based algorithm to evaluate the CNV activity based on five activity criteria, including tiny branching vessels, anastomoses and loops, peripheral arcades, and perilesional hypointense halos. The proposed algorithm comprises two steps. The first block includes the pre-processing and segmentation of CNVs in OCTA images using a modified U-Net network. The second block consists of five binary classification networks, each implemented with various models from scratch, and using transfer learning from pre-trained networks. Results: The proposed segmentation network yielded an averaged Dice coefficient of 0.86. The individual classifiers corresponding to the five activity criteria (branch, peripheral arcade, dark halo, shape, loop, and anastomoses) showed accuracies of 0.84, 0.81, 0.86, 0.85, and 0.82, respectively. The AI-based algorithm potentially allows the reliable detection and segmentation of CNV from OCTA alone, without the need for imaging with contrast agents. The evaluation of the activity criteria in CNV lesions obtains acceptable results, and this algorithm could enable the objective, repeatable assessment of CNV features.
Collapse
|
20
|
Kayadibi İ, Güraksın GE. An Explainable Fully Dense Fusion Neural Network with Deep Support Vector Machine for Retinal Disease Determination. INT J COMPUT INT SYS 2023. [DOI: 10.1007/s44196-023-00210-z] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 03/11/2023] Open
Abstract
AbstractRetinal issues are crucial because they result in visual loss. Early diagnosis can aid physicians in initiating treatment and preventing visual loss. Optical coherence tomography (OCT), which portrays retinal morphology cross-sectionally and noninvasively, is used to identify retinal abnormalities. The process of analyzing OCT images, on the other hand, takes time. This study has proposed a hybrid approach based on a fully dense fusion neural network (FD-CNN) and dual preprocessing to identify retinal diseases, such as choroidal neovascularization, diabetic macular edema, drusen from OCT images. A dual preprocessing methodology, in other words, a hybrid speckle reduction filter was initially used to diminish speckle noise present in OCT images. Secondly, the FD-CNN architecture was trained, and the features obtained from this architecture were extracted. Then Deep Support Vector Machine (D-SVM) and Deep K-Nearest Neighbor (D-KNN) classifiers were proposed to reclassify those features and tested on University of California San Diego (UCSD) and Duke OCT datasets. D-SVM demonstrated the best performance in both datasets. D-SVM achieved 99.60% accuracy, 99.60% sensitivity, 99.87% specificity, 99.60% precision and 99.60% F1 score in the UCSD dataset. It achieved 97.50% accuracy, 97.64% sensitivity, 98.91% specificity, 96.61% precision, and 97.03% F1 score in Duke dataset. Additionally, the results were compared to state-of-the-art works on the both datasets. The D-SVM was demonstrated to be an efficient and productive strategy for improving the robustness of automatic retinal disease classification. Also, in this study, it is shown that the unboxing of how AI systems' black-box choices is made by generating heat maps using the local interpretable model-agnostic explanation method, which is an explainable artificial intelligence (XAI) technique. Heat maps, in particular, may contribute to the development of more stable deep learning-based systems, as well as enhancing the confidence in the diagnosis of retinal disease in the analysis of OCT image for ophthalmologists.
Collapse
|
21
|
A review of deep learning-based multiple-lesion recognition from medical images: classification, detection and segmentation. Comput Biol Med 2023; 157:106726. [PMID: 36924732 DOI: 10.1016/j.compbiomed.2023.106726] [Citation(s) in RCA: 20] [Impact Index Per Article: 20.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/15/2022] [Revised: 02/07/2023] [Accepted: 02/27/2023] [Indexed: 03/05/2023]
Abstract
Deep learning-based methods have become the dominant methodology in medical image processing with the advancement of deep learning in natural image classification, detection, and segmentation. Deep learning-based approaches have proven to be quite effective in single lesion recognition and segmentation. Multiple-lesion recognition is more difficult than single-lesion recognition due to the little variation between lesions or the too wide range of lesions involved. Several studies have recently explored deep learning-based algorithms to solve the multiple-lesion recognition challenge. This paper includes an in-depth overview and analysis of deep learning-based methods for multiple-lesion recognition developed in recent years, including multiple-lesion recognition in diverse body areas and recognition of whole-body multiple diseases. We discuss the challenges that still persist in the multiple-lesion recognition tasks by critically assessing these efforts. Finally, we outline existing problems and potential future research areas, with the hope that this review will help researchers in developing future approaches that will drive additional advances.
Collapse
|
22
|
Liew A, Agaian S, Benbelkacem S. Distinctions between Choroidal Neovascularization and Age Macular Degeneration in Ocular Disease Predictions via Multi-Size Kernels ξcho-Weighted Median Patterns. Diagnostics (Basel) 2023; 13:diagnostics13040729. [PMID: 36832215 PMCID: PMC9956029 DOI: 10.3390/diagnostics13040729] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/28/2022] [Revised: 12/31/2022] [Accepted: 01/05/2023] [Indexed: 02/17/2023] Open
Abstract
Age-related macular degeneration is a visual disorder caused by abnormalities in a part of the eye's retina and is a leading source of blindness. The correct detection, precise location, classification, and diagnosis of choroidal neovascularization (CNV) may be challenging if the lesion is small or if Optical Coherence Tomography (OCT) images are degraded by projection and motion. This paper aims to develop an automated quantification and classification system for CNV in neovascular age-related macular degeneration using OCT angiography images. OCT angiography is a non-invasive imaging tool that visualizes retinal and choroidal physiological and pathological vascularization. The presented system is based on new retinal layers in the OCT image-specific macular diseases feature extractor, including Multi-Size Kernels ξcho-Weighted Median Patterns (MSKξMP). Computer simulations show that the proposed method: (i) outperforms current state-of-the-art methods, including deep learning techniques; and (ii) achieves an overall accuracy of 99% using ten-fold cross-validation on the Duke University dataset and over 96% on the noisy Noor Eye Hospital dataset. In addition, MSKξMP performs well in binary eye disease classifications and is more accurate than recent works in image texture descriptors.
Collapse
Affiliation(s)
- Alex Liew
- Department of Computer Science, Graduate Center of City University New York, 365 5th Ave., New York, NY 10016, USA
- Correspondence:
| | - Sos Agaian
- Department of Computer Science, Graduate Center of City University New York, 365 5th Ave., New York, NY 10016, USA
| | - Samir Benbelkacem
- Robotics and Industrial Automation Division, Centre de Développement des Technologies Avancées (CDTA), Algiers 16081, Algeria
| |
Collapse
|
23
|
Sengar N, Joshi RC, Dutta MK, Burget R. EyeDeep-Net: a multi-class diagnosis of retinal diseases using deep neural network. Neural Comput Appl 2023. [DOI: 10.1007/s00521-023-08249-x] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/22/2023]
|
24
|
Choudhary A, Ahlawat S, Urooj S, Pathak N, Lay-Ekuakille A, Sharma N. A Deep Learning-Based Framework for Retinal Disease Classification. Healthcare (Basel) 2023; 11:healthcare11020212. [PMID: 36673578 PMCID: PMC9859538 DOI: 10.3390/healthcare11020212] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/21/2022] [Revised: 12/23/2022] [Accepted: 12/29/2022] [Indexed: 01/12/2023] Open
Abstract
This study addresses the problem of the automatic detection of disease states of the retina. In order to solve the abovementioned problem, this study develops an artificially intelligent model. The model is based on a customized 19-layer deep convolutional neural network called VGG-19 architecture. The model (VGG-19 architecture) is empowered by transfer learning. The model is designed so that it can learn from a large set of images taken with optical coherence tomography (OCT) and classify them into four conditions of the retina: (1) choroidal neovascularization, (2) drusen, (3) diabetic macular edema, and (4) normal form. The training datasets (taken from publicly available sources) consist of 84,568 instances of OCT retinal images. The datasets exhibit all four classes of retinal disease mentioned above. The proposed model achieved a 99.17% classification accuracy with 0.995 specificities and 0.99 sensitivity, making it better than the existing models. In addition, the proper statistical evaluation is done on the predictions using such performance measures as (1) area under the receiver operating characteristic curve, (2) Cohen's kappa parameter, and (3) confusion matrix. Experimental results show that the proposed VGG-19 architecture coupled with transfer learning is an effective technique for automatically detecting the disease state of a retina.
Collapse
Affiliation(s)
- Amit Choudhary
- University School of Automation and Robotics, G.G.S. Indraprastha University, New Delhi 110092, India
| | - Savita Ahlawat
- Maharaja Surajmal Institute of Technology, G.G.S. Indraprastha University, New Delhi 110058, India
- Correspondence: (S.A.); (S.U.)
| | - Shabana Urooj
- Department of Electrical Engineering, College of Engineering, Princess Nourah Bint Abdulrahman University, P.O. Box 84428, Riyadh 11671, Saudi Arabia
- Correspondence: (S.A.); (S.U.)
| | - Nitish Pathak
- Department of Information Technology, Bhagwan Parshuram Institute of Technology (BPIT), G.G.S. Indraprastha University, New Delhi 110078, India
| | - Aimé Lay-Ekuakille
- Department of Innovation Engineering, University of Salento, 73100 Lecce, Italy
| | - Neelam Sharma
- Department of Artificial Intelligence and Machine Learning, Maharaja Agrasen Institute of Technology (MAIT), G.G.S. Indraprastha University, New Delhi 110086, India
| |
Collapse
|
25
|
Pavithra K, Kumar P, Geetha M, Bhandary SV. Computer aided diagnosis of diabetic macular edema in retinal fundus and OCT images: A review. Biocybern Biomed Eng 2023. [DOI: 10.1016/j.bbe.2022.12.005] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/04/2023]
|
26
|
Celebi ARC, Bulut E, Sezer A. Artificial intelligence based detection of age-related macular degeneration using optical coherence tomography with unique image preprocessing. Eur J Ophthalmol 2023; 33:65-73. [PMID: 35469472 DOI: 10.1177/11206721221096294] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/11/2023]
Abstract
PURPOSE The aim of the study is to improve the accuracy of age related macular degeneration (AMD) disease in its earlier phases with proposed Capsule Network (CapsNet) architecture trained on speckle noise reduced spectral domain optical coherence tomography (SD-OCT) images based on an optimized Bayesian non-local mean (OBNLM) filter augmentation techniques. METHODS A total of 726 local SD-OCT images were collected and labelled as 159 drusen, 145 dry AMD, 156 wet AMD and 266 normal. Region of interest (ROI) was identified. Speckle noise in SD-OCT images were reduced based on OBNLM filter. The processed images were fed to proposed CapsNet architecture to clasify SD-OCT images. Accuracy rates were calculated in both public and local dataset. RESULTS Accuracy rate of local SD-OCT image dataset classification was achieved to a value of 96.39% after performing data augmentation and speckle noise reduction with OBNLM. The performance of proposed CapsNet was also evaluated on the public Kaggle dataset under the same processing procedures and the accuracy rate was calculated as 98.07%. The sensitivity and specificity rates were 96.72% and 99.98%, respectively. CONCLUSIONS The classification success of proposed CapsNet may be improved with robust pre-processing steps like; determination of ROI and denoised SD-OCT images based on OBNLM. These impactful image preprocessing steps yielded higher accuracy rates for determining different types of AMD including its precursor lesion on the both local and public dataset with proposed CapsNet architecture.
Collapse
Affiliation(s)
- Ali Riza Cenk Celebi
- Department of Ophthalmology, Acibadem University School of Medicine, Istanbul, Turkey
| | - Erkan Bulut
- Department of Ophthalmology, Beylikduzu Public Hospital, Istanbul, Turkey
| | - Aysun Sezer
- United'Informatique et d'Ingenierie des Systemes, 52849ENSTA-ParisTech, Universite de Paris-Saclay, Villefranche Sur Mer, Provence-Alpes-Côte d'azur, France
| |
Collapse
|
27
|
Lu Z, Miao J, Dong J, Zhu S, Wang X, Feng J. Automatic classification of retinal diseases with transfer learning-based lightweight convolutional neural network. Biomed Signal Process Control 2022. [DOI: 10.1016/j.bspc.2022.104365] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/23/2022]
|
28
|
Zhao M, Lu Z, Zhu S, Wang X, Feng J. Automatic generation of retinal optical coherence tomography images based on generative adversarial networks. Med Phys 2022; 49:7357-7367. [PMID: 36122302 DOI: 10.1002/mp.15988] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/14/2022] [Revised: 07/13/2022] [Accepted: 08/28/2022] [Indexed: 12/13/2022] Open
Abstract
SIGNIFICANCE The automatic generation algorithm of optical coherence tomography (OCT) images based on generative adversarial networks (GAN) can generate a large number of simulation images by a relatively small number of real images, which can effectively improve the classification performance. AIM We proposed an automatic generation algorithm for retinal OCT images based on GAN to alleviate the problem of insufficient images with high quality in deep learning, and put the diagnosis algorithm toward clinical application. APPROACH We designed a generation network based on GAN and trained the network with a data set constructed by 2014_BOE_Srinivasan and OCT2017 to acquire three models. Then, we generated a large number of images by the three models to augment age-related macular degeneration (AMD), diabetic macular edema (DME), and normal images. We evaluated the generated images by subjective visual observation, Fréchet inception distance (FID) scores, and a classification experiment. RESULTS Visual observation shows that the generated images have clear and similar features compared with the real images. Also, the lesion regions containing similar features in the real image and the generated image are randomly distributed in the image field of view. When the FID scores of the three types of generated images are lowest, three local optimal models are obtained for AMD, DME, and normal images, indicating the generated images have high quality and diversity. Moreover, the classification experiment results show that the model performance trained with the mixed images is better than that of the model trained with real images, in which the accuracy, sensitivity, and specificity are improved by 5.56%, 8.89%, and 2.22%. In addition, compared with the generation method based on variational auto-encoder (VAE), the method improved the accuracy, sensitivity, and specificity by 1.97%, 2.97%, and 0.99%, for the same test set. CONCLUSIONS The results show that our method can augment the three kinds of OCT images, not only effectively alleviating the problem of insufficient images with high quality but also improving the diagnosis performance.
Collapse
Affiliation(s)
- Mengmeng Zhao
- Beijing University of Technology, Intelligent Physiological Measurement and Clinical Translation, International Base for Science and Technology Cooperation, Department of Biomedical Engineering, Beijing University of Technology, Beijing, China
| | - Zhenzhen Lu
- Beijing University of Technology, Intelligent Physiological Measurement and Clinical Translation, International Base for Science and Technology Cooperation, Department of Biomedical Engineering, Beijing University of Technology, Beijing, China
| | - Shuyuan Zhu
- Beijing University of Technology, Intelligent Physiological Measurement and Clinical Translation, International Base for Science and Technology Cooperation, Department of Biomedical Engineering, Beijing University of Technology, Beijing, China
| | - Xiaobing Wang
- Capital University of Physical Education and Sports, Sports and Medicine Integrative Innovation Center, Capital University of Physical Education and Sports, Beijing, China
| | - Jihong Feng
- Beijing University of Technology, Intelligent Physiological Measurement and Clinical Translation, International Base for Science and Technology Cooperation, Department of Biomedical Engineering, Beijing University of Technology, Beijing, China
| |
Collapse
|
29
|
Wongchaisuwat P, Thamphithak R, Jitpukdee P, Wongchaisuwat N. Application of Deep Learning for Automated Detection of Polypoidal Choroidal Vasculopathy in Spectral Domain Optical Coherence Tomography. Transl Vis Sci Technol 2022; 11:16. [PMID: 36219163 PMCID: PMC9580222 DOI: 10.1167/tvst.11.10.16] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/03/2022] [Accepted: 08/29/2022] [Indexed: 11/25/2022] Open
Abstract
Objective To develop an automated polypoidal choroidal vasculopathy (PCV) screening model to distinguish PCV from wet age-related macular degeneration (wet AMD). Methods A retrospective review of spectral domain optical coherence tomography (SD-OCT) images was undertaken. The included SD-OCT images were classified into two distinct categories (PCV or wet AMD) prior to the development of the PCV screening model. The automated detection of PCV using the developed model was compared with the results of gold-standard fundus fluorescein angiography and indocyanine green (FFA + ICG) angiography. A framework of SHapley Additive exPlanations was used to interpret the results from the model. Results A total of 2334 SD-OCT images were enrolled for training purposes, and an additional 1171 SD-OCT images were used for external validation. The ResNet attention model yielded superior performance with average area under the curve values of 0.8 and 0.81 for the training and external validation data sets, respectively. The sensitivity/specificity calculated at a patient level was 100%/60% and 85%/71% for the training and external validation data sets, respectively. Conclusions A conventional FFA + ICG investigation to differentiate PCV from wet AMD requires intense health care resources and adversely affects patients. A deep learning algorithm is proposed to automatically distinguish PCV from wet AMD. The developed algorithm exhibited promising performance for further development into an alternative PCV screening tool. Enhancement of the model's performance with additional data is needed prior to implementation of this diagnostic tool in real-world clinical practice. The invisibility of disease signs within SD-OCT images is the main limitation of the proposed model. Translational Relevance Basic research of deep learning algorithms was applied to differentiate PCV from wet AMD based on OCT images, benefiting a diagnosis process and minimizing a risk of ICG angiogram.
Collapse
Affiliation(s)
- Papis Wongchaisuwat
- Department of Industrial Engineering, Faculty of Engineering, Kasetsart University, Bangkok, Thailand
| | - Ranida Thamphithak
- Department of Ophthalmology, Faculty of Medicine Siriraj Hospital, Mahidol University, Bangkok, Thailand
| | - Peerakarn Jitpukdee
- Department of Industrial Engineering, Faculty of Engineering, Kasetsart University, Bangkok, Thailand
| | - Nida Wongchaisuwat
- Department of Ophthalmology, Faculty of Medicine Siriraj Hospital, Mahidol University, Bangkok, Thailand
| |
Collapse
|
30
|
Padilla-Pantoja FD, Sanchez YD, Quijano-Nieto BA, Perdomo OJ, Gonzalez FA. Etiology of Macular Edema Defined by Deep Learning in Optical Coherence Tomography Scans. Transl Vis Sci Technol 2022; 11:29. [PMID: 36169966 PMCID: PMC9526369 DOI: 10.1167/tvst.11.9.29] [Citation(s) in RCA: 4] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/24/2022] Open
Abstract
Purpose To develop an automated method based on deep learning (DL) to classify macular edema (ME) from the evaluation of optical coherence tomography (OCT) scans. Methods A total of 4230 images were obtained from data repositories of patients attended in an ophthalmology clinic in Colombia and two free open-access databases. They were annotated with four biomarkers (BMs) as intraretinal fluid, subretinal fluid, hyperreflective foci/tissue, and drusen. Then the scans were labeled as control or ocular disease among diabetic macular edema (DME), neovascular age-related macular degeneration (nAMD), and retinal vein occlusion (RVO) by two expert ophthalmologists. Our method was developed by following four consecutive phases: segmentation of BMs, the combination of BMs, feature extraction with convolutional neural networks to achieve binary classification for each disease, and, finally, multiclass classification of diseases and control images. Results The accuracy of our model for nAMD was 97%, and for DME, RVO, and control were 94%, 93%, and 93%, respectively. Area under curve values were 0.99, 0.98, 0.96, and 0.97, respectively. The mean Cohen's kappa coefficient for the multiclass classification task was 0.84. Conclusions The proposed DL model may identify OCT scans as normal and ME. In addition, it may classify its cause among three major exudative retinal diseases with high accuracy and reliability. Translational Relevance Our DL approach can optimize the efficiency and timeliness of appropriate etiological diagnosis of ME, thus improving patient access and clinical decision making. It could be useful in places with a shortage of specialists and for readers that evaluate OCT scans remotely.
Collapse
Affiliation(s)
| | - Yeison D Sanchez
- MindLab Research Group, Universidad Nacional de Colombia, Bogotá, Colombia
| | | | - Oscar J Perdomo
- School of Medicine and Health Sciences, Universidad del Rosario, Bogotá, Colombia
| | - Fabio A Gonzalez
- MindLab Research Group, Universidad Nacional de Colombia, Bogotá, Colombia
| |
Collapse
|
31
|
Tayal A, Gupta J, Solanki A, Bisht K, Nayyar A, Masud M. DL-CNN-based approach with image processing techniques for diagnosis of retinal diseases. MULTIMEDIA SYSTEMS 2022; 28:1417-1438. [DOI: 10.1007/s00530-021-00769-7] [Citation(s) in RCA: 13] [Impact Index Per Article: 6.5] [Reference Citation Analysis] [Track Full Text] [Subscribe] [Scholar Register] [Received: 10/15/2020] [Accepted: 02/24/2021] [Indexed: 08/29/2023]
|
32
|
Sun K, He M, He Z, Liu H, Pi X. EfficientNet embedded with spatial attention for recognition of multi-label fundus disease from color fundus photographs. Biomed Signal Process Control 2022. [DOI: 10.1016/j.bspc.2022.103768] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/24/2022]
|
33
|
Ma Z, Xie Q, Xie P, Fan F, Gao X, Zhu J. HCTNet: A Hybrid ConvNet-Transformer Network for Retinal Optical Coherence Tomography Image Classification. BIOSENSORS 2022; 12:542. [PMID: 35884345 PMCID: PMC9313149 DOI: 10.3390/bios12070542] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 06/09/2022] [Revised: 07/13/2022] [Accepted: 07/18/2022] [Indexed: 06/15/2023]
Abstract
Automatic and accurate optical coherence tomography (OCT) image classification is of great significance to computer-assisted diagnosis of retinal disease. In this study, we propose a hybrid ConvNet-Transformer network (HCTNet) and verify the feasibility of a Transformer-based method for retinal OCT image classification. The HCTNet first utilizes a low-level feature extraction module based on the residual dense block to generate low-level features for facilitating the network training. Then, two parallel branches of the Transformer and the ConvNet are designed to exploit the global and local context of the OCT images. Finally, a feature fusion module based on an adaptive re-weighting mechanism is employed to combine the extracted global and local features for predicting the category of OCT images in the testing datasets. The HCTNet combines the advantage of the convolutional neural network in extracting local features and the advantage of the vision Transformer in establishing long-range dependencies. A verification on two public retinal OCT datasets shows that our HCTNet method achieves an overall accuracy of 91.56% and 86.18%, respectively, outperforming the pure ViT and several ConvNet-based classification methods.
Collapse
Affiliation(s)
- Zongqing Ma
- Key Laboratory of the Ministry of Education for Optoelectronic Measurement Technology and Instrument, Beijing Information Science and Technology University, Beijing 100192, China; (Z.M.); (Q.X.); (F.F.)
- Beijing Laboratory of Biomedical Testing Technology and Instruments, Beijing Information Science and Technology University, Beijing 100192, China
| | - Qiaoxue Xie
- Key Laboratory of the Ministry of Education for Optoelectronic Measurement Technology and Instrument, Beijing Information Science and Technology University, Beijing 100192, China; (Z.M.); (Q.X.); (F.F.)
- Beijing Laboratory of Biomedical Testing Technology and Instruments, Beijing Information Science and Technology University, Beijing 100192, China
| | - Pinxue Xie
- Beijing Anzhen Hospital, Capital Medical University, Beijing 100029, China; (P.X.); (X.G.)
| | - Fan Fan
- Key Laboratory of the Ministry of Education for Optoelectronic Measurement Technology and Instrument, Beijing Information Science and Technology University, Beijing 100192, China; (Z.M.); (Q.X.); (F.F.)
- Beijing Laboratory of Biomedical Testing Technology and Instruments, Beijing Information Science and Technology University, Beijing 100192, China
| | - Xinxiao Gao
- Beijing Anzhen Hospital, Capital Medical University, Beijing 100029, China; (P.X.); (X.G.)
| | - Jiang Zhu
- Key Laboratory of the Ministry of Education for Optoelectronic Measurement Technology and Instrument, Beijing Information Science and Technology University, Beijing 100192, China; (Z.M.); (Q.X.); (F.F.)
- Beijing Laboratory of Biomedical Testing Technology and Instruments, Beijing Information Science and Technology University, Beijing 100192, China
| |
Collapse
|
34
|
Esfahani EN, Daneshmand PG, Rabbani H, Plonka G. Automatic Classification of Macular Diseases from OCT Images Using CNN Guided with Edge Convolutional Layer. ANNUAL INTERNATIONAL CONFERENCE OF THE IEEE ENGINEERING IN MEDICINE AND BIOLOGY SOCIETY. IEEE ENGINEERING IN MEDICINE AND BIOLOGY SOCIETY. ANNUAL INTERNATIONAL CONFERENCE 2022; 2022:3858-3861. [PMID: 36085830 DOI: 10.1109/embc48229.2022.9871322] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/15/2023]
Abstract
Optical Coherence Tomography (OCT) is a non-invasive imaging technology that is widely applied for the diagnosis of retinal pathologies. In general, the structural information of retinal layers plays an important role in the diagnosis of various eye diseases by ophthalmologists. In this paper, by focusing on this information, we first introduce a new layer called the edge convolutional layer (ECL) to accurately extract the retinal boundaries in different sizes and angles with a much smaller number of parameters than the conventional convolutional layer. Then, using this layer, we propose the ECL-guided convolutional neural network (ECL-CNN) method for the automatic classification of the OCT images. For the assessment of the proposed method, we utilize a publicly available data comprising 45 OCT volumes with 15 age-related macular degeneration (AMD), 15 diabetic macular edema (DME), and 15 normal volumes, captured by using the Heidelberg OCT imaging device. Experimental results demonstrate that the suggested ECL-CNN approach has an outstanding performance in OCT image classification, which achieves an average precision of 99.43% as a three-class classification work. Clinical Relevance - The objective of this research is to introduce a new approach based on CNN for the automated classification of retinal OCT images. Clinically, the ophthalmologists should manually check each cross-sectional B-scan and classify retinal pathologies from B-scan images. This manual process is tedious and time-consuming in general. Hence, an automatic computer-assisted technique for retinal OCT image classification is demanded.
Collapse
|
35
|
von der Burchard C, Sudkamp H, Tode J, Ehlken C, Purtskhvanidze K, Moltmann M, Heimes B, Koch P, Münst M, Vom Endt M, Kepp T, Theisen-Kunde D, König I, Hüttmann G, Roider J. Self-Examination Low-Cost Full-Field Optical Coherence Tomography (SELFF-OCT) for neovascular age-related macular degeneration: a cross-sectional diagnostic accuracy study. BMJ Open 2022; 12:e055082. [PMID: 35760534 PMCID: PMC9237881 DOI: 10.1136/bmjopen-2021-055082] [Citation(s) in RCA: 4] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 07/26/2021] [Accepted: 05/04/2022] [Indexed: 11/19/2022] Open
Abstract
OBJECTIVES Self-Examination Low-Cost Full-Field Optical Coherence Tomography (SELFF-OCT) is a novel OCT technology that was specifically designed for home monitoring of neovascular age-related macular degeneration (AMD). First clinical findings have been reported before. This trial investigates an improved prototype for patients with AMD and focusses on device operability and diagnostic accuracy compared with established spectral-domain OCT (SD-OCT). DESIGN Prospective single-arm diagnostic accuracy study. SETTING Tertiary care centre (University Eye Clinic). PARTICIPANTS 46 patients with age-related macular degeneration. INTERVENTIONS Patients received short training in device handling and then performed multiple self-scans with the SELFF-OCT according to a predefined protocol. Additionally, all eyes were examined with standard SD-OCT, performed by medical personnel. All images were graded by at least 2 masked investigators in a reading centre. PRIMARY OUTCOME MEASURE Rate of successful self-measurements. SECONDARY OUTCOME MEASURES Sensitivity and specificity of SELFF-OCT versus SD-OCT for different biomarkers and necessity for antivascular endothelial growth factor (anti-VEGF) treatment. RESULTS In 86% of all examined eyes, OCT self-acquisition resulted in interpretable retinal OCT volume scans. In these patients, the sensitivity for detection of anti-VEGF treatment necessity was 0.94 (95% CI 0.79 to 0.99) and specificity 0.95 (95% CI 0.82 to 0.99). CONCLUSIONS SELFF-OCT was used successfully for retinal self-examination in most patients, and it could become a valuable tool for retinal home monitoring in the future. Improvements are in progress to reduce device size and to improve handling, image quality and success rates. TRIAL REGISTRATION NUMBER DRKS00013755, CIV-17-12-022384.
Collapse
Affiliation(s)
| | | | - Jan Tode
- Department of Ophthalmology, Universitätsklinikum Schleswig-Holstein, Kiel, Germany
- Department of Ophthalmology, Hannover Medical School, Hannover, Germany
| | - Cristoph Ehlken
- Department of Ophthalmology, Universitätsklinikum Schleswig-Holstein, Kiel, Germany
| | | | | | - Britta Heimes
- Department of Ophthalmology, St. Franziskus-Hospital, Münster, Germany
| | - Peter Koch
- Medical Laser Center Lübeck GmbH, Lübeck, Germany
| | | | | | - Timo Kepp
- Medical Laser Center Lübeck GmbH, Lübeck, Germany
| | | | - Inke König
- Institute of Medical Biometry and Statistics, University of Lübeck, Lubeck, Germany
| | - Gereon Hüttmann
- Medical Laser Center Lübeck GmbH, Lübeck, Germany
- Institute of Biomedical Optics, University of Lübeck, Lubeck, Germany
| | - Johann Roider
- Department of Ophthalmology, Universitätsklinikum Schleswig-Holstein, Kiel, Germany
| |
Collapse
|
36
|
Ai Z, Huang X, Feng J, Wang H, Tao Y, Zeng F, Lu Y. FN-OCT: Disease Detection Algorithm for Retinal Optical Coherence Tomography Based on a Fusion Network. Front Neuroinform 2022; 16:876927. [PMID: 35784186 PMCID: PMC9243322 DOI: 10.3389/fninf.2022.876927] [Citation(s) in RCA: 12] [Impact Index Per Article: 6.0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/16/2022] [Accepted: 05/04/2022] [Indexed: 01/31/2023] Open
Abstract
Optical coherence tomography (OCT) is a new type of tomography that has experienced rapid development and potential in recent years. It is playing an increasingly important role in retinopathy diagnoses. At present, due to the uneven distributions of medical resources in various regions, the uneven proficiency levels of doctors in grassroots and remote areas, and the development needs of rare disease diagnosis and precision medicine, artificial intelligence technology based on deep learning can provide fast, accurate, and effective solutions for the recognition and diagnosis of retinal OCT images. To prevent vision damage and blindness caused by the delayed discovery of retinopathy, a fusion network (FN)-based retinal OCT classification algorithm (FN-OCT) is proposed in this paper to improve upon the adaptability and accuracy of traditional classification algorithms. The InceptionV3, Inception-ResNet, and Xception deep learning algorithms are used as base classifiers, a convolutional block attention mechanism (CBAM) is added after each base classifier, and three different fusion strategies are used to merge the prediction results of the base classifiers to output the final prediction results (choroidal neovascularization (CNV), diabetic macular oedema (DME), drusen, normal). The results show that in a classification problem involving the UCSD common retinal OCT dataset (108,312 OCT images from 4,686 patients), compared with that of the InceptionV3 network model, the prediction accuracy of FN-OCT is improved by 5.3% (accuracy = 98.7%, area under the curve (AUC) = 99.1%). The predictive accuracy and AUC achieved on an external dataset for the classification of retinal OCT diseases are 92 and 94.5%, respectively, and gradient-weighted class activation mapping (Grad-CAM) is used as a visualization tool to verify the effectiveness of the proposed FNs. This finding indicates that the developed fusion algorithm can significantly improve the performance of classifiers while providing a powerful tool and theoretical support for assisting with the diagnosis of retinal OCT.
Collapse
Affiliation(s)
- Zhuang Ai
- Department of Research and Development, Sinopharm Genomics Technology Co., Ltd., Jiangsu, China
| | - Xuan Huang
- Department of Ophthalmology, Beijing Chao-Yang Hospital, Capital Medical University, Beijing, China
- Medical Research Center, Beijing Chao-Yang Hospital, Capital Medical University, Beijing, China
| | - Jing Feng
- Department of Ophthalmology, Beijing Chao-Yang Hospital, Capital Medical University, Beijing, China
| | - Hui Wang
- Department of Ophthalmology, Beijing Chao-Yang Hospital, Capital Medical University, Beijing, China
| | - Yong Tao
- Department of Ophthalmology, Beijing Chao-Yang Hospital, Capital Medical University, Beijing, China
| | - Fanxin Zeng
- Department of Clinical Research Center, Dazhou Central Hospital, Sichuan, China
| | - Yaping Lu
- Department of Research and Development, Sinopharm Genomics Technology Co., Ltd., Jiangsu, China
| |
Collapse
|
37
|
Ou X, Gao L, Quan X, Zhang H, Yang J, Li W. BFENet: A two-stream interaction CNN method for multi-label ophthalmic diseases classification with bilateral fundus images. COMPUTER METHODS AND PROGRAMS IN BIOMEDICINE 2022; 219:106739. [PMID: 35344766 DOI: 10.1016/j.cmpb.2022.106739] [Citation(s) in RCA: 2] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 12/15/2021] [Revised: 02/23/2022] [Accepted: 03/07/2022] [Indexed: 06/14/2023]
Abstract
BACKGROUND AND OBJECTIVE Early fundus screening and timely treatment of ophthalmology diseases can effectively prevent blindness. Previous studies just focus on fundus images of single eye without utilizing the useful relevant information of the left and right eyes. While clinical ophthalmologists usually use binocular fundus images to help ocular disease diagnosis. Besides, previous works usually target only one ocular diseases at a time. Considering the importance of patient-level bilateral eye diagnosis and multi-label ophthalmic diseases classification, we propose a bilateral feature enhancement network (BFENet) to address the above two problems. METHODS We propose a two-stream interactive CNN architecture for multi-label ophthalmic diseases classification with bilateral fundus images. Firstly, we design a feature enhancement module, which makes use of the interaction between bilateral fundus images to strengthen the extracted feature information. Specifically, attention mechanism is used to learn the interdependence between local and global information in the designed interactive architecture for two-stream, which leads to the reweighting of these features, and recover more details. In order to capture more disease characteristics, we further design a novel multiscale module, which enriches the feature maps by superimposing feature information of different resolutions images extracted through dilated convolution. RESULTS In the off-site set, the Kappa, F1, AUC and Final score are 0.535, 0.892, 0.912 and 0.780, respectively. In the on-site set, the Kappa, F1, AUC and Final score are 0.513, 0.886, 0.903 and 0.767 respectively. Comparing with existing methods, BFENet achieves the best classification performance. CONCLUSIONS Comprehensive experiments are conducted to demonstrate the effectiveness of this proposed model. Besides, our method can be extended to similar tasks where the correlation between different images is important.
Collapse
Affiliation(s)
- Xingyuan Ou
- College of Artificial Intelligence, Nankai University, Tianjin, China
| | - Li Gao
- Ophthalmology, Tianjin Huanhu Hospital, Tianjin, China
| | - Xiongwen Quan
- College of Artificial Intelligence, Nankai University, Tianjin, China.
| | - Han Zhang
- College of Artificial Intelligence, Nankai University, Tianjin, China
| | - Jinglong Yang
- College of Artificial Intelligence, Nankai University, Tianjin, China
| | - Wei Li
- College of Artificial Intelligence, Nankai University, Tianjin, China
| |
Collapse
|
38
|
Wen H, Zhao J, Xiang S, Lin L, Liu C, Wang T, An L, Liang L, Huang B. Towards more efficient ophthalmic disease classification and lesion location via convolution transformer. COMPUTER METHODS AND PROGRAMS IN BIOMEDICINE 2022; 220:106832. [PMID: 35525213 DOI: 10.1016/j.cmpb.2022.106832] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 08/13/2021] [Revised: 04/01/2022] [Accepted: 04/21/2022] [Indexed: 06/14/2023]
Abstract
OBJECTIVE A retina optical coherence tomography (OCT) image differs from a traditional image due to its significant speckle noise, irregularity, and inconspicuous features. A conventional deep learning architecture cannot effectively improve the classification accuracy, sensitivity, and specificity of OCT images, and noisy images are not conducive to further diagnosis. This paper proposes a novel lesion-localization convolution transformer (LLCT) method, which combines both convolution and self-attention to classify ophthalmic diseases more accurately and localize the lesions in retina OCT images. METHODS A novel architecture design is accomplished through applying customized feature maps generated by convolutional neutral network (CNN) as the input sequence of self-attention network. This design takes advantages of CNN's extracting image features and transformer's consideration of global context and dynamic attention. Part of the model is backward propagated to calculate the gradient as a weight parameter, which is multiplied and summed with the global features generated by the forward propagation process to locate the lesion. RESULTS Extensive experiments show that our proposed design achieves improvement of about 7.6% in overall accuracy, 10.9% in overall sensitivity, and 9.2% in overall specificity compared with previous methods. And the lesions can be localized without the labeling data of lesion location in OCT images. CONCLUSION The results prove that our method significantly improves the performance and reduces the computation complexity in artificial intelligence assisted analysis of ophthalmic disease through OCT images. SIGNIFICANCE Our method has a significance boost in ophthalmic disease classification and location via convolution transformer. This is applicable to assist ophthalmologists greatly.1.
Collapse
Affiliation(s)
- Huajie Wen
- College of Big Data and Internet, Shenzhen Technology University, Shenzhen 518118, China; College of Applied Science, Shenzhen University, Shenzhen 518060, China
| | - Jian Zhao
- College of Big Data and Internet, Shenzhen Technology University, Shenzhen 518118, China
| | - Shaohua Xiang
- College of Big Data and Internet, Shenzhen Technology University, Shenzhen 518118, China
| | - Lin Lin
- College of Big Data and Internet, Shenzhen Technology University, Shenzhen 518118, China
| | - Chengjian Liu
- College of Big Data and Internet, Shenzhen Technology University, Shenzhen 518118, China
| | - Tao Wang
- College of Big Data and Internet, Shenzhen Technology University, Shenzhen 518118, China
| | - Lin An
- Guangdong Vision Medical Science & Technology Co., Ltd. Foshan 528000, China
| | - Lixin Liang
- College of Big Data and Internet, Shenzhen Technology University, Shenzhen 518118, China.
| | - Bingding Huang
- College of Big Data and Internet, Shenzhen Technology University, Shenzhen 518118, China.
| |
Collapse
|
39
|
Wang W, Li X, Xu Z, Yu W, Zhao J, Ding D, Chen Y. Learning Two-Stream CNN for Multi-Modal Age-related Macular Degeneration Categorization. IEEE J Biomed Health Inform 2022; 26:4111-4122. [PMID: 35503853 DOI: 10.1109/jbhi.2022.3171523] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/09/2022]
Abstract
This paper tackles automated categorization of Age-related Macular Degeneration (AMD), a common macular disease among people over 50. Previous research efforts mainly focus on AMD categorization with a single-modal input, let it be a color fundus photograph (CFP) or an OCT B-scan image. By contrast, we consider AMD categorization given a multi-modal input, a direction that is clinically meaningful yet mostly unexplored. Contrary to the prior art that takes a traditional approach of feature extraction plus classifier training that cannot be jointly optimized, we opt for end-to-end multi-modal Convolutional Neural Networks (MM-CNN). Our MM-CNN is instantiated by a two-stream CNN, with spatially-invariant fusion to combine information from the CFP and OCT streams. In order to visually interpret the contribution of the individual modalities to the final prediction, we extend the class activation mapping (CAM) technique to the multi-modal scenario. For effective training of MM-CNN, we develop two data augmentation methods. One is GAN-based CFP/OCT image synthesis, with our novel use of CAMs as conditional input of a high-resolution image-to-image translation GAN. The other method is Loose Pairing, which pairs a CFP image and an OCT image on the basis of their classes instead of eye identities. Experiments on a clinical dataset consisting of 1,094 CFP images and 1,289 OCT images acquired from 1,093 distinct eyes show that the proposed solution obtains better F1 and Accuracy than multiple baselines for multi-modal AMD categorization. Code and data are available at https://github.com/li-xirong/mmc-amd.
Collapse
|
40
|
Khan MS, Tafshir N, Alam KN, Dhruba AR, Khan MM, Albraikan AA, Almalki FA. Deep Learning for Ocular Disease Recognition: An Inner-Class Balance. COMPUTATIONAL INTELLIGENCE AND NEUROSCIENCE 2022; 2022:5007111. [PMID: 35528343 PMCID: PMC9071974 DOI: 10.1155/2022/5007111] [Citation(s) in RCA: 4] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 02/24/2022] [Revised: 03/18/2022] [Accepted: 04/12/2022] [Indexed: 12/25/2022]
Abstract
It can be challenging for doctors to identify eye disorders early enough using fundus pictures. Diagnosing ocular illnesses by hand is time-consuming, error-prone, and complicated. Therefore, an automated ocular disease detection system with computer-aided tools is necessary to detect various eye disorders using fundus pictures. Such a system is now possible as a consequence of deep learning algorithms that have improved image classification capabilities. A deep-learning-based approach to targeted ocular detection is presented in this study. For this study, we used state-of-the-art image classification algorithms, such as VGG-19, to classify the ODIR dataset, which contains 5000 images of eight different classes of the fundus. These classes represent different ocular diseases. However, the dataset within these classes is highly unbalanced. To resolve this issue, the work suggested converting this multiclass classification problem into a binary classification problem and taking the same number of images for both classifications. Then, the binary classifications were trained with VGG-19. The accuracy of the VGG-19 model was 98.13% for the normal (N) versus pathological myopia (M) class; the model reached an accuracy of 94.03% for normal (N) versus cataract (C), and the model provided an accuracy of 90.94% for normal (N) versus glaucoma (G). All of the other models also improve the accuracy when the data is balanced.
Collapse
Affiliation(s)
- Md Shakib Khan
- Department of Electrical and Computer Engineering, North South University, Bashundhara, Dhaka 1229, Bangladesh
| | - Nafisa Tafshir
- Department of Electrical and Computer Engineering, North South University, Bashundhara, Dhaka 1229, Bangladesh
| | - Kazi Nabiul Alam
- Department of Electrical and Computer Engineering, North South University, Bashundhara, Dhaka 1229, Bangladesh
| | - Abdur Rab Dhruba
- Department of Electrical and Computer Engineering, North South University, Bashundhara, Dhaka 1229, Bangladesh
| | - Mohammad Monirujjaman Khan
- Department of Electrical and Computer Engineering, North South University, Bashundhara, Dhaka 1229, Bangladesh
| | - Amani Abdulrahman Albraikan
- Department of Computer Science, College of Computer and Information Sciences, Princess Nourah Bint Abdulrahman University, P.O. Box 84428, Riyadh 11671, Saudi Arabia
| | - Faris A. Almalki
- Department of Computer Engineering, College of Computers and Information Technology, Taif University, P.O. Box 11099, Taif 21944, Saudi Arabia
| |
Collapse
|
41
|
Kim HE, Cosa-Linan A, Santhanam N, Jannesari M, Maros ME, Ganslandt T. Transfer learning for medical image classification: a literature review. BMC Med Imaging 2022; 22:69. [PMID: 35418051 PMCID: PMC9007400 DOI: 10.1186/s12880-022-00793-7] [Citation(s) in RCA: 113] [Impact Index Per Article: 56.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/25/2021] [Accepted: 03/30/2022] [Indexed: 02/07/2023] Open
Abstract
BACKGROUND Transfer learning (TL) with convolutional neural networks aims to improve performances on a new task by leveraging the knowledge of similar tasks learned in advance. It has made a major contribution to medical image analysis as it overcomes the data scarcity problem as well as it saves time and hardware resources. However, transfer learning has been arbitrarily configured in the majority of studies. This review paper attempts to provide guidance for selecting a model and TL approaches for the medical image classification task. METHODS 425 peer-reviewed articles were retrieved from two databases, PubMed and Web of Science, published in English, up until December 31, 2020. Articles were assessed by two independent reviewers, with the aid of a third reviewer in the case of discrepancies. We followed the PRISMA guidelines for the paper selection and 121 studies were regarded as eligible for the scope of this review. We investigated articles focused on selecting backbone models and TL approaches including feature extractor, feature extractor hybrid, fine-tuning and fine-tuning from scratch. RESULTS The majority of studies (n = 57) empirically evaluated multiple models followed by deep models (n = 33) and shallow (n = 24) models. Inception, one of the deep models, was the most employed in literature (n = 26). With respect to the TL, the majority of studies (n = 46) empirically benchmarked multiple approaches to identify the optimal configuration. The rest of the studies applied only a single approach for which feature extractor (n = 38) and fine-tuning from scratch (n = 27) were the two most favored approaches. Only a few studies applied feature extractor hybrid (n = 7) and fine-tuning (n = 3) with pretrained models. CONCLUSION The investigated studies demonstrated the efficacy of transfer learning despite the data scarcity. We encourage data scientists and practitioners to use deep models (e.g. ResNet or Inception) as feature extractors, which can save computational costs and time without degrading the predictive power.
Collapse
Affiliation(s)
- Hee E Kim
- Department of Biomedical Informatics at the Center for Preventive Medicine and Digital Health (CPD-BW), Medical Faculty Mannheim, Heidelberg University, Theodor-Kutzer-Ufer 1-3, 68167, Mannheim, Germany.
| | - Alejandro Cosa-Linan
- Department of Biomedical Informatics at the Center for Preventive Medicine and Digital Health (CPD-BW), Medical Faculty Mannheim, Heidelberg University, Theodor-Kutzer-Ufer 1-3, 68167, Mannheim, Germany
| | - Nandhini Santhanam
- Department of Biomedical Informatics at the Center for Preventive Medicine and Digital Health (CPD-BW), Medical Faculty Mannheim, Heidelberg University, Theodor-Kutzer-Ufer 1-3, 68167, Mannheim, Germany
| | - Mahboubeh Jannesari
- Department of Biomedical Informatics at the Center for Preventive Medicine and Digital Health (CPD-BW), Medical Faculty Mannheim, Heidelberg University, Theodor-Kutzer-Ufer 1-3, 68167, Mannheim, Germany
| | - Mate E Maros
- Department of Biomedical Informatics at the Center for Preventive Medicine and Digital Health (CPD-BW), Medical Faculty Mannheim, Heidelberg University, Theodor-Kutzer-Ufer 1-3, 68167, Mannheim, Germany
| | - Thomas Ganslandt
- Department of Biomedical Informatics at the Center for Preventive Medicine and Digital Health (CPD-BW), Medical Faculty Mannheim, Heidelberg University, Theodor-Kutzer-Ufer 1-3, 68167, Mannheim, Germany
- Chair of Medical Informatics, Friedrich-Alexander-Universität Erlangen-Nürnberg, Wetterkreuz 15, 91058, Erlangen, Germany
| |
Collapse
|
42
|
Research on Improved Deep Convolutional Generative Adversarial Networks for Insufficient Samples of Gas Turbine Rotor System Fault Diagnosis. APPLIED SCIENCES-BASEL 2022. [DOI: 10.3390/app12073606] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 02/01/2023]
Abstract
In gas turbine rotor systems, an intelligent data-driven fault diagnosis method is an important means to monitor the health status of the gas turbine, and it is necessary to obtain sufficient fault data to train the intelligent diagnosis model. In the actual operation of a gas turbine, the collected gas turbine fault data are limited, and the small and imbalanced fault samples seriously affect the accuracy of the fault diagnosis method. Focusing on the imbalance of gas turbine fault data, an Improved Deep Convolutional Generative Adversarial Network (Improved DCGAN) suitable for gas turbine signals is proposed here, and a structural optimization of the generator and a gradient penalty improvement in the loss function are introduced to generate effective fault data and improve the classification accuracy. The experimental results of the gas turbine test bench demonstrate that the proposed method can generate effective fault samples as a supplementary set of fault samples to balance the dataset, effectively improve the fault classification and diagnosis performance of gas turbine rotors in the case of small samples, and provide an effective method for gas turbine fault diagnosis.
Collapse
|
43
|
Saha D, Manickavasagan A. Chickpea varietal classification using deep convolutional neural networks with transfer learning. J FOOD PROCESS ENG 2022. [DOI: 10.1111/jfpe.13975] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/12/2022]
Affiliation(s)
- Dhritiman Saha
- School of Engineering University of Guelph Guelph Ontario Canada
- Food Grains & Oilseeds Processing Division ICAR—Central Institute of Post‐Harvest Engineering and Technology (CIPHET) Ludhiana Punjab India
| | | |
Collapse
|
44
|
Al Turk L, Georgieva D, Alsawadi H, Wang S, Krause P, Alsawadi H, Alshamrani AZ, Saleh GM, Tang HL. Learning to Discover Explainable Clinical Features With Minimum Supervision. Transl Vis Sci Technol 2022; 11:11. [PMID: 35015061 PMCID: PMC8762682 DOI: 10.1167/tvst.11.1.11] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/24/2022] Open
Abstract
Purpose To compare supervised transfer learning to semisupervised learning for their ability to learn in-depth knowledge with limited data in the optical coherence tomography (OCT) domain. Methods Transfer learning with EfficientNet-B4 and semisupervised learning with SimCLR are used in this work. The largest public OCT dataset, consisting of 108,312 images and four categories (choroidal neovascularization, diabetic macular edema, drusen, and normal) is used. In addition, two smaller datasets are constructed, containing 31,200 images for the limited version and 4000 for the mini version of the dataset. To illustrate the effectiveness of the developed models, local interpretable model-agnostic explanations and class activation maps are used as explainability techniques. Results The proposed transfer learning approach using the EfficientNet-B4 model trained on the limited dataset achieves an accuracy of 0.976 (95% confidence interval [CI], 0.963, 0.983), sensitivity of 0.973 and specificity of 0.991. The semisupervised based solution with SimCLR using 10% labeled data and the limited dataset performs with an accuracy of 0.946 (95% CI, 0.932, 0.960), sensitivity of 0.941, and specificity of 0.983. Conclusions Semisupervised learning has a huge potential for datasets that contain both labeled and unlabeled inputs, generally, with a significantly smaller number of labeled samples. The semisupervised based solution provided with merely 10% labeled data achieves very similar performance to the supervised transfer learning that uses 100% labeled samples. Translational Relevance Semisupervised learning enables building performant models while requiring less expertise effort and time by using to good advantage the abundant amount of available unlabeled data along with the labeled samples.
Collapse
Affiliation(s)
- Lutfiah Al Turk
- Department of Statistics, King Abdulaziz University, Jeddah, Kingdom of Saudi Arabia
| | - Darina Georgieva
- Department of Computer Science, University of Surrey, Guildford, Surrey, UK
| | - Hassan Alsawadi
- Department of Electrical and Computer Engineering, King Abdulaziz, University, Jeddah, Kingdom of Saudi Arabia
| | - Su Wang
- Department of Computer Science, University of Surrey, Guildford, Surrey, UK
| | - Paul Krause
- Department of Computer Science, University of Surrey, Guildford, Surrey, UK
| | - Hend Alsawadi
- Faculty of Medicine, King Abdulaziz University, Jeddah, Kingdom of Saudi Arabia
| | | | - George M Saleh
- NIHR Biomedical Research Centre at Moorfields Eye Hospital and the UCL Institute of Ophthalmology, UK
| | | |
Collapse
|
45
|
Shao A, Jin K, Li Y, Lou L, Zhou W, Ye J. Overview of global publications on machine learning in diabetic retinopathy from 2011 to 2021: Bibliometric analysis. Front Endocrinol (Lausanne) 2022; 13:1032144. [PMID: 36589855 PMCID: PMC9797582 DOI: 10.3389/fendo.2022.1032144] [Citation(s) in RCA: 4] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 08/30/2022] [Accepted: 12/05/2022] [Indexed: 12/23/2022] Open
Abstract
PURPOSE To comprehensively analyze and discuss the publications on machine learning (ML) in diabetic retinopathy (DR) following a bibliometric approach. METHODS The global publications on ML in DR from 2011 to 2021 were retrieved from the Web of Science Core Collection (WoSCC) database. We analyzed the publication and citation trend over time and identified highly-cited articles, prolific countries, institutions, journals and the most relevant research domains. VOSviewer and Wordcloud are used to visualize the mainstream research topics and evolution of subtopics in the form of co-occurrence maps of keywords. RESULTS By analyzing a total of 1147 relevant publications, this study found a rapid increase in the number of annual publications, with an average growth rate of 42.68%. India and China were the most productive countries. IEEE Access was the most productive journal in this field. In addition, some notable common points were found in the highly-cited articles. The keywords analysis showed that "diabetic retinopathy", "classification", and "fundus images" were the most frequent keywords for the entire period, as automatic diagnosis of DR was always the mainstream topic in the relevant field. The evolution of keywords highlighted some breakthroughs, including "deep learning" and "optical coherence tomography", indicating the advance in technologies and changes in the research attention. CONCLUSIONS As new research topics have emerged and evolved, studies are becoming increasingly diverse and extensive. Multiple modalities of medical data, new ML techniques and constantly optimized algorithms are the future trends in this multidisciplinary field.
Collapse
Affiliation(s)
- An Shao
- Department of Ophthalmology, the Second Affiliated Hospital of Zhejiang University, College of Medicine, Hangzhou, China
| | - Kai Jin
- Department of Ophthalmology, the Second Affiliated Hospital of Zhejiang University, College of Medicine, Hangzhou, China
| | - Yunxiang Li
- College of Computer Science and Technology, Hangzhou Dianzi University, Hangzhou, China
| | - Lixia Lou
- Department of Ophthalmology, the Second Affiliated Hospital of Zhejiang University, College of Medicine, Hangzhou, China
| | - Wuyuan Zhou
- Zhejiang Academy of Science and Technology Information, Hangzhou, China
- *Correspondence: Juan Ye, ; Wuyuan Zhou,
| | - Juan Ye
- Department of Ophthalmology, the Second Affiliated Hospital of Zhejiang University, College of Medicine, Hangzhou, China
- *Correspondence: Juan Ye, ; Wuyuan Zhou,
| |
Collapse
|
46
|
Chen Q, Xue B, Zhang M. Genetic Programming for Instance Transfer Learning in Symbolic Regression. IEEE TRANSACTIONS ON CYBERNETICS 2022; 52:25-38. [PMID: 32092029 DOI: 10.1109/tcyb.2020.2969689] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/10/2023]
Abstract
Transfer learning has attracted more attention in the machine-learning community recently. It aims to improve the learning performance on the domain of interest with the help of the knowledge acquired from a similar domain(s). However, there is only a limited number of research on tackling transfer learning in genetic programming for symbolic regression. This article attempts to fill this gap by proposing a new instance weighting framework for transfer learning in genetic programming-based symbolic regression. In the new framework, differential evolution is employed to search for optimal weights for source-domain instances, which helps genetic programming to identify more useful source-domain instances and learn from them. Meanwhile, a density estimation method is used to provide good starting points to help the search for the optimal weights while discarding some irrelevant or less important source-domain instances before learning regression models. The experimental results show that compared with genetic programming and support vector regression that learn only from the target instances, and learning from a mixture of instances from the source and target domains without any transfer learning component, the proposed method can evolve regression models which not only achieve notably better cross-domain generalization performance in stability but also reduce the trend of overfitting effectively. Meanwhile, these models are generally much simpler than those generated by the other GP methods.
Collapse
|
47
|
|
48
|
Xu L, Wang L, Cheng S, Li Y. MHANet: A hybrid attention mechanism for retinal diseases classification. PLoS One 2021; 16:e0261285. [PMID: 34914763 PMCID: PMC8675717 DOI: 10.1371/journal.pone.0261285] [Citation(s) in RCA: 7] [Impact Index Per Article: 2.3] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/06/2021] [Accepted: 11/26/2021] [Indexed: 12/04/2022] Open
Abstract
With the increase of patients with retinopathy, retinopathy recognition has become a research hotspot. In this article, we describe the etiology and symptoms of three kinds of retinal diseases, including drusen(DRUSEN), choroidal neovascularization(CNV) and diabetic macular edema(DME). In addition, we also propose a hybrid attention mechanism to classify and recognize different types of retinopathy images. In particular, the hybrid attention mechanism proposed in this paper includes parallel spatial attention mechanism and channel attention mechanism. It can extract the key features in the channel dimension and spatial dimension of retinopathy images, and reduce the negative impact of background information on classification results. The experimental results show that the hybrid attention mechanism proposed in this paper can better assist the network to focus on extracting thr fetures of the retinopathy area and enhance the adaptability to the differences of different data sets. Finally, the hybrid attention mechanism achieved 96.5% and 99.76% classification accuracy on two public OCT data sets of retinopathy, respectively.
Collapse
Affiliation(s)
- Lianghui Xu
- College of Information Science and Engineering, Xinjiang University, Urumqi, China
| | - Liejun Wang
- College of Information Science and Engineering, Xinjiang University, Urumqi, China
- * E-mail:
| | - Shuli Cheng
- College of Information Science and Engineering, Xinjiang University, Urumqi, China
| | - Yongming Li
- College of Information Science and Engineering, Xinjiang University, Urumqi, China
| |
Collapse
|
49
|
Wang YZ, Wu W, Birch DG. A Hybrid Model Composed of Two Convolutional Neural Networks (CNNs) for Automatic Retinal Layer Segmentation of OCT Images in Retinitis Pigmentosa (RP). Transl Vis Sci Technol 2021; 10:9. [PMID: 34751740 PMCID: PMC8590180 DOI: 10.1167/tvst.10.13.9] [Citation(s) in RCA: 5] [Impact Index Per Article: 1.7] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/23/2022] Open
Abstract
Purpose We propose and evaluate a hybrid model composed of two convolutional neural networks (CNNs) with different architectures for automatic segmentation of retina layers in spectral domain optical coherence tomography (SD-OCT) B-scans of retinitis pigmentosa (RP). Methods The hybrid model consisted of a U-Net for initial semantic segmentation and a sliding-window (SW) CNN for refinement by correcting the segmentation errors of U-Net. The U-Net construction followed Ronneberger et al. (2015) with an input image size of 256 × 32. The SW model was similar to our previously reported approach. Training image patches were generated from 480 horizontal midline B-scans obtained from 220 patients with RP and 20 normal participants. Testing images were 160 midline B-scans from a separate group of 80 patients with RP. The Spectralis segmentation of B-scans was manually corrected for the boundaries of the inner limiting membrane, inner nuclear layer, ellipsoid zone (EZ), retinal pigment epithelium, and Bruch's membrane by one grader for the training set and two for the testing set. The trained U-Net and SW, as well as the hybrid model, were used to classify all pixels in the testing B-scans. Bland–Altman and correlation analyses were conducted to compare layer boundary lines, EZ width, and photoreceptor outer segment (OS) length and area determined by the models to those by human graders. Results The mean times to classify a B-scan image were 0.3, 65.7, and 2.4 seconds for U-Net, SW, and the hybrid model, respectively. The mean ± SD accuracies to segment retinal layers were 90.8% ± 4.8% and 90.7% ± 4.0% for U-Net and SW, respectively. The hybrid model improved mean ± SD accuracy to 91.5% ± 4.8% (P < 0.039 vs. U-Net), resulting in an improvement in layer boundary segmentation as revealed by Bland–Altman analyses. EZ width, OS length, and OS area measured by the models were highly correlated with those measured by the human graders (r > 0.95 for EZ width; r > 0.83 for OS length; r > 0.97 for OS area; P < 0.05). The hybrid model further improved the performance of measuring retinal layer thickness by correcting misclassification of retinal layers from U-Net. Conclusions While the performances of U-Net and the SW model were comparable in delineating various retinal layers, U-Net was much faster than the SW model to segment B-scan images. The hybrid model that combines the two improves automatic retinal layer segmentation from OCT images in RP. Translational Relevance A hybrid deep machine learning model composed of CNNs with different architectures can be more effective than either model separately for automatic analysis of SD-OCT scan images, which is becoming increasingly necessary with current high-resolution, high-density volume scans.
Collapse
Affiliation(s)
- Yi-Zhong Wang
- Retina Foundation of the Southwest, Dallas, TX, USA.,Department of Ophthalmology, University of Texas Southwestern Medical Center at Dallas, Dallas, TX, USA
| | - Wenxuan Wu
- Retina Foundation of the Southwest, Dallas, TX, USA
| | - David G Birch
- Retina Foundation of the Southwest, Dallas, TX, USA.,Department of Ophthalmology, University of Texas Southwestern Medical Center at Dallas, Dallas, TX, USA
| |
Collapse
|
50
|
Atteia G, Abdel Samee N, Zohair Hassan H. DFTSA-Net: Deep Feature Transfer-Based Stacked Autoencoder Network for DME Diagnosis. ENTROPY 2021; 23:e23101251. [PMID: 34681974 PMCID: PMC8534911 DOI: 10.3390/e23101251] [Citation(s) in RCA: 7] [Impact Index Per Article: 2.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 08/28/2021] [Revised: 09/14/2021] [Accepted: 09/23/2021] [Indexed: 12/13/2022]
Abstract
Diabetic macular edema (DME) is the most common cause of irreversible vision loss in diabetes patients. Early diagnosis of DME is necessary for effective treatment of the disease. Visual detection of DME in retinal screening images by ophthalmologists is a time-consuming process. Recently, many computer-aided diagnosis systems have been developed to assist doctors by detecting DME automatically. In this paper, a new deep feature transfer-based stacked autoencoder neural network system is proposed for the automatic diagnosis of DME in fundus images. The proposed system integrates the power of pretrained convolutional neural networks as automatic feature extractors with the power of stacked autoencoders in feature selection and classification. Moreover, the system enables extracting a large set of features from a small input dataset using four standard pretrained deep networks: ResNet-50, SqueezeNet, Inception-v3, and GoogLeNet. The most informative features are then selected by a stacked autoencoder neural network. The stacked network is trained in a semi-supervised manner and is used for the classification of DME. It is found that the introduced system achieves a maximum classification accuracy of 96.8%, sensitivity of 97.5%, and specificity of 95.5%. The proposed system shows a superior performance over the original pretrained network classifiers and state-of-the-art findings.
Collapse
Affiliation(s)
- Ghada Atteia
- Information Technology Department, College of Computer and Information Sciences, Princess Nourah Bint Abdulrahman University, Riyadh 11461, Saudi Arabia;
- Correspondence: or
| | - Nagwan Abdel Samee
- Information Technology Department, College of Computer and Information Sciences, Princess Nourah Bint Abdulrahman University, Riyadh 11461, Saudi Arabia;
- Computer Engineering Department, Misr University for Science and Technology, Giza 12511, Egypt
| | - Hassan Zohair Hassan
- Department of Mechanical Engineering, College of Engineering, Alfaisal University, Takhassusi Street, P.O. Box 50927, Riyadh 11533, Saudi Arabia;
| |
Collapse
|