51
|
Morya AK, Janti SS, Sisodiya P, Tejaswini A, Prasad R, Mali KR, Gurnani B. Everything real about unreal artificial intelligence in diabetic retinopathy and in ocular pathologies. World J Diabetes 2022; 13:822-834. [PMID: 36311999 PMCID: PMC9606792 DOI: 10.4239/wjd.v13.i10.822] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 05/25/2022] [Revised: 08/11/2022] [Accepted: 09/10/2022] [Indexed: 02/05/2023] Open
Abstract
Artificial Intelligence is a multidisciplinary field with the aim of building platforms that can make machines act, perceive, reason intelligently and whose goal is to automate activities that presently require human intelligence. From the cornea to the retina, artificial intelligence (AI) is expected to help ophthalmologists diagnose and treat ocular diseases. In ophthalmology, computerized analytics are being viewed as efficient and more objective ways to interpret the series of images and come to a conclusion. AI can be used to diagnose and grade diabetic retinopathy, glaucoma, age-related macular degeneration, cataracts, IOL power calculation, retinopathy of prematurity and keratoconus. This review article intends to discuss various aspects of artificial intelligence in ophthalmology.
Collapse
Affiliation(s)
- Arvind Kumar Morya
- Department of Ophthalmology, All India Institute of Medical Sciences Bibinagar, Hyderabad 508126, Telangana, India
| | - Siddharam S Janti
- Department of Ophthalmology, All India Institute of Medical Sciences Bibinagar, Hyderabad 508126, Telangana, India
| | - Priya Sisodiya
- Department of Ophthalmology, Sadguru Netra Chikitsalaya, Chitrakoot 485001, Madhya Pradesh, India
| | - Antervedi Tejaswini
- Department of Ophthalmology, All India Institute of Medical Sciences Bibinagar, Hyderabad 508126, Telangana, India
| | - Rajendra Prasad
- Department of Ophthalmology, R P Eye Institute, New Delhi 110001, New Delhi, India
| | - Kalpana R Mali
- Department of Pharmacology, All India Institute of Medical Sciences, Bibinagar, Hyderabad 508126, Telangana, India
| | - Bharat Gurnani
- Department of Ophthalmology, Aravind Eye Hospital and Post Graduate Institute of Ophthalmology, Pondicherry 605007, Pondicherry, India
| |
Collapse
|
52
|
Wongchaisuwat P, Thamphithak R, Jitpukdee P, Wongchaisuwat N. Application of Deep Learning for Automated Detection of Polypoidal Choroidal Vasculopathy in Spectral Domain Optical Coherence Tomography. Transl Vis Sci Technol 2022; 11:16. [PMID: 36219163 PMCID: PMC9580222 DOI: 10.1167/tvst.11.10.16] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/25/2022] Open
Abstract
Objective To develop an automated polypoidal choroidal vasculopathy (PCV) screening model to distinguish PCV from wet age-related macular degeneration (wet AMD). Methods A retrospective review of spectral domain optical coherence tomography (SD-OCT) images was undertaken. The included SD-OCT images were classified into two distinct categories (PCV or wet AMD) prior to the development of the PCV screening model. The automated detection of PCV using the developed model was compared with the results of gold-standard fundus fluorescein angiography and indocyanine green (FFA + ICG) angiography. A framework of SHapley Additive exPlanations was used to interpret the results from the model. Results A total of 2334 SD-OCT images were enrolled for training purposes, and an additional 1171 SD-OCT images were used for external validation. The ResNet attention model yielded superior performance with average area under the curve values of 0.8 and 0.81 for the training and external validation data sets, respectively. The sensitivity/specificity calculated at a patient level was 100%/60% and 85%/71% for the training and external validation data sets, respectively. Conclusions A conventional FFA + ICG investigation to differentiate PCV from wet AMD requires intense health care resources and adversely affects patients. A deep learning algorithm is proposed to automatically distinguish PCV from wet AMD. The developed algorithm exhibited promising performance for further development into an alternative PCV screening tool. Enhancement of the model's performance with additional data is needed prior to implementation of this diagnostic tool in real-world clinical practice. The invisibility of disease signs within SD-OCT images is the main limitation of the proposed model. Translational Relevance Basic research of deep learning algorithms was applied to differentiate PCV from wet AMD based on OCT images, benefiting a diagnosis process and minimizing a risk of ICG angiogram.
Collapse
Affiliation(s)
- Papis Wongchaisuwat
- Department of Industrial Engineering, Faculty of Engineering, Kasetsart University, Bangkok, Thailand
| | - Ranida Thamphithak
- Department of Ophthalmology, Faculty of Medicine Siriraj Hospital, Mahidol University, Bangkok, Thailand
| | - Peerakarn Jitpukdee
- Department of Industrial Engineering, Faculty of Engineering, Kasetsart University, Bangkok, Thailand
| | - Nida Wongchaisuwat
- Department of Ophthalmology, Faculty of Medicine Siriraj Hospital, Mahidol University, Bangkok, Thailand
| |
Collapse
|
53
|
Self-supervised patient-specific features learning for OCT image classification. Med Biol Eng Comput 2022; 60:2851-2863. [DOI: 10.1007/s11517-022-02627-8] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/13/2021] [Accepted: 04/28/2022] [Indexed: 11/26/2022]
|
54
|
|
55
|
Tampu IE, Eklund A, Haj-Hosseini N. Inflation of test accuracy due to data leakage in deep learning-based classification of OCT images. Sci Data 2022; 9:580. [PMID: 36138025 PMCID: PMC9500039 DOI: 10.1038/s41597-022-01618-6] [Citation(s) in RCA: 14] [Impact Index Per Article: 7.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/15/2022] [Accepted: 08/09/2022] [Indexed: 11/10/2022] Open
Abstract
In the application of deep learning on optical coherence tomography (OCT) data, it is common to train classification networks using 2D images originating from volumetric data. Given the micrometer resolution of OCT systems, consecutive images are often very similar in both visible structures and noise. Thus, an inappropriate data split can result in overlap between the training and testing sets, with a large portion of the literature overlooking this aspect. In this study, the effect of improper dataset splitting on model evaluation is demonstrated for three classification tasks using three OCT open-access datasets extensively used, Kermany's and Srinivasan's ophthalmology datasets, and AIIMS breast tissue dataset. Results show that the classification performance is inflated by 0.07 up to 0.43 in terms of Matthews Correlation Coefficient (accuracy: 5% to 30%) for models tested on datasets with improper splitting, highlighting the considerable effect of dataset handling on model evaluation. This study intends to raise awareness on the importance of dataset splitting given the increased research interest in implementing deep learning on OCT data.
Collapse
Affiliation(s)
- Iulian Emil Tampu
- Department of Biomedical Engineering, Linköping University, 581 85, Linköping, Sweden. .,Center for Medical Image Science and Visualization, Linköping University, 581 85, Linköping, Sweden.
| | - Anders Eklund
- Department of Biomedical Engineering, Linköping University, 581 85, Linköping, Sweden.,Center for Medical Image Science and Visualization, Linköping University, 581 85, Linköping, Sweden.,Division of Statistics & Machine Learning, Department of Computer and Information Science, Linköping University, 581 83, Linköping, Sweden
| | - Neda Haj-Hosseini
- Department of Biomedical Engineering, Linköping University, 581 85, Linköping, Sweden.,Center for Medical Image Science and Visualization, Linköping University, 581 85, Linköping, Sweden
| |
Collapse
|
56
|
The Role of Medical Image Modalities and AI in the Early Detection, Diagnosis and Grading of Retinal Diseases: A Survey. Bioengineering (Basel) 2022; 9:bioengineering9080366. [PMID: 36004891 PMCID: PMC9405367 DOI: 10.3390/bioengineering9080366] [Citation(s) in RCA: 4] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/03/2022] [Revised: 07/28/2022] [Accepted: 08/01/2022] [Indexed: 11/16/2022] Open
Abstract
Traditional dilated ophthalmoscopy can reveal diseases, such as age-related macular degeneration (AMD), diabetic retinopathy (DR), diabetic macular edema (DME), retinal tear, epiretinal membrane, macular hole, retinal detachment, retinitis pigmentosa, retinal vein occlusion (RVO), and retinal artery occlusion (RAO). Among these diseases, AMD and DR are the major causes of progressive vision loss, while the latter is recognized as a world-wide epidemic. Advances in retinal imaging have improved the diagnosis and management of DR and AMD. In this review article, we focus on the variable imaging modalities for accurate diagnosis, early detection, and staging of both AMD and DR. In addition, the role of artificial intelligence (AI) in providing automated detection, diagnosis, and staging of these diseases will be surveyed. Furthermore, current works are summarized and discussed. Finally, projected future trends are outlined. The work done on this survey indicates the effective role of AI in the early detection, diagnosis, and staging of DR and/or AMD. In the future, more AI solutions will be presented that hold promise for clinical applications.
Collapse
|
57
|
Inverted papilloma and nasal polyp classification using a deep convolutional network integrated with an attention mechanism. Comput Biol Med 2022; 149:105976. [DOI: 10.1016/j.compbiomed.2022.105976] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/07/2022] [Revised: 07/18/2022] [Accepted: 08/13/2022] [Indexed: 11/18/2022]
|
58
|
Tayal A, Gupta J, Solanki A, Bisht K, Nayyar A, Masud M. DL-CNN-based approach with image processing techniques for diagnosis of retinal diseases. MULTIMEDIA SYSTEMS 2022; 28:1417-1438. [DOI: 10.1007/s00530-021-00769-7] [Citation(s) in RCA: 13] [Impact Index Per Article: 6.5] [Reference Citation Analysis] [Track Full Text] [Subscribe] [Scholar Register] [Received: 10/15/2020] [Accepted: 02/24/2021] [Indexed: 08/29/2023]
|
59
|
Ma Z, Xie Q, Xie P, Fan F, Gao X, Zhu J. HCTNet: A Hybrid ConvNet-Transformer Network for Retinal Optical Coherence Tomography Image Classification. BIOSENSORS 2022; 12:542. [PMID: 35884345 PMCID: PMC9313149 DOI: 10.3390/bios12070542] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 06/09/2022] [Revised: 07/13/2022] [Accepted: 07/18/2022] [Indexed: 06/15/2023]
Abstract
Automatic and accurate optical coherence tomography (OCT) image classification is of great significance to computer-assisted diagnosis of retinal disease. In this study, we propose a hybrid ConvNet-Transformer network (HCTNet) and verify the feasibility of a Transformer-based method for retinal OCT image classification. The HCTNet first utilizes a low-level feature extraction module based on the residual dense block to generate low-level features for facilitating the network training. Then, two parallel branches of the Transformer and the ConvNet are designed to exploit the global and local context of the OCT images. Finally, a feature fusion module based on an adaptive re-weighting mechanism is employed to combine the extracted global and local features for predicting the category of OCT images in the testing datasets. The HCTNet combines the advantage of the convolutional neural network in extracting local features and the advantage of the vision Transformer in establishing long-range dependencies. A verification on two public retinal OCT datasets shows that our HCTNet method achieves an overall accuracy of 91.56% and 86.18%, respectively, outperforming the pure ViT and several ConvNet-based classification methods.
Collapse
Affiliation(s)
- Zongqing Ma
- Key Laboratory of the Ministry of Education for Optoelectronic Measurement Technology and Instrument, Beijing Information Science and Technology University, Beijing 100192, China; (Z.M.); (Q.X.); (F.F.)
- Beijing Laboratory of Biomedical Testing Technology and Instruments, Beijing Information Science and Technology University, Beijing 100192, China
| | - Qiaoxue Xie
- Key Laboratory of the Ministry of Education for Optoelectronic Measurement Technology and Instrument, Beijing Information Science and Technology University, Beijing 100192, China; (Z.M.); (Q.X.); (F.F.)
- Beijing Laboratory of Biomedical Testing Technology and Instruments, Beijing Information Science and Technology University, Beijing 100192, China
| | - Pinxue Xie
- Beijing Anzhen Hospital, Capital Medical University, Beijing 100029, China; (P.X.); (X.G.)
| | - Fan Fan
- Key Laboratory of the Ministry of Education for Optoelectronic Measurement Technology and Instrument, Beijing Information Science and Technology University, Beijing 100192, China; (Z.M.); (Q.X.); (F.F.)
- Beijing Laboratory of Biomedical Testing Technology and Instruments, Beijing Information Science and Technology University, Beijing 100192, China
| | - Xinxiao Gao
- Beijing Anzhen Hospital, Capital Medical University, Beijing 100029, China; (P.X.); (X.G.)
| | - Jiang Zhu
- Key Laboratory of the Ministry of Education for Optoelectronic Measurement Technology and Instrument, Beijing Information Science and Technology University, Beijing 100192, China; (Z.M.); (Q.X.); (F.F.)
- Beijing Laboratory of Biomedical Testing Technology and Instruments, Beijing Information Science and Technology University, Beijing 100192, China
| |
Collapse
|
60
|
Esfahani EN, Daneshmand PG, Rabbani H, Plonka G. Automatic Classification of Macular Diseases from OCT Images Using CNN Guided with Edge Convolutional Layer. ANNUAL INTERNATIONAL CONFERENCE OF THE IEEE ENGINEERING IN MEDICINE AND BIOLOGY SOCIETY. IEEE ENGINEERING IN MEDICINE AND BIOLOGY SOCIETY. ANNUAL INTERNATIONAL CONFERENCE 2022; 2022:3858-3861. [PMID: 36085830 DOI: 10.1109/embc48229.2022.9871322] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/15/2023]
Abstract
Optical Coherence Tomography (OCT) is a non-invasive imaging technology that is widely applied for the diagnosis of retinal pathologies. In general, the structural information of retinal layers plays an important role in the diagnosis of various eye diseases by ophthalmologists. In this paper, by focusing on this information, we first introduce a new layer called the edge convolutional layer (ECL) to accurately extract the retinal boundaries in different sizes and angles with a much smaller number of parameters than the conventional convolutional layer. Then, using this layer, we propose the ECL-guided convolutional neural network (ECL-CNN) method for the automatic classification of the OCT images. For the assessment of the proposed method, we utilize a publicly available data comprising 45 OCT volumes with 15 age-related macular degeneration (AMD), 15 diabetic macular edema (DME), and 15 normal volumes, captured by using the Heidelberg OCT imaging device. Experimental results demonstrate that the suggested ECL-CNN approach has an outstanding performance in OCT image classification, which achieves an average precision of 99.43% as a three-class classification work. Clinical Relevance - The objective of this research is to introduce a new approach based on CNN for the automated classification of retinal OCT images. Clinically, the ophthalmologists should manually check each cross-sectional B-scan and classify retinal pathologies from B-scan images. This manual process is tedious and time-consuming in general. Hence, an automatic computer-assisted technique for retinal OCT image classification is demanded.
Collapse
|
61
|
Ai Z, Huang X, Feng J, Wang H, Tao Y, Zeng F, Lu Y. FN-OCT: Disease Detection Algorithm for Retinal Optical Coherence Tomography Based on a Fusion Network. Front Neuroinform 2022; 16:876927. [PMID: 35784186 PMCID: PMC9243322 DOI: 10.3389/fninf.2022.876927] [Citation(s) in RCA: 9] [Impact Index Per Article: 4.5] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/16/2022] [Accepted: 05/04/2022] [Indexed: 01/31/2023] Open
Abstract
Optical coherence tomography (OCT) is a new type of tomography that has experienced rapid development and potential in recent years. It is playing an increasingly important role in retinopathy diagnoses. At present, due to the uneven distributions of medical resources in various regions, the uneven proficiency levels of doctors in grassroots and remote areas, and the development needs of rare disease diagnosis and precision medicine, artificial intelligence technology based on deep learning can provide fast, accurate, and effective solutions for the recognition and diagnosis of retinal OCT images. To prevent vision damage and blindness caused by the delayed discovery of retinopathy, a fusion network (FN)-based retinal OCT classification algorithm (FN-OCT) is proposed in this paper to improve upon the adaptability and accuracy of traditional classification algorithms. The InceptionV3, Inception-ResNet, and Xception deep learning algorithms are used as base classifiers, a convolutional block attention mechanism (CBAM) is added after each base classifier, and three different fusion strategies are used to merge the prediction results of the base classifiers to output the final prediction results (choroidal neovascularization (CNV), diabetic macular oedema (DME), drusen, normal). The results show that in a classification problem involving the UCSD common retinal OCT dataset (108,312 OCT images from 4,686 patients), compared with that of the InceptionV3 network model, the prediction accuracy of FN-OCT is improved by 5.3% (accuracy = 98.7%, area under the curve (AUC) = 99.1%). The predictive accuracy and AUC achieved on an external dataset for the classification of retinal OCT diseases are 92 and 94.5%, respectively, and gradient-weighted class activation mapping (Grad-CAM) is used as a visualization tool to verify the effectiveness of the proposed FNs. This finding indicates that the developed fusion algorithm can significantly improve the performance of classifiers while providing a powerful tool and theoretical support for assisting with the diagnosis of retinal OCT.
Collapse
Affiliation(s)
- Zhuang Ai
- Department of Research and Development, Sinopharm Genomics Technology Co., Ltd., Jiangsu, China
| | - Xuan Huang
- Department of Ophthalmology, Beijing Chao-Yang Hospital, Capital Medical University, Beijing, China
- Medical Research Center, Beijing Chao-Yang Hospital, Capital Medical University, Beijing, China
| | - Jing Feng
- Department of Ophthalmology, Beijing Chao-Yang Hospital, Capital Medical University, Beijing, China
| | - Hui Wang
- Department of Ophthalmology, Beijing Chao-Yang Hospital, Capital Medical University, Beijing, China
| | - Yong Tao
- Department of Ophthalmology, Beijing Chao-Yang Hospital, Capital Medical University, Beijing, China
| | - Fanxin Zeng
- Department of Clinical Research Center, Dazhou Central Hospital, Sichuan, China
| | - Yaping Lu
- Department of Research and Development, Sinopharm Genomics Technology Co., Ltd., Jiangsu, China
| |
Collapse
|
62
|
Wen H, Zhao J, Xiang S, Lin L, Liu C, Wang T, An L, Liang L, Huang B. Towards more efficient ophthalmic disease classification and lesion location via convolution transformer. COMPUTER METHODS AND PROGRAMS IN BIOMEDICINE 2022; 220:106832. [PMID: 35525213 DOI: 10.1016/j.cmpb.2022.106832] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 08/13/2021] [Revised: 04/01/2022] [Accepted: 04/21/2022] [Indexed: 06/14/2023]
Abstract
OBJECTIVE A retina optical coherence tomography (OCT) image differs from a traditional image due to its significant speckle noise, irregularity, and inconspicuous features. A conventional deep learning architecture cannot effectively improve the classification accuracy, sensitivity, and specificity of OCT images, and noisy images are not conducive to further diagnosis. This paper proposes a novel lesion-localization convolution transformer (LLCT) method, which combines both convolution and self-attention to classify ophthalmic diseases more accurately and localize the lesions in retina OCT images. METHODS A novel architecture design is accomplished through applying customized feature maps generated by convolutional neutral network (CNN) as the input sequence of self-attention network. This design takes advantages of CNN's extracting image features and transformer's consideration of global context and dynamic attention. Part of the model is backward propagated to calculate the gradient as a weight parameter, which is multiplied and summed with the global features generated by the forward propagation process to locate the lesion. RESULTS Extensive experiments show that our proposed design achieves improvement of about 7.6% in overall accuracy, 10.9% in overall sensitivity, and 9.2% in overall specificity compared with previous methods. And the lesions can be localized without the labeling data of lesion location in OCT images. CONCLUSION The results prove that our method significantly improves the performance and reduces the computation complexity in artificial intelligence assisted analysis of ophthalmic disease through OCT images. SIGNIFICANCE Our method has a significance boost in ophthalmic disease classification and location via convolution transformer. This is applicable to assist ophthalmologists greatly.1.
Collapse
Affiliation(s)
- Huajie Wen
- College of Big Data and Internet, Shenzhen Technology University, Shenzhen 518118, China; College of Applied Science, Shenzhen University, Shenzhen 518060, China
| | - Jian Zhao
- College of Big Data and Internet, Shenzhen Technology University, Shenzhen 518118, China
| | - Shaohua Xiang
- College of Big Data and Internet, Shenzhen Technology University, Shenzhen 518118, China
| | - Lin Lin
- College of Big Data and Internet, Shenzhen Technology University, Shenzhen 518118, China
| | - Chengjian Liu
- College of Big Data and Internet, Shenzhen Technology University, Shenzhen 518118, China
| | - Tao Wang
- College of Big Data and Internet, Shenzhen Technology University, Shenzhen 518118, China
| | - Lin An
- Guangdong Vision Medical Science & Technology Co., Ltd. Foshan 528000, China
| | - Lixin Liang
- College of Big Data and Internet, Shenzhen Technology University, Shenzhen 518118, China.
| | - Bingding Huang
- College of Big Data and Internet, Shenzhen Technology University, Shenzhen 518118, China.
| |
Collapse
|
63
|
Medical Image Classification Utilizing Ensemble Learning and Levy Flight-Based Honey Badger Algorithm on 6G-Enabled Internet of Things. COMPUTATIONAL INTELLIGENCE AND NEUROSCIENCE 2022; 2022:5830766. [PMID: 35676950 PMCID: PMC9168094 DOI: 10.1155/2022/5830766] [Citation(s) in RCA: 4] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 02/14/2022] [Revised: 03/20/2022] [Accepted: 04/30/2022] [Indexed: 12/23/2022]
Abstract
Recently, the 6G-enabled Internet of Medical Things (IoMT) has played a key role in the development of functional health systems due to the massive data generated daily from the hospitals. Therefore, the automatic detection and prediction of future risks such as pneumonia and retinal diseases are still under research and study. However, traditional approaches did not yield good results for accurate diagnosis. In this paper, a robust 6G-enabled IoMT framework is proposed for medical image classification with an ensemble learning (EL)-based model. EL is achieved using MobileNet and DenseNet architecture as a feature extraction backbone. In addition, the developed framework uses a modified honey badger algorithm (HBA) based on Levy flight (LFHBA) as a feature selection method that aims to remove the irrelevant features from those extracted features using the EL model. For evaluation of the performance of the proposed framework, the chest X-ray (CXR) dataset and the optical coherence tomography (OCT) dataset were employed. The accuracy of our technique was 87.10% on the CXR dataset and 94.32% on OCT dataset—both very good results. Compared to other current methods, the proposed method is more accurate and efficient than other well-known and popular algorithms.
Collapse
|
64
|
Saleh N, Abdel Wahed M, Salaheldin AM. Computer-aided diagnosis system for retinal disorder classification using optical coherence tomography images. BIOMED ENG-BIOMED TE 2022; 67:283-294. [PMID: 35585773 DOI: 10.1515/bmt-2021-0330] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/03/2021] [Accepted: 05/03/2022] [Indexed: 11/15/2022]
Abstract
The incidence of vision impairment is rapidly increasing. Diagnosis and classifying retinal abnormalities in ophthalmological applications is a significant challenge. Using Optical Coherence Tomography (OCT), the study aims to develop a computer aided diagnosis system for detecting and classifying retinal disorders. Choroidal neovascularization, diabetic macular edema, drusen, and normal cases are the investigated groups. Both deep learning and machine learning are combined to build the system. The SqueezeNet neural network was modified to extract features. The Support Vector Machine (SVM), K-Nearest Neighbor (K-NN), Decision Tree (DT), and Ensemble Model (EM) algorithms were used for disorder classification. The Bayesian optimization technique was also used to determine the best hyperparameters for each model. The model' performance was evaluated through nine criteria using 12,000 OCT images. The results have demonstrated accuracies of 97.39, 97.47, 96.98, and 95.25% for the SVM, K-NN, DT, and EM, respectively. When results are compared to relevant studies in terms of accuracy and tested samples, they show superior performance. As a result, a novel computer-aided diagnosis system for detecting and classifying retinal diseases has been developed, reducing human error while also saving time.
Collapse
Affiliation(s)
- Neven Saleh
- Systems and Biomedical Engineering Department, Higher Institute of Engineering in El-Shorouk city, Shorouk Academy, Cairo, Egypt
| | - Manal Abdel Wahed
- Systems and Biomedical Engineering Department, Faculty of Engineering, Cairo University, Giza, Egypt
| | - Ahmed M Salaheldin
- Systems and Biomedical Engineering Department, Higher Institute of Engineering in El-Shorouk city, Shorouk Academy, Cairo, Egypt
| |
Collapse
|
65
|
A Novel -Gram-Based Image Classification Model and Its Applications in Diagnosing Thyroid Nodule and Retinal OCT Images. COMPUTATIONAL AND MATHEMATICAL METHODS IN MEDICINE 2022; 2022:3151554. [PMID: 35547561 PMCID: PMC9085325 DOI: 10.1155/2022/3151554] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 12/06/2021] [Revised: 04/14/2022] [Accepted: 04/16/2022] [Indexed: 11/18/2022]
Abstract
Imbalanced classes and dimensional disasters are critical challenges in medical image classification. As a classical machine learning model, the n-gram model has shown excellent performance in addressing this issue in text classification. In this study, we proposed an algorithm to classify medical images by extracting their n-gram semantic features. This algorithm first converts an image classification problem to a text classification problem by building an n-gram corpus for an image. After that, the algorithm was based on the n-gram model to classify images. The algorithm was evaluated by two independent public datasets. The first experiment is to diagnose benign and malignant thyroid nodules. The best area under the curve (AUC) is 0.989. The second experiment is to diagnose the type of fundus lesion. The best result is that it correctly identified 86.667% of patients with dry age-related macular degeneration (AMD), 93.333% of patients with diabetic macular edema (DME), and 93.333% of normal individuals.
Collapse
|
66
|
Toğaçar M, Ergen B, Tümen V. Use of dominant activations obtained by processing OCT images with the CNNs and slime mold method in retinal disease detection. Biocybern Biomed Eng 2022. [DOI: 10.1016/j.bbe.2022.05.005] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/02/2022]
|
67
|
Sotoudeh-Paima S, Jodeiri A, Hajizadeh F, Soltanian-Zadeh H. Multi-scale convolutional neural network for automated AMD classification using retinal OCT images. Comput Biol Med 2022; 144:105368. [DOI: 10.1016/j.compbiomed.2022.105368] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/07/2021] [Revised: 02/28/2022] [Accepted: 02/28/2022] [Indexed: 11/29/2022]
|
68
|
Diagnostic accuracy of current machine learning classifiers for age-related macular degeneration: a systematic review and meta-analysis. Eye (Lond) 2022; 36:994-1004. [PMID: 33958739 PMCID: PMC9046206 DOI: 10.1038/s41433-021-01540-y] [Citation(s) in RCA: 8] [Impact Index Per Article: 4.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/05/2020] [Revised: 02/23/2021] [Accepted: 04/06/2021] [Indexed: 02/06/2023] Open
Abstract
BACKGROUND AND OBJECTIVE The objective of this study was to systematically review and meta-analyze the diagnostic accuracy of current machine learning classifiers for age-related macular degeneration (AMD). Artificial intelligence diagnostic algorithms can automatically detect and diagnose AMD through training data from large sets of fundus or OCT images. The use of AI algorithms is a powerful tool, and it is a method of obtaining a cost-effective, simple, and fast diagnosis of AMD. METHODS MEDLINE, EMBASE, CINAHL, and ProQuest Dissertations and Theses were searched systematically and thoroughly. Conferences held through Association for Research in Vision and Ophthalmology, American Academy of Ophthalmology, and Canadian Society of Ophthalmology were searched. Studies were screened using Covidence software and data on sensitivity, specificity and area under curve were extracted from the included studies. STATA 15.0 was used to conduct the meta-analysis. RESULTS Our search strategy identified 307 records from online databases and 174 records from gray literature. Total of 13 records, 64,798 subjects (and 612,429 images), were used for the quantitative analysis. The pooled estimate for sensitivity was 0.918 [95% CI: 0.678, 0.98] and specificity was 0.888 [95% CI: 0.578, 0.98] for AMD screening using machine learning classifiers. The relative odds of a positive screen test in AMD cases were 89.74 [95% CI: 3.05-2641.59] times more likely than a negative screen test in non-AMD cases. The positive likelihood ratio was 8.22 [95% CI: 1.52-44.48] and the negative likelihood ratio was 0.09 [95% CI: 0.02-0.52]. CONCLUSION The included studies show promising results for the diagnostic accuracy of the machine learning classifiers for AMD and its implementation in clinical settings.
Collapse
|
69
|
Zhou K, Li J, Luo W, Li Z, Yang J, Fu H, Cheng J, Liu J, Gao S. Proxy-Bridged Image Reconstruction Network for Anomaly Detection in Medical Images. IEEE TRANSACTIONS ON MEDICAL IMAGING 2022; 41:582-594. [PMID: 34644250 DOI: 10.1109/tmi.2021.3118223] [Citation(s) in RCA: 7] [Impact Index Per Article: 3.5] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/13/2023]
Abstract
Anomaly detection in medical images refers to the identification of abnormal images with only normal images in the training set. Most existing methods solve this problem with a self-reconstruction framework, which tends to learn an identity mapping and reduces the sensitivity to anomalies. To mitigate this problem, in this paper, we propose a novel Proxy-bridged Image Reconstruction Network (ProxyAno) for anomaly detection in medical images. Specifically, we use an intermediate proxy to bridge the input image and the reconstructed image. We study different proxy types, and we find that the superpixel-image (SI) is the best one. We set all pixels' intensities within each superpixel as their average intensity, and denote this image as SI. The proposed ProxyAno consists of two modules, a Proxy Extraction Module and an Image Reconstruction Module. In the Proxy Extraction Module, a memory is introduced to memorize the feature correspondence for normal image to its corresponding SI, while the memorized correspondence does not apply to the abnormal images, which leads to the information loss for abnormal image and facilitates the anomaly detection. In the Image Reconstruction Module, we map an SI to its reconstructed image. Further, we crop a patch from the image and paste it on the normal SI to mimic the anomalies, and enforce the network to reconstruct the normal image even with the pseudo abnormal SI. In this way, our network enlarges the reconstruction error for anomalies. Extensive experiments on brain MR images, retinal OCT images and retinal fundus images verify the effectiveness of our method for both image-level and pixel-level anomaly detection.
Collapse
|
70
|
Continual Learning Objective for Analyzing Complex Knowledge Representations. SENSORS 2022; 22:s22041667. [PMID: 35214568 PMCID: PMC8879446 DOI: 10.3390/s22041667] [Citation(s) in RCA: 12] [Impact Index Per Article: 6.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 12/28/2021] [Revised: 02/11/2022] [Accepted: 02/11/2022] [Indexed: 02/04/2023]
Abstract
Human beings tend to incrementally learn from the rapidly changing environment without comprising or forgetting the already learned representations. Although deep learning also has the potential to mimic such human behaviors to some extent, it suffers from catastrophic forgetting due to which its performance on already learned tasks drastically decreases while learning about newer knowledge. Many researchers have proposed promising solutions to eliminate such catastrophic forgetting during the knowledge distillation process. However, to our best knowledge, there is no literature available to date that exploits the complex relationships between these solutions and utilizes them for the effective learning that spans over multiple datasets and even multiple domains. In this paper, we propose a continual learning objective that encompasses mutual distillation loss to understand such complex relationships and allows deep learning models to effectively retain the prior knowledge while adapting to the new classes, new datasets, and even new applications. The proposed objective was rigorously tested on nine publicly available, multi-vendor, and multimodal datasets that span over three applications, and it achieved the top-1 accuracy of 0.9863% and an F1-score of 0.9930.
Collapse
|
71
|
Automatic Detection of Age-Related Macular Degeneration Based on Deep Learning and Local Outlier Factor Algorithm. Diagnostics (Basel) 2022; 12:diagnostics12020532. [PMID: 35204621 PMCID: PMC8871377 DOI: 10.3390/diagnostics12020532] [Citation(s) in RCA: 9] [Impact Index Per Article: 4.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/19/2022] [Revised: 02/13/2022] [Accepted: 02/17/2022] [Indexed: 02/06/2023] Open
Abstract
Age-related macular degeneration (AMD) is a retinal disorder affecting the elderly, and society’s aging population means that the disease is becoming increasingly prevalent. The vision in patients with early AMD is usually unaffected or nearly normal but central vision may be weakened or even lost if timely treatment is not performed. Therefore, early diagnosis is particularly important to prevent the further exacerbation of AMD. This paper proposed a novel automatic detection method of AMD from optical coherence tomography (OCT) images based on deep learning and a local outlier factor (LOF) algorithm. A ResNet-50 model with L2-constrained softmax loss was retrained to extract features from OCT images and the LOF algorithm was used as the classifier. The proposed method was trained on the UCSD dataset and tested on both the UCSD dataset and Duke dataset, with an accuracy of 99.87% and 97.56%, respectively. Even though the model was only trained on the UCSD dataset, it obtained good detection accuracy when tested on another dataset. Comparison with other methods also indicates the efficiency of the proposed method in detecting AMD.
Collapse
|
72
|
Dholakia KY, Guevara-Torres A, Feng G, Power D, Schallek J. In Vivo Capillary Structure and Blood Cell Flux in the Normal and Diabetic Mouse Eye. Invest Ophthalmol Vis Sci 2022; 63:18. [PMID: 35138346 PMCID: PMC8842443 DOI: 10.1167/iovs.63.2.18] [Citation(s) in RCA: 7] [Impact Index Per Article: 3.5] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/31/2023] Open
Abstract
Purpose To characterize the early structural and functional changes in the retinal microvasculature in response to hyperglycemia in the Ins2Akita mouse. Methods A custom phase-contrast adaptive optics scanning light ophthalmoscope was used to image retinal capillaries of 9 Ins2Akita positive (hyperglycemic) and 9 Ins2Akita negative (euglycemic) mice from postnatal weeks 5 to 18. A 15 kHz point scan was used to image capillaries and measure red blood cell flux at biweekly intervals; measurements were performed manually. Retinal thickness and fundus photos were captured monthly using a commercial scanning laser ophthalmoscope/optical coherence tomography. Retinal thickness was calculated using a custom algorithm. Blood glucose and weight were tracked throughout the duration of the study. Results Elevated blood glucose (>250 mg/dL) was observed at 4 to 5 weeks of age in Ins2Akita mice and remained elevated throughout the study, whereas euglycemic littermates maintained normal glucose levels. There was no significant difference in red blood cell flux, capillary anatomy, lumen diameter, or occurrence of stalled capillaries between hyperglycemic and euglycemic mice between postnatal weeks 5 and 18. Hyperglycemic mice had a thinner retina than euglycemic littermates (p < 0.001), but retinal thickness did not change with duration of hyperglycemia despite glucose levels that were more than twice times normal. Conclusions In early stages of hyperglycemia, retinal microvasculature structure (lumen diameter, capillary anatomy) and function (red blood cell flux, capillary perfusion) were not impaired despite 3 months of chronically elevated blood glucose. These findings suggest that hyperglycemia alone for 3 months does not alter capillary structure or function in profoundly hyperglycemic mice.
Collapse
Affiliation(s)
- Kosha Y Dholakia
- Department of Biomedical Engineering, University of Rochester, Rochester, New York, United States.,Center for Visual Science, University of Rochester, Rochester, New York, United States
| | - Andres Guevara-Torres
- Center for Visual Science, University of Rochester, Rochester, New York, United States.,The Institute of Optics, University of Rochester, Rochester, New York, United States
| | - Guanping Feng
- Department of Biomedical Engineering, University of Rochester, Rochester, New York, United States.,Center for Visual Science, University of Rochester, Rochester, New York, United States
| | - Derek Power
- Center for Visual Science, University of Rochester, Rochester, New York, United States.,Flaum Eye Institute, University of Rochester, Rochester, New York, United States
| | - Jesse Schallek
- Center for Visual Science, University of Rochester, Rochester, New York, United States.,Flaum Eye Institute, University of Rochester, Rochester, New York, United States.,Department of Neuroscience, University of Rochester, Rochester, New York, United States
| |
Collapse
|
73
|
Gour N, Tanveer M, Khanna P. Challenges for ocular disease identification in the era of artificial intelligence. Neural Comput Appl 2022. [DOI: 10.1007/s00521-021-06770-5] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/19/2022]
|
74
|
Shao A, Jin K, Li Y, Lou L, Zhou W, Ye J. Overview of global publications on machine learning in diabetic retinopathy from 2011 to 2021: Bibliometric analysis. Front Endocrinol (Lausanne) 2022; 13:1032144. [PMID: 36589855 PMCID: PMC9797582 DOI: 10.3389/fendo.2022.1032144] [Citation(s) in RCA: 4] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 08/30/2022] [Accepted: 12/05/2022] [Indexed: 12/23/2022] Open
Abstract
PURPOSE To comprehensively analyze and discuss the publications on machine learning (ML) in diabetic retinopathy (DR) following a bibliometric approach. METHODS The global publications on ML in DR from 2011 to 2021 were retrieved from the Web of Science Core Collection (WoSCC) database. We analyzed the publication and citation trend over time and identified highly-cited articles, prolific countries, institutions, journals and the most relevant research domains. VOSviewer and Wordcloud are used to visualize the mainstream research topics and evolution of subtopics in the form of co-occurrence maps of keywords. RESULTS By analyzing a total of 1147 relevant publications, this study found a rapid increase in the number of annual publications, with an average growth rate of 42.68%. India and China were the most productive countries. IEEE Access was the most productive journal in this field. In addition, some notable common points were found in the highly-cited articles. The keywords analysis showed that "diabetic retinopathy", "classification", and "fundus images" were the most frequent keywords for the entire period, as automatic diagnosis of DR was always the mainstream topic in the relevant field. The evolution of keywords highlighted some breakthroughs, including "deep learning" and "optical coherence tomography", indicating the advance in technologies and changes in the research attention. CONCLUSIONS As new research topics have emerged and evolved, studies are becoming increasingly diverse and extensive. Multiple modalities of medical data, new ML techniques and constantly optimized algorithms are the future trends in this multidisciplinary field.
Collapse
Affiliation(s)
- An Shao
- Department of Ophthalmology, the Second Affiliated Hospital of Zhejiang University, College of Medicine, Hangzhou, China
| | - Kai Jin
- Department of Ophthalmology, the Second Affiliated Hospital of Zhejiang University, College of Medicine, Hangzhou, China
| | - Yunxiang Li
- College of Computer Science and Technology, Hangzhou Dianzi University, Hangzhou, China
| | - Lixia Lou
- Department of Ophthalmology, the Second Affiliated Hospital of Zhejiang University, College of Medicine, Hangzhou, China
| | - Wuyuan Zhou
- Zhejiang Academy of Science and Technology Information, Hangzhou, China
- *Correspondence: Juan Ye, ; Wuyuan Zhou,
| | - Juan Ye
- Department of Ophthalmology, the Second Affiliated Hospital of Zhejiang University, College of Medicine, Hangzhou, China
- *Correspondence: Juan Ye, ; Wuyuan Zhou,
| |
Collapse
|
75
|
Huang X, Wang H, She C, Feng J, Liu X, Hu X, Chen L, Tao Y. Artificial intelligence promotes the diagnosis and screening of diabetic retinopathy. Front Endocrinol (Lausanne) 2022; 13:946915. [PMID: 36246896 PMCID: PMC9559815 DOI: 10.3389/fendo.2022.946915] [Citation(s) in RCA: 6] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 05/18/2022] [Accepted: 09/12/2022] [Indexed: 11/13/2022] Open
Abstract
Deep learning evolves into a new form of machine learning technology that is classified under artificial intelligence (AI), which has substantial potential for large-scale healthcare screening and may allow the determination of the most appropriate specific treatment for individual patients. Recent developments in diagnostic technologies facilitated studies on retinal conditions and ocular disease in metabolism and endocrinology. Globally, diabetic retinopathy (DR) is regarded as a major cause of vision loss. Deep learning systems are effective and accurate in the detection of DR from digital fundus photographs or optical coherence tomography. Thus, using AI techniques, systems with high accuracy and efficiency can be developed for diagnosing and screening DR at an early stage and without the resources that are only accessible in special clinics. Deep learning enables early diagnosis with high specificity and sensitivity, which makes decisions based on minimally handcrafted features paving the way for personalized DR progression real-time monitoring and in-time ophthalmic or endocrine therapies. This review will discuss cutting-edge AI algorithms, the automated detecting systems of DR stage grading and feature segmentation, the prediction of DR outcomes and therapeutics, and the ophthalmic indications of other systemic diseases revealed by AI.
Collapse
Affiliation(s)
- Xuan Huang
- Department of Ophthalmology, Beijing Chaoyang Hospital, Capital Medical University, Beijing, China
- Medical Research Center, Beijing Chaoyang Hospital, Capital Medical University, Beijing, China
| | - Hui Wang
- Department of Ophthalmology, Beijing Chaoyang Hospital, Capital Medical University, Beijing, China
| | - Chongyang She
- Department of Ophthalmology, Beijing Chaoyang Hospital, Capital Medical University, Beijing, China
| | - Jing Feng
- Department of Ophthalmology, Beijing Chaoyang Hospital, Capital Medical University, Beijing, China
| | - Xuhui Liu
- Department of Ophthalmology, Beijing Chaoyang Hospital, Capital Medical University, Beijing, China
| | - Xiaofeng Hu
- Department of Ophthalmology, Beijing Chaoyang Hospital, Capital Medical University, Beijing, China
| | - Li Chen
- Department of Ophthalmology, Beijing Chaoyang Hospital, Capital Medical University, Beijing, China
| | - Yong Tao
- Department of Ophthalmology, Beijing Chaoyang Hospital, Capital Medical University, Beijing, China
- *Correspondence: Yong Tao,
| |
Collapse
|
76
|
|
77
|
Liu X, Bai Y, Cao J, Yao J, Zhang Y, Wang M. Joint disease classification and lesion segmentation via one-stage attention-based convolutional neural network in OCT images. Biomed Signal Process Control 2022. [DOI: 10.1016/j.bspc.2021.103087] [Citation(s) in RCA: 2] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/02/2022]
|
78
|
Modern trends in diagnostics and prediction of results of anti-vascular endothelial growth factor therapy of pigment epithelial detachment in neovascular agerelated macular degeneration using deep machine learning method (literature review). ACTA BIOMEDICA SCIENTIFICA 2021. [DOI: 10.29413/abs.2021-6.6-1.22] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/21/2022] Open
Abstract
Detachment of the pigment epithelium is the separation of the basement membrane of the retinal pigment epithelium from the inner collagen layer of Bruch’s membrane, which occurs in 80 % of cases in patients with neovascular age-related macular degeneration. The outcome of anti-VEGF therapy for pigment epithelial detachment may be adherence of the pigment epithelium, the formation of pigment epithelium tear, or preservation of the detachment. The pigment epithelium tear of 3–4th degrees can lead to a sharp decrease in visual acuity.Most retrospective studies confi rm the absence of a proven correlation between anatomical and functional outcomes in the treatment of pigment epithelial detachment in cases of maintaining the integrity of the pigment epithelium monolayer, and therefore the main attention of researchers is focused on studying the morphological features of pigment epithelial detachment during therapy with angiogenesis inhibitors. Modern technologies of spectral optical coherence tomography make it possible to evaluate detailed quantitative parameters of pigment epithelium detachment, such as height, width, maximum linear diameter, area, volume and refl ectivity within the detachment.Groups of Russian and foreign authors identify various biomarkers recorded on optical coherence tomography images. Dynamic registration of such biomarkers expands the ability of clinicians to predict morphological changes in pigment epithelial detachment during anti-VEGF therapy, as well as to optimize treatment regimens to prevent complications in the form of pigment epithelium tear leading to a decrease in visual acuity.Modern methods of deep machine learning and the use of neural networks allow achieving higher accuracy in diff erentiating the types of retinal fluids and automating the quantitative determination of fl uid under the pigment epithelium. These technologies allow achieving a high level of compliance with manual expert assessment and increasing the accuracy and speed of predicting morphological results of treatment of pigment epithelium detachments.
Collapse
|
79
|
Xu L, Wang L, Cheng S, Li Y. MHANet: A hybrid attention mechanism for retinal diseases classification. PLoS One 2021; 16:e0261285. [PMID: 34914763 PMCID: PMC8675717 DOI: 10.1371/journal.pone.0261285] [Citation(s) in RCA: 4] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/06/2021] [Accepted: 11/26/2021] [Indexed: 12/04/2022] Open
Abstract
With the increase of patients with retinopathy, retinopathy recognition has become a research hotspot. In this article, we describe the etiology and symptoms of three kinds of retinal diseases, including drusen(DRUSEN), choroidal neovascularization(CNV) and diabetic macular edema(DME). In addition, we also propose a hybrid attention mechanism to classify and recognize different types of retinopathy images. In particular, the hybrid attention mechanism proposed in this paper includes parallel spatial attention mechanism and channel attention mechanism. It can extract the key features in the channel dimension and spatial dimension of retinopathy images, and reduce the negative impact of background information on classification results. The experimental results show that the hybrid attention mechanism proposed in this paper can better assist the network to focus on extracting thr fetures of the retinopathy area and enhance the adaptability to the differences of different data sets. Finally, the hybrid attention mechanism achieved 96.5% and 99.76% classification accuracy on two public OCT data sets of retinopathy, respectively.
Collapse
Affiliation(s)
- Lianghui Xu
- College of Information Science and Engineering, Xinjiang University, Urumqi, China
| | - Liejun Wang
- College of Information Science and Engineering, Xinjiang University, Urumqi, China
- * E-mail:
| | - Shuli Cheng
- College of Information Science and Engineering, Xinjiang University, Urumqi, China
| | - Yongming Li
- College of Information Science and Engineering, Xinjiang University, Urumqi, China
| |
Collapse
|
80
|
Elsharkawy M, Elrazzaz M, Ghazal M, Alhalabi M, Soliman A, Mahmoud A, El-Daydamony E, Atwan A, Thanos A, Sandhu HS, Giridharan G, El-Baz A. Role of Optical Coherence Tomography Imaging in Predicting Progression of Age-Related Macular Disease: A Survey. Diagnostics (Basel) 2021; 11:diagnostics11122313. [PMID: 34943550 PMCID: PMC8699887 DOI: 10.3390/diagnostics11122313] [Citation(s) in RCA: 4] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/05/2021] [Revised: 12/03/2021] [Accepted: 12/07/2021] [Indexed: 11/16/2022] Open
Abstract
In developed countries, age-related macular degeneration (AMD), a retinal disease, is the main cause of vision loss in the elderly. Optical Coherence Tomography (OCT) is currently the gold standard for assessing individuals for initial AMD diagnosis. In this paper, we look at how OCT imaging can be used to diagnose AMD. Our main aim is to examine and compare automated computer-aided diagnostic (CAD) systems for diagnosing and grading of AMD. We provide a brief summary, outlining the main aspects of performance assessment and providing a basis for current research in AMD diagnosis. As a result, the only viable alternative is to prevent AMD and stop both this devastating eye condition and unwanted visual impairment. On the other hand, the grading of AMD is very important in order to detect early AMD and prevent patients from reaching advanced AMD disease. In light of this, we explore the remaining issues with automated systems for AMD detection based on OCT imaging, as well as potential directions for diagnosis and monitoring systems based on OCT imaging and telemedicine applications.
Collapse
Affiliation(s)
- Mohamed Elsharkawy
- Bioengineering Department, University of Louisville, Louisville, KY 40292, USA; (M.E.); (M.E.); (A.S.); (A.M.); (H.S.S.); (G.G.)
| | - Mostafa Elrazzaz
- Bioengineering Department, University of Louisville, Louisville, KY 40292, USA; (M.E.); (M.E.); (A.S.); (A.M.); (H.S.S.); (G.G.)
| | - Mohammed Ghazal
- Electrical and Computer Engineering Department, College of Engineering, Abu Dhabi University, Abu Dhabi 59911, United Arab Emirates; (M.G.); (M.A.)
| | - Marah Alhalabi
- Electrical and Computer Engineering Department, College of Engineering, Abu Dhabi University, Abu Dhabi 59911, United Arab Emirates; (M.G.); (M.A.)
| | - Ahmed Soliman
- Bioengineering Department, University of Louisville, Louisville, KY 40292, USA; (M.E.); (M.E.); (A.S.); (A.M.); (H.S.S.); (G.G.)
| | - Ali Mahmoud
- Bioengineering Department, University of Louisville, Louisville, KY 40292, USA; (M.E.); (M.E.); (A.S.); (A.M.); (H.S.S.); (G.G.)
| | - Eman El-Daydamony
- Faculty of Computers and Information, Mansoura University, Mansoura 35516, Egypt; (E.E.-D.); (A.A.)
| | - Ahmed Atwan
- Faculty of Computers and Information, Mansoura University, Mansoura 35516, Egypt; (E.E.-D.); (A.A.)
| | | | - Harpal Singh Sandhu
- Bioengineering Department, University of Louisville, Louisville, KY 40292, USA; (M.E.); (M.E.); (A.S.); (A.M.); (H.S.S.); (G.G.)
| | - Guruprasad Giridharan
- Bioengineering Department, University of Louisville, Louisville, KY 40292, USA; (M.E.); (M.E.); (A.S.); (A.M.); (H.S.S.); (G.G.)
| | - Ayman El-Baz
- Bioengineering Department, University of Louisville, Louisville, KY 40292, USA; (M.E.); (M.E.); (A.S.); (A.M.); (H.S.S.); (G.G.)
- Correspondence:
| |
Collapse
|
81
|
Correlation of Volume of Macular Edema with Retinal Tomography Features in Diabetic Retinopathy Eyes. J Pers Med 2021; 11:jpm11121337. [PMID: 34945810 PMCID: PMC8708057 DOI: 10.3390/jpm11121337] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/18/2021] [Revised: 11/25/2021] [Accepted: 12/01/2021] [Indexed: 11/17/2022] Open
Abstract
Optical coherence tomography (OCT) enables the detection of macular edema, a significant pathological outcome of diabetic retinopathy (DR). The aim of the study was to correlate edema volume with the severity of diabetic retinopathy and response to treatment with intravitreal injections (compared to baseline). Diabetic retinopathy (DR; n = 181) eyes were imaged with OCT (Heidelberg Engineering, Germany). They were grouped as responders (a decrease in thickness after intravitreal injection of Bevacizumab), non-responders (persistent edema or reduced decrease in thickness), recurrent (recurrence of edema after injection), and treatment naïve (no change in edema at follow-up without any injection). The post-treatment imaging of eyes was included for all groups, except for the treatment naïve group. All eyes underwent a 9 × 6 mm raster scan to measure the edema volume (EV). Central foveal thickness (CFT), central foveal volume (CFV), and total retinal volume (TRV) were obtained from the early treatment diabetic retinopathy study (ETDRS) map. The median EV increased with DR severity, with PDR having the greatest EV (4.01 mm3). This correlated positively with TRV (p < 0.001). Median CFV and CFT were the greatest in severe NPDR. Median EV was the greatest in the recurrent eyes (4.675 mm3) and lowest (1.6 mm3) in the treatment naïve group. Responders and non-responders groups had median values of 3.65 and 3.93 mm3, respectively. This trend was not observed with CFV, CFT, and TRV. A linear regression yielded threshold values of CFV (~0.3 mm3), CFT (~386 µm), and TRV (~9.06 mm3), above which EV may be detected by the current scanner. In this study, EV provided a better distinction between the response groups when compared to retinal tomography parameters. The EV increased with disease severity. Thus, EV can be a more precise parameter to identify subclinical edema and aid in better treatment planning.
Collapse
|
82
|
Barua PD, Chan WY, Dogan S, Baygin M, Tuncer T, Ciaccio EJ, Islam N, Cheong KH, Shahid ZS, Acharya UR. Multilevel Deep Feature Generation Framework for Automated Detection of Retinal Abnormalities Using OCT Images. ENTROPY (BASEL, SWITZERLAND) 2021; 23:1651. [PMID: 34945957 PMCID: PMC8700736 DOI: 10.3390/e23121651] [Citation(s) in RCA: 10] [Impact Index Per Article: 3.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 10/11/2021] [Revised: 11/22/2021] [Accepted: 11/25/2021] [Indexed: 01/04/2023]
Abstract
Optical coherence tomography (OCT) images coupled with many learning techniques have been developed to diagnose retinal disorders. This work aims to develop a novel framework for extracting deep features from 18 pre-trained convolutional neural networks (CNN) and to attain high performance using OCT images. In this work, we have developed a new framework for automated detection of retinal disorders using transfer learning. This model consists of three phases: deep fused and multilevel feature extraction, using 18 pre-trained networks and tent maximal pooling, feature selection with ReliefF, and classification using the optimized classifier. The novelty of this proposed framework is the feature generation using widely used CNNs and to select the most suitable features for classification. The extracted features using our proposed intelligent feature extractor are fed to iterative ReliefF (IRF) to automatically select the best feature vector. The quadratic support vector machine (QSVM) is utilized as a classifier in this work. We have developed our model using two public OCT image datasets, and they are named database 1 (DB1) and database 2 (DB2). The proposed framework can attain 97.40% and 100% classification accuracies using the two OCT datasets, DB1 and DB2, respectively. These results illustrate the success of our model.
Collapse
Affiliation(s)
- Prabal Datta Barua
- School of Management & Enterprise, University of Southern Queensland, Toowoomba, QLD 4350, Australia;
- Faculty of Engineering and Information Technology, University of Technology Sydney, Sydney, NSW 2007, Australia
- Cogninet Brain Team, Cogninet Australia, Sydney, NSW 2010, Australia
| | - Wai Yee Chan
- University Malaya Research Imaging Centre, Department of Biomedical Imaging, Faculty of Medicine, University of Malaya, Kuala Lumpur 59100, Malaysia;
| | - Sengul Dogan
- Department of Digital Forensics Engineering, College of Technology, Firat University, Elazig 23002, Turkey; (S.D.); (T.T.)
| | - Mehmet Baygin
- Department of Computer Engineering, College of Engineering, Ardahan University, Ardahan 75000, Turkey;
| | - Turker Tuncer
- Department of Digital Forensics Engineering, College of Technology, Firat University, Elazig 23002, Turkey; (S.D.); (T.T.)
| | - Edward J. Ciaccio
- Department of Medicine, Columbia University Irving Medical Center, New York, NY 10032-3784, USA;
| | - Nazrul Islam
- Glaucoma Faculty, Bangladesh Eye Hospital & Institute, Dhaka 1206, Bangladesh;
| | - Kang Hao Cheong
- Science, Mathematics and Technology Cluster, Singapore University of Technology and Design, Singapore 487372, Singapore
| | - Zakia Sultana Shahid
- Department of Ophthalmology, Anwer Khan Modern Medical College, Dhaka 1205, Bangladesh;
| | - U. Rajendra Acharya
- Department of Electronics and Computer Engineering, Ngee Ann Polytechnic, Singapore 599489, Singapore
- Department of Biomedical Engineering, School of Science and Technology, SUSS University, Singapore 129799, Singapore
- Department of Biomedical Informatics and Medical Engineering, Asia University, Taichung 41354, Taiwan
| |
Collapse
|
83
|
Mezni I, Ben Slama A, Mbarki Z, Seddik H, Trabelsi H. Automated identification of SD-optical coherence tomography derived macular diseases by combining 3D-block-matching and deep learning techniques. COMPUTER METHODS IN BIOMECHANICS AND BIOMEDICAL ENGINEERING: IMAGING & VISUALIZATION 2021. [DOI: 10.1080/21681163.2021.1926329] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 10/21/2022]
Affiliation(s)
- Ilhem Mezni
- ISTMT, Laboratory of Biophysics and Medical Technologies (LRBTM), LR13ES07, University of Tunis El Manar, Tunis, Tunisia
| | - Amine Ben Slama
- ISTMT, Laboratory of Biophysics and Medical Technologies (LRBTM), LR13ES07, University of Tunis El Manar, Tunis, Tunisia
| | | | | | - Hedi Trabelsi
- ISTMT, Laboratory of Biophysics and Medical Technologies (LRBTM), LR13ES07, University of Tunis El Manar, Tunis, Tunisia
| |
Collapse
|
84
|
Wang H, Won D, Yoon SW. An adaptive neural architecture optimization model for retinal disorder diagnosis on 3D medical images. Appl Soft Comput 2021. [DOI: 10.1016/j.asoc.2021.107686] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/20/2022]
|
85
|
Chakroborty S, Gupta M, Devishamani CS, Patel K, Ankit C, Ganesh Babu TC, Raman R. Narrative review of artificial intelligence in diabetic macular edema: Diagnosis and predicting treatment response using optical coherence tomography. Indian J Ophthalmol 2021; 69:2999-3008. [PMID: 34708735 PMCID: PMC8725112 DOI: 10.4103/ijo.ijo_1482_21] [Citation(s) in RCA: 6] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/04/2022] Open
Abstract
Diabetic macular edema (DME), being a frequent manifestation of DR, disrupts the retinal symmetry. This event is particularly triggered by vascular endothelial growth factors (VEGF). Intravitreal injections of anti-VEGFs have been the most practiced treatment but an expensive option. A major challenge associated with this treatment is determining an optimal treatment regimen and differentiating patients who do not respond to anti-VEGF. As it has a significant burden for both the patient and the health care providers if the patient is not responding, any clinically acceptable method to predict the treatment outcomes holds huge value in the efficient management of DME. In such situations, artificial intelligence (AI) or machine learning (ML)-based algorithms come useful as they can analyze past clinical details of the patients and help clinicians to predict the patient's response to an anti-VEGF agent. The work presented here attempts to review the literature that is available from the peer research community to discuss solutions provided by AI/ML methodologies to tackle challenges in DME management. Lastly, a possibility for using two different types of data has been proposed, which is believed to be the key differentiators as compared to the similar and recent contributions from the peer research community.
Collapse
Affiliation(s)
- Sandipan Chakroborty
- Center for Applications and Research in India, Carl Zeiss India (Bangalore) Pvt. Ltd., Bengaluru, Karnataka, India
| | - Mansi Gupta
- Center for Applications and Research in India, Carl Zeiss India (Bangalore) Pvt. Ltd., Bengaluru, Karnataka, India
| | | | - Krunalkumar Patel
- Center for Applications and Research in India, Carl Zeiss India (Bangalore) Pvt. Ltd., Bengaluru, Karnataka, India
| | - Chavan Ankit
- Center for Applications and Research in India, Carl Zeiss India (Bangalore) Pvt. Ltd., Bengaluru, Karnataka, India
| | - T C Ganesh Babu
- Center for Applications and Research in India, Carl Zeiss India (Bangalore) Pvt. Ltd., Bengaluru, Karnataka, India
| | - Rajiv Raman
- Shri Bhagwan Mahavir Vitreoretinal Services, Sankara Nethralaya, Chennai, Tamil Nadu, India
| |
Collapse
|
86
|
Zhang Y, Li M, Ji Z, Fan W, Yuan S, Liu Q, Chen Q. Twin self-supervision based semi-supervised learning (TS-SSL): Retinal anomaly classification in SD-OCT images. Neurocomputing 2021. [DOI: 10.1016/j.neucom.2021.08.051] [Citation(s) in RCA: 5] [Impact Index Per Article: 1.7] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/29/2023]
|
87
|
Duwairi RM, Al-Zboon SA, Al-Dwairi RA, Obaidi A. A Deep Learning Model and a Dataset for Diagnosing Ophthalmology Diseases. JOURNAL OF INFORMATION & KNOWLEDGE MANAGEMENT 2021. [DOI: 10.1142/s0219649221500362] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 11/18/2022]
Abstract
The rapid development of artificial neural network techniques, especially convolutional neural networks, encouraged the researchers to adapt such techniques in the medical domain. Specifically, to provide assist tools to help the professionals in patients’ diagnosis. The main problem faced by the researchers in the medical domain is the lack of available annotated datasets which can be used to train and evaluate large and complex deep neural networks. In this paper, to assist researchers who are interested in applying deep learning techniques to aid the ophthalmologists in diagnosing eye-related diseases, we provide an optical coherence tomography dataset with collaboration with ophthalmologists from the King Abdullah University Hospital, Irbid, Jordan. This dataset consists of 21,991 OCT images distributed over seven eye diseases in addition to normal images (no disease), namely, Choroidal Neovascularisation, Full Macular Hole (Full Thickness), Partial Macular Hole, Central Serous Retinopathy, Geographic atrophy, Macular Retinal Oedema, and Vitreomacular Traction. To the best of our knowledge, this dataset is the largest of its kind, where images belong to actual patients from Jordan and the annotation was carried out by ophthalmologists. Two classification tasks were applied to this dataset; a binary classification to distinguish between images which belong to healthy eyes (normal) and images which belong to diseased eyes (abnormal). The second classification task is a multi-class classification, where the deep neural network is trained to distinguish between the seven diseases listed above in addition to the normal case. In both classification tasks, the U-Net neural network was modified and subsequently utilised. This modification adds an additional block of layers to the original U-Net model to become capable of handling classification as the original network is used for image segmentation. The results of the binary classification were equal to 84.90% and 69.50% as accuracy and quadratic weighted kappa, respectively. The results of the multi-class classification, by contrast, were equal to 63.68% and 66.06% as accuracy and quadratic weighted kappa, respectively.
Collapse
Affiliation(s)
- Rehab M. Duwairi
- Department of Computer Information Systems, Jordan University of Science and Technology, Irbid, Jordan
| | - Saad A. Al-Zboon
- Department of Computer Science, Jordan University of Science and Technology, Irbid, Jordan
| | - Rami A. Al-Dwairi
- Division of Ophthalmology, Department of Special Surgery, Faculty of Medicine, Jordan University of Science and Technology, Irbid, Jordan
| | - Ahmad Obaidi
- King Abdullah University Hospital, Irbid, Jordan
| |
Collapse
|
88
|
Thomas A, Harikrishnan PM, Ramachandran R, Ramachandran S, Manoj R, Palanisamy P, Gopi VP. A novel multiscale and multipath convolutional neural network based age-related macular degeneration detection using OCT images. COMPUTER METHODS AND PROGRAMS IN BIOMEDICINE 2021; 209:106294. [PMID: 34364184 DOI: 10.1016/j.cmpb.2021.106294] [Citation(s) in RCA: 9] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 05/23/2020] [Accepted: 07/15/2021] [Indexed: 06/13/2023]
Abstract
BACKGROUND AND OBJECTIVE One of the significant retinal diseases that affected older people is called Age-related Macular Degeneration (AMD). The first stage creates a blur effect on vision and later leads to central vision loss. Most people overlooked the primary stage blurring and converted it into an advanced stage. There is no proper treatment to cure the disease. So the early detection of AMD is essential to prevent its extension into the advanced stage. This paper proposes a novel deep Convolutional Neural Network (CNN) architecture to automate AMD diagnosis early from Optical Coherence Tomographic (OCT) images. METHODS The proposed architecture is a multiscale and multipath CNN with six convolutional layers. The multiscale convolution layer permits the network to produce many local structures with various filter dimensions. The multipath feature extraction permits CNN to merge more features regarding the sparse local and fine global structures. The performance of the proposed architecture is evaluated through ten-fold cross-validation methods using different classifiers like support vector machine, multi-layer perceptron, and random forest. RESULTS The proposed CNN with the random forest classifier gives the best classification accuracy results. The proposed method is tested on data set 1, data set 2, data set 3, data set 4, and achieved an accuracy of 0.9666, 0.9897, 0.9974, and 0.9978 respectively, with random forest classifier. Also, we tested the combination of first three data sets and achieved an accuracy of 0.9902. CONCLUSIONS An efficient algorithm for detecting AMD from OCT images is proposed based on a multiscale and multipath CNN architecture. Comparison with other approaches produced results that exhibit the efficiency of the proposed algorithm in the detection of AMD. The proposed architecture can be applied in rapid screening of the eye for the early detection of AMD. Due to less complexity and fewer learnable parameters.
Collapse
Affiliation(s)
- Anju Thomas
- Department of Electronics and Communication Engineering, National Institute of Technology Tiruchirappalli, Tamilnadu 620015, India.
| | - P M Harikrishnan
- Department of Electronics and Communication Engineering, National Institute of Technology Tiruchirappalli, Tamilnadu 620015, India.
| | - Rajiv Ramachandran
- Department of Electronics and Communication Engineering, National Institute of Technology Tiruchirappalli, Tamilnadu 620015, India.
| | - Srikkanth Ramachandran
- Department of Electronics and Communication Engineering, National Institute of Technology Tiruchirappalli, Tamilnadu 620015, India.
| | - Rigved Manoj
- Department of Electronics and Communication Engineering, National Institute of Technology Tiruchirappalli, Tamilnadu 620015, India.
| | - P Palanisamy
- Department of Electronics and Communication Engineering, National Institute of Technology Tiruchirappalli, Tamilnadu 620015, India.
| | - Varun P Gopi
- Department of Electronics and Communication Engineering, National Institute of Technology Tiruchirappalli, Tamilnadu 620015, India.
| |
Collapse
|
89
|
CDC-Net: Cascaded decoupled convolutional network for lesion-assisted detection and grading of retinopathy using optical coherence tomography (OCT) scans. Biomed Signal Process Control 2021. [DOI: 10.1016/j.bspc.2021.103030] [Citation(s) in RCA: 9] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/12/2023]
|
90
|
Lakshminarayanan V, Kheradfallah H, Sarkar A, Jothi Balaji J. Automated Detection and Diagnosis of Diabetic Retinopathy: A Comprehensive Survey. J Imaging 2021; 7:165. [PMID: 34460801 PMCID: PMC8468161 DOI: 10.3390/jimaging7090165] [Citation(s) in RCA: 22] [Impact Index Per Article: 7.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/30/2021] [Revised: 08/23/2021] [Accepted: 08/24/2021] [Indexed: 12/16/2022] Open
Abstract
Diabetic Retinopathy (DR) is a leading cause of vision loss in the world. In the past few years, artificial intelligence (AI) based approaches have been used to detect and grade DR. Early detection enables appropriate treatment and thus prevents vision loss. For this purpose, both fundus and optical coherence tomography (OCT) images are used to image the retina. Next, Deep-learning (DL)-/machine-learning (ML)-based approaches make it possible to extract features from the images and to detect the presence of DR, grade its severity and segment associated lesions. This review covers the literature dealing with AI approaches to DR such as ML and DL in classification and segmentation that have been published in the open literature within six years (2016-2021). In addition, a comprehensive list of available DR datasets is reported. This list was constructed using both the PICO (P-Patient, I-Intervention, C-Control, O-Outcome) and Preferred Reporting Items for Systematic Review and Meta-analysis (PRISMA) 2009 search strategies. We summarize a total of 114 published articles which conformed to the scope of the review. In addition, a list of 43 major datasets is presented.
Collapse
Affiliation(s)
- Vasudevan Lakshminarayanan
- Theoretical and Experimental Epistemology Lab, School of Optometry and Vision Science, University of Waterloo, Waterloo, ON N2L 3G1, Canada;
| | - Hoda Kheradfallah
- Theoretical and Experimental Epistemology Lab, School of Optometry and Vision Science, University of Waterloo, Waterloo, ON N2L 3G1, Canada;
| | - Arya Sarkar
- Department of Computer Engineering, University of Engineering and Management, Kolkata 700 156, India;
| | | |
Collapse
|
91
|
Region of interest-based predictive algorithm for subretinal hemorrhage detection using faster R-CNN. Soft comput 2021; 25:15255-15268. [PMID: 34421341 PMCID: PMC8371433 DOI: 10.1007/s00500-021-06098-1] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Accepted: 07/30/2021] [Indexed: 11/04/2022]
Abstract
Macular edema (ME) is an essential sort of macular issue caused due to the storing of fluid underneath the macula. Age-related Macular Degeneration (AMD) and diabetic macular edema (DME) are the two customary visual contaminations that can lead to fragmentary or complete vision loss. This paper proposes a deep learning-based predictive algorithm that can be used to detect the presence of a Subretinal hemorrhage. Region Convolutional Neural Network (R-CNN) and faster R-CNN are used to develop the predictive algorithm that can improve the classification accuracy. This method initially detects the presence of Subretinal hemorrhage, and it then segments the Region of Interest (ROI) by a semantic segmentation process. The segmented ROI is applied to a predictive algorithm which is derived from the Fast Region Convolutional Neural Network algorithm, that can categorize the Subretinal hemorrhage as responsive or non-responsive. The dataset, provided by a medical institution, comprised of optical coherence tomography (OCT) images of both pre- and post-treatment images, was used for training the proposed Faster Region Convolutional Neural Network (Faster R-CNN). We also used the Kaggle dataset for performance comparison with the traditional methods that are derived from the convolutional neural network (CNN) algorithm. The evaluation results using the Kaggle dataset and the hospital images provide an average sensitivity, selectivity, and accuracy of 85.3%, 89.64%, and 93.48% respectively. Further, the proposed method provides a time complexity in testing as 2.64s, which is less than the traditional schemes like CNN, R-CNN, and Fast R-CNN.
Collapse
|
92
|
Luo Y, Xu Q, Hou Y, Liu L, Wu M. Cross-domain retinopathy classification with optical coherence tomography images via a novel deep domain adaptation method. JOURNAL OF BIOPHOTONICS 2021; 14:e202100096. [PMID: 33934549 DOI: 10.1002/jbio.202100096] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 03/22/2021] [Revised: 04/22/2021] [Accepted: 04/27/2021] [Indexed: 06/12/2023]
Abstract
Deep learning based retinopathy classification with optical coherence tomography (OCT) images has recently attracted great attention. However, existing deep learning methods fail to work well when training and testing datasets are different due to the general issue of domain shift between datasets caused by different collection devices, subjects, imaging parameters, etc. To address this practical and challenging issue, we propose a novel deep domain adaptation (DDA) method to train a model on a labeled dataset and adapt it to an unlabelled dataset (collected under different conditions). It consists of two modules for domain alignment, that is, adversarial learning and entropy minimization. We conduct extensive experiments on three public datasets to evaluate the performance of the proposed method. The results indicate that there are large domain shifts between datasets, resulting a poor performance for conventional deep learning methods. The proposed DDA method can significantly outperform existing methods for retinopathy classification with OCT images. It achieves retinopathy classification accuracies of 0.915, 0.959 and 0.990 under three cross-domain (cross-dataset) scenarios. Moreover, it obtains a comparable performance with human experts on a dataset where no labeled data in this dataset have been used to train the proposed DDA method. We have also visualized the learnt features by using the t-distributed stochastic neighbor embedding (t-SNE) technique. The results demonstrate that the proposed method can learn discriminative features for retinopathy classification.
Collapse
Affiliation(s)
- Yuemei Luo
- School of Electrical and Electronic Engineering, Nanyang Technological University, Singapore, Singapore
| | - Qing Xu
- Institute for Infocomm Research, Agency for Science, Technology and Research (A*STAR), Singapore, Singapore
| | - Yubo Hou
- Institute for Infocomm Research, Agency for Science, Technology and Research (A*STAR), Singapore, Singapore
| | - Linbo Liu
- School of Electrical and Electronic Engineering, Nanyang Technological University, Singapore, Singapore
- School of Chemical and Biomedical Engineering, Nanyang Technological University, Singapore, Singapore
| | - Min Wu
- Institute for Infocomm Research, Agency for Science, Technology and Research (A*STAR), Singapore, Singapore
| |
Collapse
|
93
|
Luo Y, Xu Q, Jin R, Wu M, Liu L. Automatic detection of retinopathy with optical coherence tomography images via a semi-supervised deep learning method. BIOMEDICAL OPTICS EXPRESS 2021; 12:2684-2702. [PMID: 34123497 PMCID: PMC8176801 DOI: 10.1364/boe.418364] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Received: 12/22/2020] [Revised: 03/27/2021] [Accepted: 04/02/2021] [Indexed: 05/03/2023]
Abstract
Automatic detection of retinopathy via computer vision techniques is of great importance for clinical applications. However, traditional deep learning based methods in computer vision require a large amount of labeled data, which are expensive and may not be available in clinical applications. To mitigate this issue, in this paper, we propose a semi-supervised deep learning method built upon pre-trained VGG-16 and virtual adversarial training (VAT) for the detection of retinopathy with optical coherence tomography (OCT) images. It only requires very few labeled and a number of unlabeled OCT images for model training. In experiments, we have evaluated the proposed method on two popular datasets. With only 80 labeled OCT images, the proposed method can achieve classification accuracies of 0.942 and 0.936, sensitivities of 0.942 and 0.936, specificities of 0.971 and 0.979, and AUCs (Area under the ROC Curves) of 0.997 and 0.993 on the two datasets, respectively. When comparing with human experts, it achieves expert level with 80 labeled OCT images and outperforms four out of six experts with 200 labeled OCT images. Furthermore, we also adopt the Gradient Class Activation Map (Grad-CAM) method to visualize the key regions that the proposed method focuses on when making predictions. It shows that the proposed method can accurately recognize the key patterns of the input OCT images when predicting retinopathy.
Collapse
Affiliation(s)
- Yuemei Luo
- School of Electrical and Electronic
Engineering, Nanyang Technological
University, Singapore, 639798, Singapore
| | - Qing Xu
- Institute for Infocomm
Research, Agency for Science, Technology and Research
(A*STAR), Singapore, 138632, Singapore
| | - Ruibing Jin
- Institute for Infocomm
Research, Agency for Science, Technology and Research
(A*STAR), Singapore, 138632, Singapore
| | - Min Wu
- Institute for Infocomm
Research, Agency for Science, Technology and Research
(A*STAR), Singapore, 138632, Singapore
| | - Linbo Liu
- School of Electrical and Electronic
Engineering, Nanyang Technological
University, Singapore, 639798, Singapore
- School of Chemical and Biomedical
Engineering, Nanyang Technological
University, Singapore, 637459, Singapore
| |
Collapse
|
94
|
Thomas A, P. M. H, K. Krishna A, P. P, Gopi VP. A novel multiscale convolutional neural network based age-related macular degeneration detection using OCT images. Biomed Signal Process Control 2021. [DOI: 10.1016/j.bspc.2021.102538] [Citation(s) in RCA: 5] [Impact Index Per Article: 1.7] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 02/06/2023]
|
95
|
Anvari P, Ashrafkhorasani M, Habibi A, Falavarjani KG. Artifacts in Optical Coherence Tomography Angiography. J Ophthalmic Vis Res 2021; 16:271-286. [PMID: 34055264 PMCID: PMC8126744 DOI: 10.18502/jovr.v16i2.9091] [Citation(s) in RCA: 9] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/25/2020] [Accepted: 01/05/2021] [Indexed: 01/10/2023] Open
Abstract
We performed a comprehensive search of the published literature in PubMed and Google Scholar to identify types, prevalence, etiology, clinical impact, and current methods for correction of various artifacts in optical coherence tomography angiography (OCTA) images. We found that the prevalence of OCTA image artifacts is fairly high. Artifacts associated with eye motion, misidentification of retinal layers, projections, and low optical coherence tomography signal are the most prevalent types. Artifacts in OCTA images are the major limitations of this diagnostic modality in clinical practice and identification of these artifacts and measures to mitigate them are essential for correct diagnosis and follow-up of patients.
Collapse
Affiliation(s)
- Pasha Anvari
- Eye Research Center, The Five Senses Institute, Rassoul Akram Hospital, Iran University of Medical Sciences, Tehran, Iran
| | - Maryam Ashrafkhorasani
- Eye Research Center, The Five Senses Institute, Rassoul Akram Hospital, Iran University of Medical Sciences, Tehran, Iran
| | - Abbas Habibi
- Eye Research Center, The Five Senses Institute, Rassoul Akram Hospital, Iran University of Medical Sciences, Tehran, Iran
| | - Khalil Ghasemi Falavarjani
- Eye Research Center, The Five Senses Institute, Rassoul Akram Hospital, Iran University of Medical Sciences, Tehran, Iran.,Stem Cell and Regenerative Medicine Research Center, Iran University of Medical Sciences, Tehran, Iran
| |
Collapse
|
96
|
Assessing Changes in Diabetic Retinopathy Caused by Diabetes Mellitus and Glaucoma Using Support Vector Machines in Combination with Differential Evolution Algorithm. APPLIED SCIENCES-BASEL 2021. [DOI: 10.3390/app11093944] [Citation(s) in RCA: 9] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 11/16/2022]
Abstract
The aim of this study is to evaluate the changes related to diabetic retinopathy (DR) (no changes, small or moderate changes) in patients with glaucoma and diabetes using artificial intelligence instruments: Support Vector Machines (SVM) in combination with a powerful optimization algorithm—Differential Evolution (DE). In order to classify the DR changes and to make predictions in various situations, an approach including SVM optimized with DE was applied. The role of the optimizer was to automatically determine the SVM parameters that lead to the lowest classification error. The study was conducted on a sample of 52 patients: particularly, 101 eyes with glaucoma and diabetes mellitus, in the Ophthalmology Clinic I of the “St. Spiridon” Clinical Hospital of Iaşi. The criteria considered in the modelling action were normal or hypertensive open-angle glaucoma, intraocular hypertension and associated diabetes. The patients with other types of glaucoma pseudoexfoliation, pigment, cortisone, neovascular and primitive angle-closure, and those without associated diabetes, were excluded. The assessment of diabetic retinopathy changes were carried out with Volk lens and Fundus Camera Zeiss retinal photography on the dilated pupil, inspecting all quadrants. The criteria for classifying the DR (early treatment diabetic retinopathy study—ETDRS) changes were: without changes (absence of DR), mild forma nonproliferative diabetic retinopathy (the presence of a single micro aneurysm), moderate form (micro aneurysms, hemorrhages in 2–3 quadrants, venous dilatations and soft exudates in a quadrant), severe form (micro aneurysms, hemorrhages in all quadrants, venous dilatation in 2–3 quadrants) and proliferative diabetic retinopathy (disk and retinal neovascularization in different quadrants). Any new clinical element that occurred in subsequent checks, which led to their inclusion in severe nonproliferative or proliferative forms of diabetic retinopathy, was considered to be the result of the progression of diabetic retinopathy. The results obtained were very good; in the testing phase, a 95.23% accuracy has been obtained, only one sample being wrongly classified. The effectiveness of the classification algorithm (SVM), developed in optimal form with DE, and used in predictions of retinal changes related to diabetes, was demonstrated.
Collapse
|
97
|
Chen M, Jin K, You K, Xu Y, Wang Y, Yip CC, Wu J, Ye J. Automatic detection of leakage point in central serous chorioretinopathy of fundus fluorescein angiography based on time sequence deep learning. Graefes Arch Clin Exp Ophthalmol 2021; 259:2401-2411. [PMID: 33846835 DOI: 10.1007/s00417-021-05151-x] [Citation(s) in RCA: 7] [Impact Index Per Article: 2.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/19/2020] [Revised: 02/16/2021] [Accepted: 03/02/2021] [Indexed: 01/23/2023] Open
Abstract
PURPOSE To detect the leakage points of central serous chorioretinopathy (CSC) automatically from dynamic images of fundus fluorescein angiography (FFA) using a deep learning algorithm (DLA). METHODS The study included 2104 FFA images from 291 FFA sequences of 291 eyes (137 right eyes and 154 left eyes) from 262 patients. The leakage points were segmented with an attention gated network (AGN). The optic disk (OD) and macula region were segmented simultaneously using a U-net. To reduce the number of false positives based on time sequence, the leakage points were matched according to their positions in relation to the OD and macula. RESULTS With the AGN alone, the number of cases whose detection results perfectly matched the ground truth was only 37 out of 61 cases (60.7%) in the test set. The dice on the lesion level were 0.811. Using an elimination procedure to remove false positives, the number of accurate detection cases increased to 57 (93.4%). The dice on the lesion level also improved to 0.949. CONCLUSIONS Using DLA, the CSC leakage points in FFA can be identified reproducibly and accurately with a good match to the ground truth. This novel finding may pave the way for potential application of artificial intelligence to guide laser therapy.
Collapse
Affiliation(s)
- Menglu Chen
- Department of Ophthalmology, the Second Affiliated Hospital of Zhejiang University, College of Medicine, Hangzhou, 310009, China
| | - Kai Jin
- Department of Ophthalmology, the Second Affiliated Hospital of Zhejiang University, College of Medicine, Hangzhou, 310009, China
| | - Kun You
- Hangzhou Truth Medical Technology Ltd, Hangzhou, 311215, China
| | - Yufeng Xu
- Department of Ophthalmology, the Second Affiliated Hospital of Zhejiang University, College of Medicine, Hangzhou, 310009, China
| | - Yao Wang
- Department of Ophthalmology, the Second Affiliated Hospital of Zhejiang University, College of Medicine, Hangzhou, 310009, China
| | - Chee-Chew Yip
- Department of Ophthalmology, Khoo Teck Puat Hospital, Yishun Central, Singapore
| | - Jian Wu
- College of Computer Science and Technology, Zhejiang University, Hangzhou, 310027, China.
| | - Juan Ye
- Department of Ophthalmology, the Second Affiliated Hospital of Zhejiang University, College of Medicine, Hangzhou, 310009, China.
| |
Collapse
|
98
|
Perepelkina T, Fulton AB. Artificial Intelligence (AI) Applications for Age-Related Macular Degeneration (AMD) and Other Retinal Dystrophies. Semin Ophthalmol 2021; 36:304-309. [PMID: 33764255 DOI: 10.1080/08820538.2021.1896756] [Citation(s) in RCA: 7] [Impact Index Per Article: 2.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 12/30/2022]
Abstract
Artificial intelligence (AI), with its subdivisions (machine and deep learning), is a new branch of computer science that has shown impressive results across a variety of domains. The applications of AI to medicine and biology are being widely investigated. Medical specialties that rely heavily on images, including radiology, dermatology, oncology and ophthalmology, were the first to explore AI approaches in analysis and diagnosis. Applications of AI in ophthalmology have concentrated on diseases with high prevalence, such as diabetic retinopathy, retinopathy of prematurity, age-related macular degeneration (AMD), and glaucoma. Here we provide an overview of AI applications for diagnosis, classification, and clinical management of AMD and other macular dystrophies.
Collapse
Affiliation(s)
- Tatiana Perepelkina
- Department of Ophthalmology, Boston Children's Hospital, Harvard Medical School, Boston, United States
| | - Anne B Fulton
- Department of Ophthalmology, Boston Children's Hospital, Harvard Medical School, Boston, United States
| |
Collapse
|
99
|
Hani M, Ben Slama A, Zghal I, Trabelsi H. Appropriate identification of age-related macular degeneration using OCT images. COMPUTER METHODS IN BIOMECHANICS AND BIOMEDICAL ENGINEERING: IMAGING & VISUALIZATION 2021. [DOI: 10.1080/21681163.2020.1827041] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 10/23/2022]
Affiliation(s)
- Marwa Hani
- University of Tunis El Manar, ISTMT, Laboratory of Biophysics and Medical Technologies (BTM), LR13ES07, 1006, Tunis, Tunisia
| | - Amine Ben Slama
- University of Tunis El Manar, ISTMT, Laboratory of Biophysics and Medical Technologies (BTM), LR13ES07, 1006, Tunis, Tunisia
| | - Imen Zghal
- Hedi Raies Institute of Ophtalmology, Beb Sâadoun, 1007, Tunis, Tunisia
| | - Hedi Trabelsi
- University of Tunis El Manar, ISTMT, Laboratory of Biophysics and Medical Technologies (BTM), LR13ES07, 1006, Tunis, Tunisia
| |
Collapse
|
100
|
Thomas A, Sunija AP, Manoj R, Ramachandran R, Ramachandran S, Varun PG, Palanisamy P. RPE layer detection and baseline estimation using statistical methods and randomization for classification of AMD from retinal OCT. COMPUTER METHODS AND PROGRAMS IN BIOMEDICINE 2021; 200:105822. [PMID: 33190943 DOI: 10.1016/j.cmpb.2020.105822] [Citation(s) in RCA: 7] [Impact Index Per Article: 2.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 04/19/2020] [Accepted: 10/27/2020] [Indexed: 06/11/2023]
Abstract
BACKGROUND AND OBJECTIVE Age-related macular degeneration (AMD) is a condition of the eye that affects the aged people. Optical coherence tomography (OCT) is a diagnostic tool capable of analyzing and identifying the disease affected retinal layers with high resolution. The objective of this work is to extract the retinal pigment epithelium (RPE) layer and the baseline (natural eye curvature, particular to every patient) from retinal spectral-domain OCT (SD-OCT) images. It uses them to find the height of drusen (abnormalities) in the RPE layer and classify it as AMD or normal. METHODS In the proposed work, the contrast enhancement based adaptive denoising technique is used for speckle elimination. Pixel grouping and iterative elimination based on the knowledge of typical layer intensities and positions are used to obtain the RPE layer. Using this estimate, randomization techniques are employed, followed by polynomial fitting and drusen removal to arrive at a baseline estimate. The classification is based on the drusen height obtained by taking the difference between the RPE and baseline levels. We have used a patient, wise classification approach where a patient is classified diseased if more than a threshold number of patient images have drusen of more than a certain height. Since all slices of an affected patient will not show drusen, we are justified in adopting this technique. RESULTS The proposed method is tested on a public data set of 2130 images/slices, which belonged to 30 patient volumes (15 AMD and 15 Normal) and achieved an overall accuracy of 96.66%, with no false positives. In comparison with existing works, the proposed method achieved higher overall accuracy and a better baseline estimate. CONCLUSIONS The proposed work focuses on AMD/normal classification using a statistical approach. It does not require any training. The proposed method modifies the motion restoration paradigm to obtain an application-specific denoising algorithm. The existing RPE detection algorithm is modified significantly to make it robust and applicable even to images where the RPE is not very evident/there is a significant amount of perforations (drusen). The baseline estimation algorithm employs a powerful combination of randomization, iterative polynomial fitting, and pixel elimination in contrast to mere fitting techniques. The main highlight of this work is, it achieved an exact estimation of the baseline in the retinal image compared to the existing methods.
Collapse
Affiliation(s)
- Anju Thomas
- Department of Electronics and Communication Engineering, National Institute of Technology Tiruchirappalli, Tamilnadu 620015, India.
| | - A P Sunija
- Department of Electronics and Communication Engineering, National Institute of Technology Tiruchirappalli, Tamilnadu 620015, India.
| | - Rigved Manoj
- Department of Electronics and Communication Engineering, National Institute of Technology Tiruchirappalli, Tamilnadu 620015, India.
| | - Rajiv Ramachandran
- Department of Electronics and Communication Engineering, National Institute of Technology Tiruchirappalli, Tamilnadu 620015, India.
| | - Srikkanth Ramachandran
- Department of Electronics and Communication Engineering, National Institute of Technology Tiruchirappalli, Tamilnadu 620015, India.
| | - P Gopi Varun
- Department of Electronics and Communication Engineering, National Institute of Technology Tiruchirappalli, Tamilnadu 620015, India.
| | - P Palanisamy
- Department of Electronics and Communication Engineering, National Institute of Technology Tiruchirappalli, Tamilnadu 620015, India.
| |
Collapse
|