1
|
Târcoveanu F, Leon F, Lisa C, Curteanu S, Feraru A, Ali K, Anton N. The use of artificial neural networks in studying the progression of glaucoma. Sci Rep 2024; 14:19597. [PMID: 39179625 PMCID: PMC11344130 DOI: 10.1038/s41598-024-70748-1] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/10/2024] [Accepted: 08/20/2024] [Indexed: 08/26/2024] Open
Abstract
In ophthalmology, artificial intelligence methods show great promise due to their potential to enhance clinical observations with predictive capabilities and support physicians in diagnosing and treating patients. This paper focuses on modelling glaucoma evolution because it requires early diagnosis, individualized treatment, and lifelong monitoring. Glaucoma is a chronic, progressive, irreversible, multifactorial optic neuropathy that primarily affects elderly individuals. It is important to emphasize that the processed data are taken from medical records, unlike other studies in the literature that rely on image acquisition and processing. Although more challenging to handle, this approach has the advantage of including a wide range of parameters in large numbers, which can highlight their potential influence. Artificial neural networks are used to study glaucoma progression, designed through successive trials for near-optimal configurations using the NeuroSolutions and PyTorch frameworks. Furthermore, different problems are formulated to demonstrate the influence of various structural and functional parameters on the study of glaucoma progression. Optimal neural networks were obtained using a program written in Python using the PyTorch deep learning framework. For various tasks, very small errors in training and validation, under 5%, were obtained. It has been demonstrated that very good results can be achieved, making them credible and useful for medical practice.
Collapse
Affiliation(s)
- Filip Târcoveanu
- Ophthalmology Department, Faculty of Medicine, University of Medicine and Pharmacy "Gr. T. Popa" Iasi, University Street No 16, 700115, Iasi, Romania
| | - Florin Leon
- Faculty of Automatic Control and Computer Engineering, "Gheorghe Asachi" Technical University of Iasi, 27 Mangeron Street, 700050, Iasi, Romania
| | - Cătălin Lisa
- Department of Chemical Engineering, Faculty of Chemical Engineering and Environmental Protection "Cristofor Simionescu", "Gheorghe Asachi" Technical University of Iasi, 73 Mangeron Street, 700050, Iasi, Romania
| | - Silvia Curteanu
- Department of Chemical Engineering, Faculty of Chemical Engineering and Environmental Protection "Cristofor Simionescu", "Gheorghe Asachi" Technical University of Iasi, 73 Mangeron Street, 700050, Iasi, Romania.
| | - Andreea Feraru
- Faculty of Economic Science, "Vasile Alecsandri" University of Bacau, Calea Marasesti 156, 600115, Bacau, Romania
| | - Kashif Ali
- Countess of Chester Hospital, Liverpool Rd, Chester, CH21UL, UK
| | - Nicoleta Anton
- Ophthalmology Department, Faculty of Medicine, University of Medicine and Pharmacy "Gr. T. Popa" Iasi, University Street No 16, 700115, Iasi, Romania.
| |
Collapse
|
2
|
Christopher M, Hallaj S, Jiravarnsirikul A, Baxter SL, Zangwill LM. Novel Technologies in Artificial Intelligence and Telemedicine for Glaucoma Screening. J Glaucoma 2024; 33:S26-S32. [PMID: 38506792 DOI: 10.1097/ijg.0000000000002367] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/11/2024] [Accepted: 01/22/2024] [Indexed: 03/21/2024]
Abstract
PURPOSE To provide an overview of novel technologies in telemedicine and artificial intelligence (AI) approaches for cost-effective glaucoma screening. METHODS/RESULTS A narrative review was performed by summarizing research results, recent developments in glaucoma detection and care, and considerations related to telemedicine and AI in glaucoma screening. Telemedicine and AI approaches provide the opportunity for novel glaucoma screening programs in primary care, optometry, portable, and home-based settings. These approaches offer several advantages for glaucoma screening, including increasing access to care, lowering costs, identifying patients in need of urgent treatment, and enabling timely diagnosis and early intervention. However, challenges remain in implementing these systems, including integration into existing clinical workflows, ensuring equity for patients, and meeting ethical and regulatory requirements. Leveraging recent work towards standardized data acquisition as well as tools and techniques developed for automated diabetic retinopathy screening programs may provide a model for a cost-effective approach to glaucoma screening. CONCLUSION Leveraging novel technologies and advances in telemedicine and AI-based approaches to glaucoma detection show promise for improving our ability to detect moderate and advanced glaucoma in primary care settings and target higher individuals at high risk for having the disease.
Collapse
Affiliation(s)
- Mark Christopher
- Viterbi Family Department of Ophthalmology, Hamilton Glaucoma Center
- Viterbi Family Department of Ophthalmology, Division of Ophthalmology Informatics and Data Science, Shiley Eye Institute
| | - Shahin Hallaj
- Viterbi Family Department of Ophthalmology, Hamilton Glaucoma Center
- Viterbi Family Department of Ophthalmology, Division of Ophthalmology Informatics and Data Science, Shiley Eye Institute
| | - Anuwat Jiravarnsirikul
- Viterbi Family Department of Ophthalmology, Hamilton Glaucoma Center
- Department of Medicine, Division of Biomedical Informatics, University of California San Diego, La Jolla, CA
| | - Sally L Baxter
- Viterbi Family Department of Ophthalmology, Hamilton Glaucoma Center
- Viterbi Family Department of Ophthalmology, Division of Ophthalmology Informatics and Data Science, Shiley Eye Institute
- Department of Medicine, Division of Biomedical Informatics, University of California San Diego, La Jolla, CA
| | - Linda M Zangwill
- Viterbi Family Department of Ophthalmology, Hamilton Glaucoma Center
- Viterbi Family Department of Ophthalmology, Division of Ophthalmology Informatics and Data Science, Shiley Eye Institute
| |
Collapse
|
3
|
Pham AT, Pan AA, Bradley C, Hou K, Herbert P, Johnson C, Wall M, Yohannan J. Detecting Visual Field Worsening From Optic Nerve Head and Macular Optical Coherence Tomography Thickness Measurements. Transl Vis Sci Technol 2024; 13:12. [PMID: 39115839 PMCID: PMC11316451 DOI: 10.1167/tvst.13.8.12] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/12/2023] [Accepted: 06/20/2024] [Indexed: 08/12/2024] Open
Abstract
Purpose Compare the use of optic disc and macular optical coherence tomography measurements to predict glaucomatous visual field (VF) worsening. Methods Machine learning and statistical models were trained on 924 eyes (924 patients) with circumpapillary retinal nerve fiber layer (cp-RNFL) or ganglion cell inner plexiform layer (GC-IPL) thickness measurements. The probability of 24-2 VF worsening was predicted using both trend-based and event-based progression definitions of VF worsening. Additionally, the cp-RNFL and GC-IPL predictions were combined to produce a combined prediction. A held-out test set of 617 eyes was used to calculate the area under the curve (AUC) to compare cp-RNFL, GC-IPL, and combined predictions. Results The AUCs for cp-RNFL, GC-IPL, and combined predictions with the statistical and machine learning models were 0.72, 0.69, 0.73, and 0.78, 0.75, 0.81, respectively, when using trend-based analysis as ground truth. The differences in performance between the cp-RNFL, GC-IPL, and combined predictions were not statistically significant. AUCs were highest in glaucoma suspects using cp-RNFL predictions and highest in moderate/advanced glaucoma using GC-IPL predictions. The AUCs for the statistical and machine learning models were 0.63, 0.68, 0.69, and 0.72, 0.69, 0.73, respectively, when using event-based analysis. AUCs decreased with increasing disease severity for all predictions. Conclusions cp-RNFL and GC-IPL similarly predicted VF worsening overall, but cp-RNFL performed best in early glaucoma stages and GC-IPL in later stages. Combining both did not enhance detection significantly. Translational Relevance cp-RNFL best predicted trend-based 24-2 VF progression in early-stage disease, while GC-IPL best predicted progression in late-stage disease. Combining both features led to minimal improvement in predicting progression.
Collapse
Affiliation(s)
- Alex T. Pham
- Wilmer Eye Institute, Johns Hopkins University School of Medicine, Baltimore, MD, USA
| | - Annabelle A. Pan
- Wilmer Eye Institute, Johns Hopkins University School of Medicine, Baltimore, MD, USA
| | - Chris Bradley
- Wilmer Eye Institute, Johns Hopkins University School of Medicine, Baltimore, MD, USA
| | - Kaihua Hou
- Malone Center for Engineering in Healthcare, Johns Hopkins University, Baltimore, MD, USA
| | - Patrick Herbert
- Malone Center for Engineering in Healthcare, Johns Hopkins University, Baltimore, MD, USA
| | | | | | - Jithin Yohannan
- Wilmer Eye Institute, Johns Hopkins University School of Medicine, Baltimore, MD, USA
- Malone Center for Engineering in Healthcare, Johns Hopkins University, Baltimore, MD, USA
| |
Collapse
|
4
|
Jalili J, Nadimi M, Jafari B, Esfandiari A, Mojarad M, Subramanian PS, Aghsaei Fard M. Vessel Density Features of Optical Coherence Tomography Angiography for Classification of Optic Neuropathies Using Machine Learning. J Neuroophthalmol 2024; 44:41-46. [PMID: 37440373 DOI: 10.1097/wno.0000000000001925] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 07/15/2023]
Abstract
BACKGROUND To evaluate the classification performance of machine learning based on the 4 vessel density features of peripapillary optical coherence tomography angiography (OCT-A) for classifying healthy, nonarteritic anterior ischemic optic neuropathy (NAION), and optic neuritis (ON) eyes. METHODS Forty-five eyes of 45 NAION patients, 32 eyes of 32 ON patients, and 76 eyes of 76 healthy individuals with optic nerve head OCT-A were included. Four vessel density features of OCT-A images were developed using a threshold-based segmentation method and were integrated in 3 models of machine learning classifiers. Classification performances of support vector machine (SVM), random forest, and Gaussian Naive Bayes (GNB) models were evaluated with the area under the receiver-operating-characteristic curve (AUC) and accuracy. RESULTS We divided 121 images into a 70% training set and 30% test set. For ON-NAION classification, best results were achieved with 50% threshold, in which 3 classifiers (SVM, RF, and GNB) discriminated ON from NAION with an AUC of 1 and accuracy of 1. For ON-Normal classification, with 100% threshold, SVM and RF classifiers were able to discriminate normal from ON with AUCs of 1 and accuracies of 1. For NAION-normal classification, with 50% threshold, the SVM and RF classified the NAION from normal with AUC and accuracy of 1. CONCLUSIONS ML based on the combined peripapillary vessel density features of total vessels and capillaries in the whole image and ring image could provide excellent performance for NAION and ON distinction.
Collapse
Affiliation(s)
- Jalil Jalili
- Biomedical Engineering Unit (JJ, MN), Cardiovascular Disease Research Center, Heshmat Hospital, School of Medicine, Guilan University of Medical Sciences, Rasht, Iran; Farabi Eye Hospital (BJ, AE, MAF), Tehran University of Medical Sciences, Tehran, Iran; School of Medicine (MM), Guilan University of Medical Sciences, Rasht, Iran; and Department of Ophthalmology (PSS), University of Colorado, School of Medicine, Aurora, Colorado
| | | | | | | | | | | | | |
Collapse
|
5
|
Jalili J, Nadimi M, Jafari B, Esfandiari A, Sadeghi R, Ghahari P, Sajedi M, Safizade M, Aghsaei Fard M. Vessel Density Features of Optical Coherence Tomography Angiography for Classification of Glaucoma Using Machine Learning. J Glaucoma 2023; 32:1006-1010. [PMID: 37974327 DOI: 10.1097/ijg.0000000000002329] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/23/2023] [Accepted: 10/10/2023] [Indexed: 11/19/2023]
Abstract
PRCIS Machine learning (ML) based on the optical coherence tomography angiography vessel density features with different thresholds using a support vector machine (SVM) model provides excellent performance for glaucoma detection. BACKGROUND To assess the classification performance of ML based on the 4 vessel density features of peripapillary optical coherence tomography angiography for glaucoma detection. METHODS Images from 119 eyes of 119 glaucoma patients and 76 eyes of 76 healthy individuals were included. Four vessel density features of optical coherence tomography angiography images were developed using a threshold-based segmentation method and were integrated into 3 models of machine learning classifiers. Images were divided into 70% training set and 30% test set. Classification performances of SVM, random forest, and Gaussian Naive Bayes models were evaluated with the area under the receiver operating characteristic curve (AUC) and accuracy. RESULTS Glaucoma eyes had lower vessel densities at different thresholds. For differentiating glaucoma eyes, the best results were achieved with 70% and 100% thresholds, in which SVM classifier discriminated glaucoma from healthy eyes with an AUC of 1 and accuracy of 1. After SVM, the random forest classifier with 100% thresholds showed an AUC of 0.993 and an accuracy of 0.994. Furthermore, the AUC of our ML performance (SVM) was 0.96 in a subgroup analysis of mild and moderate glaucoma eyes. CONCLUSIONS ML based on the combined peripapillary vessel density features of total vessels and capillaries in the whole image and ring image could provide excellent performance for glaucoma detection.
Collapse
Affiliation(s)
- Jalil Jalili
- Biomedical Engineering Unit, Cardiovascular Disease Research Center, Heshmat Hospital, School of Medicine, Guilan University of Medical Sciences, Rasht
| | - Mohadeseh Nadimi
- Biomedical Engineering Unit, Cardiovascular Disease Research Center, Heshmat Hospital, School of Medicine, Guilan University of Medical Sciences, Rasht
| | - Behzad Jafari
- Farabi Eye Hospital, Tehran University of Medical Sciences
| | | | - Reza Sadeghi
- Farabi Eye Hospital, Tehran University of Medical Sciences
| | - Parichehr Ghahari
- School of Medicine, Tehran University of Medical Sciences, Tehran, Iran
| | | | - Mona Safizade
- Farabi Eye Hospital, Tehran University of Medical Sciences
| | | |
Collapse
|
6
|
Yousefi S. Clinical Applications of Artificial Intelligence in Glaucoma. J Ophthalmic Vis Res 2023; 18:97-112. [PMID: 36937202 PMCID: PMC10020779 DOI: 10.18502/jovr.v18i1.12730] [Citation(s) in RCA: 8] [Impact Index Per Article: 8.0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/16/2022] [Accepted: 11/05/2022] [Indexed: 02/25/2023] Open
Abstract
Ophthalmology is one of the major imaging-intensive fields of medicine and thus has potential for extensive applications of artificial intelligence (AI) to advance diagnosis, drug efficacy, and other treatment-related aspects of ocular disease. AI has made impressive progress in ophthalmology within the past few years and two autonomous AI-enabled systems have received US regulatory approvals for autonomously screening for mid-level or advanced diabetic retinopathy and macular edema. While no autonomous AI-enabled system for glaucoma screening has yet received US regulatory approval, numerous assistive AI-enabled software tools are already employed in commercialized instruments for quantifying retinal images and visual fields to augment glaucoma research and clinical practice. In this literature review (non-systematic), we provide an overview of AI applications in glaucoma, and highlight some limitations and considerations for AI integration and adoption into clinical practice.
Collapse
Affiliation(s)
- Siamak Yousefi
- Department of Ophthalmology, University of Tennessee Health Science Center, Memphis, TN, USA
- Department of Genetics, Genomics, and Informatics, University of Tennessee Health Science Center, Memphis, TN, USA
| |
Collapse
|
7
|
Niranjan V, Uttarkar A, Kaul A, Varghese M. A Machine Learning-Based Approach Using Multi-omics Data to Predict Metabolic Pathways. Methods Mol Biol 2023; 2553:441-452. [PMID: 36227554 DOI: 10.1007/978-1-0716-2617-7_19] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 06/16/2023]
Abstract
The integrative method approaches are continuously evolving to provide accurate insights from the data that is received through experimentation on various biological systems. Multi-omics data can be integrated with predictive machine learning algorithms in order to provide results with high accuracy. This protocol chapter defines the steps required for the ML-multi-omics integration methods that are applied on biological datasets for its analysis and the visual interpretation of the results thus obtained.
Collapse
Affiliation(s)
- Vidya Niranjan
- Department of Biotechnology, R V College of Engineering, Mysuru Road, Kengeri, Bengaluru, India.
| | - Akshay Uttarkar
- Department of Biotechnology, R V College of Engineering, Mysuru Road, Kengeri, Bengaluru, India
| | - Aakaanksha Kaul
- Department of Biotechnology, R V College of Engineering, Mysuru Road, Kengeri, Bengaluru, India
| | - Maryanne Varghese
- Department of Biotechnology, R V College of Engineering, Mysuru Road, Kengeri, Bengaluru, India
| |
Collapse
|
8
|
Anton N, Doroftei B, Curteanu S, Catãlin L, Ilie OD, Târcoveanu F, Bogdănici CM. Comprehensive Review on the Use of Artificial Intelligence in Ophthalmology and Future Research Directions. Diagnostics (Basel) 2022; 13:100. [PMID: 36611392 PMCID: PMC9818832 DOI: 10.3390/diagnostics13010100] [Citation(s) in RCA: 8] [Impact Index Per Article: 4.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/03/2022] [Revised: 12/12/2022] [Accepted: 12/26/2022] [Indexed: 12/31/2022] Open
Abstract
BACKGROUND Having several applications in medicine, and in ophthalmology in particular, artificial intelligence (AI) tools have been used to detect visual function deficits, thus playing a key role in diagnosing eye diseases and in predicting the evolution of these common and disabling diseases. AI tools, i.e., artificial neural networks (ANNs), are progressively involved in detecting and customized control of ophthalmic diseases. The studies that refer to the efficiency of AI in medicine and especially in ophthalmology were analyzed in this review. MATERIALS AND METHODS We conducted a comprehensive review in order to collect all accounts published between 2015 and 2022 that refer to these applications of AI in medicine and especially in ophthalmology. Neural networks have a major role in establishing the demand to initiate preliminary anti-glaucoma therapy to stop the advance of the disease. RESULTS Different surveys in the literature review show the remarkable benefit of these AI tools in ophthalmology in evaluating the visual field, optic nerve, and retinal nerve fiber layer, thus ensuring a higher precision in detecting advances in glaucoma and retinal shifts in diabetes. We thus identified 1762 applications of artificial intelligence in ophthalmology: review articles and research articles (301 pub med, 144 scopus, 445 web of science, 872 science direct). Of these, we analyzed 70 articles and review papers (diabetic retinopathy (N = 24), glaucoma (N = 24), DMLV (N = 15), other pathologies (N = 7)) after applying the inclusion and exclusion criteria. CONCLUSION In medicine, AI tools are used in surgery, radiology, gynecology, oncology, etc., in making a diagnosis, predicting the evolution of a disease, and assessing the prognosis in patients with oncological pathologies. In ophthalmology, AI potentially increases the patient's access to screening/clinical diagnosis and decreases healthcare costs, mainly when there is a high risk of disease or communities face financial shortages. AI/DL (deep learning) algorithms using both OCT and FO images will change image analysis techniques and methodologies. Optimizing these (combined) technologies will accelerate progress in this area.
Collapse
Affiliation(s)
- Nicoleta Anton
- Faculty of Medicine, University of Medicine and Pharmacy “Grigore T. Popa”, University Street, No 16, 700115 Iasi, Romania
| | - Bogdan Doroftei
- Faculty of Medicine, University of Medicine and Pharmacy “Grigore T. Popa”, University Street, No 16, 700115 Iasi, Romania
| | - Silvia Curteanu
- Department of Chemical Engineering, Cristofor Simionescu Faculty of Chemical Engineering and Environmental Protection, Gheorghe Asachi Technical University, Prof.dr.doc Dimitrie Mangeron Avenue, No 67, 700050 Iasi, Romania
| | - Lisa Catãlin
- Department of Chemical Engineering, Cristofor Simionescu Faculty of Chemical Engineering and Environmental Protection, Gheorghe Asachi Technical University, Prof.dr.doc Dimitrie Mangeron Avenue, No 67, 700050 Iasi, Romania
| | - Ovidiu-Dumitru Ilie
- Department of Biology, Faculty of Biology, “Alexandru Ioan Cuza” University, Carol I Avenue, No 20A, 700505 Iasi, Romania
| | - Filip Târcoveanu
- Faculty of Medicine, University of Medicine and Pharmacy “Grigore T. Popa”, University Street, No 16, 700115 Iasi, Romania
| | - Camelia Margareta Bogdănici
- Faculty of Medicine, University of Medicine and Pharmacy “Grigore T. Popa”, University Street, No 16, 700115 Iasi, Romania
| |
Collapse
|
9
|
Li W, Shao C, Zhou H, Du H, Chen H, Wan H, He Y. Multi-omics research strategies in ischemic stroke: A multidimensional perspective. Ageing Res Rev 2022; 81:101730. [PMID: 36087702 DOI: 10.1016/j.arr.2022.101730] [Citation(s) in RCA: 19] [Impact Index Per Article: 9.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/19/2022] [Revised: 08/23/2022] [Accepted: 09/03/2022] [Indexed: 01/31/2023]
Abstract
Ischemic stroke (IS) is a multifactorial and heterogeneous neurological disorder with high rate of death and long-term impairment. Despite years of studies, there are still no stroke biomarkers for clinical practice, and the molecular mechanisms of stroke remain largely unclear. The high-throughput omics approach provides new avenues for discovering biomarkers of IS and explaining its pathological mechanisms. However, single-omics approaches only provide a limited understanding of the biological pathways of diseases. The integration of multiple omics data means the simultaneous analysis of thousands of genes, RNAs, proteins and metabolites, revealing networks of interactions between multiple molecular levels. Integrated analysis of multi-omics approaches will provide helpful insights into stroke pathogenesis, therapeutic target identification and biomarker discovery. Here, we consider advances in genomics, transcriptomics, proteomics and metabolomics and outline their use in discovering the biomarkers and pathological mechanisms of IS. We then delineate strategies for achieving integration at the multi-omics level and discuss how integrative omics and systems biology can contribute to our understanding and management of IS.
Collapse
Affiliation(s)
- Wentao Li
- School of Pharmaceutical Sciences, Zhejiang Chinese Medical University, Hangzhou 310053, China.
| | - Chongyu Shao
- School of Life Sciences, Zhejiang Chinese Medical University, Hangzhou 310053, China.
| | - Huifen Zhou
- School of Life Sciences, Zhejiang Chinese Medical University, Hangzhou 310053, China.
| | - Haixia Du
- School of Life Sciences, Zhejiang Chinese Medical University, Hangzhou 310053, China.
| | - Haiyang Chen
- School of Pharmaceutical Sciences, Zhejiang Chinese Medical University, Hangzhou 310053, China.
| | - Haitong Wan
- School of Life Sciences, Zhejiang Chinese Medical University, Hangzhou 310053, China.
| | - Yu He
- School of Pharmaceutical Sciences, Zhejiang Chinese Medical University, Hangzhou 310053, China.
| |
Collapse
|
10
|
Deep Learning Image Analysis of Optical Coherence Tomography Angiography Measured Vessel Density Improves Classification of Healthy and Glaucoma Eyes. Am J Ophthalmol 2022; 236:298-308. [PMID: 34780803 PMCID: PMC10042115 DOI: 10.1016/j.ajo.2021.11.008] [Citation(s) in RCA: 15] [Impact Index Per Article: 7.5] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/06/2021] [Revised: 11/07/2021] [Accepted: 11/08/2021] [Indexed: 11/23/2022]
Abstract
PURPOSE To compare convolutional neural network (CNN) analysis of en face vessel density images to gradient boosting classifier (GBC) analysis of instrument-provided, feature-based optical coherence tomography angiography (OCTA) vessel density measurements and OCT retinal nerve fiber layer (RNFL) thickness measurements for classifying healthy and glaucomatous eyes. DESIGN Comparison of diagnostic approaches. METHODS A total of 130 eyes of 80 healthy individuals and 275 eyes of 185 glaucoma patients with optic nerve head (ONH) OCTA and OCT imaging were included. Classification performance of a VGG16 CNN trained and tested on entire en face 4.5 × 4.5-mm radial peripapillary capillary OCTA ONH images was compared to the performance of separate GBC models trained and tested on standard OCTA and OCT measurements. Five-fold cross-validation was used to test predictions for CNNs and GBCs. Areas under the precision recall curves (AUPRC) were calculated to control for training/test set size imbalance and were compared. RESULTS Adjusted AUPRCs for GBC models were 0.89 (95% CI = 0.82, 0.92) for whole image vessel density GBC, 0.89 (0.83, 0.92) for whole image capillary density GBC, 0.91 (0.88, 0.93) for combined whole image vessel and whole image capillary density GBC, and 0.93 (0.91, 095) for RNFL thickness GBC. The adjusted AUPRC using CNN analysis of en face vessel density images was 0.97 (0.95, 0.99) resulting in significantly improved classification compared to GBC OCTA-based results and GBC OCT-based results (P ≤ 0.01 for all comparisons). CONCLUSION Deep learning en face image analysis improves on feature-based GBC models for classifying healthy and glaucoma eyes.
Collapse
|
11
|
Abstract
Early detection and monitoring are critical to the diagnosis and management of glaucoma, a progressive optic neuropathy that causes irreversible blindness. Optical coherence tomography (OCT) has become a commonly utilized imaging modality that aids in the detection and monitoring of structural glaucomatous damage. Since its inception in 1991, OCT has progressed through multiple iterations, from time-domain OCT, to spectral-domain OCT, to swept-source OCT, all of which have progressively improved the resolution and speed of scans. Even newer technological advancements and OCT applications, such as adaptive optics, visible-light OCT, and OCT-angiography, have enriched the use of OCT in the evaluation of glaucoma. This article reviews current commercial and state-of-the-art OCT technologies and analytic techniques in the context of their utility for glaucoma diagnosis and management, as well as promising future directions.
Collapse
Affiliation(s)
- Alexi Geevarghese
- Department of Ophthalmology, NYU Langone Health, NYU Grossman School of Medicine, New York, NY 10016, USA;
| | - Gadi Wollstein
- Department of Ophthalmology, NYU Langone Health, NYU Grossman School of Medicine, New York, NY 10016, USA;
- Department of Biomedical Engineering, NYU Tandon School of Engineering, Brooklyn, New York 11201, USA
- Center for Neural Science, NYU College of Arts and Sciences, New York, NY 10003, USA
| | - Hiroshi Ishikawa
- Department of Ophthalmology, NYU Langone Health, NYU Grossman School of Medicine, New York, NY 10016, USA;
- Department of Biomedical Engineering, NYU Tandon School of Engineering, Brooklyn, New York 11201, USA
| | - Joel S Schuman
- Department of Ophthalmology, NYU Langone Health, NYU Grossman School of Medicine, New York, NY 10016, USA;
- Department of Biomedical Engineering, NYU Tandon School of Engineering, Brooklyn, New York 11201, USA
- Center for Neural Science, NYU College of Arts and Sciences, New York, NY 10003, USA
- Department of Physiology and Neuroscience, NYU Langone Health, NYU Grossman School of Medicine, New York, NY 10016, USA
| |
Collapse
|
12
|
Reel PS, Reel S, Pearson E, Trucco E, Jefferson E. Using machine learning approaches for multi-omics data analysis: A review. Biotechnol Adv 2021; 49:107739. [PMID: 33794304 DOI: 10.1016/j.biotechadv.2021.107739] [Citation(s) in RCA: 284] [Impact Index Per Article: 94.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/15/2020] [Revised: 03/01/2021] [Accepted: 03/25/2021] [Indexed: 02/06/2023]
Abstract
With the development of modern high-throughput omic measurement platforms, it has become essential for biomedical studies to undertake an integrative (combined) approach to fully utilise these data to gain insights into biological systems. Data from various omics sources such as genetics, proteomics, and metabolomics can be integrated to unravel the intricate working of systems biology using machine learning-based predictive algorithms. Machine learning methods offer novel techniques to integrate and analyse the various omics data enabling the discovery of new biomarkers. These biomarkers have the potential to help in accurate disease prediction, patient stratification and delivery of precision medicine. This review paper explores different integrative machine learning methods which have been used to provide an in-depth understanding of biological systems during normal physiological functioning and in the presence of a disease. It provides insight and recommendations for interdisciplinary professionals who envisage employing machine learning skills in multi-omics studies.
Collapse
Affiliation(s)
- Parminder S Reel
- Division of Population Health and Genomics, School of Medicine, University of Dundee, Dundee, United Kingdom
| | - Smarti Reel
- Division of Population Health and Genomics, School of Medicine, University of Dundee, Dundee, United Kingdom
| | - Ewan Pearson
- Division of Population Health and Genomics, School of Medicine, University of Dundee, Dundee, United Kingdom
| | - Emanuele Trucco
- VAMPIRE project, Computing, School of Science and Engineering, University of Dundee, Dundee, United Kingdom
| | - Emily Jefferson
- Division of Population Health and Genomics, School of Medicine, University of Dundee, Dundee, United Kingdom.
| |
Collapse
|
13
|
Assessing Changes in Diabetic Retinopathy Caused by Diabetes Mellitus and Glaucoma Using Support Vector Machines in Combination with Differential Evolution Algorithm. APPLIED SCIENCES-BASEL 2021. [DOI: 10.3390/app11093944] [Citation(s) in RCA: 9] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 11/16/2022]
Abstract
The aim of this study is to evaluate the changes related to diabetic retinopathy (DR) (no changes, small or moderate changes) in patients with glaucoma and diabetes using artificial intelligence instruments: Support Vector Machines (SVM) in combination with a powerful optimization algorithm—Differential Evolution (DE). In order to classify the DR changes and to make predictions in various situations, an approach including SVM optimized with DE was applied. The role of the optimizer was to automatically determine the SVM parameters that lead to the lowest classification error. The study was conducted on a sample of 52 patients: particularly, 101 eyes with glaucoma and diabetes mellitus, in the Ophthalmology Clinic I of the “St. Spiridon” Clinical Hospital of Iaşi. The criteria considered in the modelling action were normal or hypertensive open-angle glaucoma, intraocular hypertension and associated diabetes. The patients with other types of glaucoma pseudoexfoliation, pigment, cortisone, neovascular and primitive angle-closure, and those without associated diabetes, were excluded. The assessment of diabetic retinopathy changes were carried out with Volk lens and Fundus Camera Zeiss retinal photography on the dilated pupil, inspecting all quadrants. The criteria for classifying the DR (early treatment diabetic retinopathy study—ETDRS) changes were: without changes (absence of DR), mild forma nonproliferative diabetic retinopathy (the presence of a single micro aneurysm), moderate form (micro aneurysms, hemorrhages in 2–3 quadrants, venous dilatations and soft exudates in a quadrant), severe form (micro aneurysms, hemorrhages in all quadrants, venous dilatation in 2–3 quadrants) and proliferative diabetic retinopathy (disk and retinal neovascularization in different quadrants). Any new clinical element that occurred in subsequent checks, which led to their inclusion in severe nonproliferative or proliferative forms of diabetic retinopathy, was considered to be the result of the progression of diabetic retinopathy. The results obtained were very good; in the testing phase, a 95.23% accuracy has been obtained, only one sample being wrongly classified. The effectiveness of the classification algorithm (SVM), developed in optimal form with DE, and used in predictions of retinal changes related to diabetes, was demonstrated.
Collapse
|
14
|
Bowd C, Belghith A, Proudfoot JA, Zangwill LM, Christopher M, Goldbaum MH, Hou H, Penteado RC, Moghimi S, Weinreb RN. Gradient-Boosting Classifiers Combining Vessel Density and Tissue Thickness Measurements for Classifying Early to Moderate Glaucoma. Am J Ophthalmol 2020; 217:131-139. [PMID: 32222368 DOI: 10.1016/j.ajo.2020.03.024] [Citation(s) in RCA: 22] [Impact Index Per Article: 5.5] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/14/2019] [Revised: 03/16/2020] [Accepted: 03/17/2020] [Indexed: 02/09/2023]
Abstract
PURPOSE To compare gradient-boosting classifier (GBC) analysis of optical coherence tomography angiography (OCTA)-measured vessel density (VD) and OCT-measured tissue thickness to standard OCTA VD and OCT thickness parameters for classifying healthy eyes and eyes with early to moderate glaucoma. DESIGN Comparison of diagnostic tools. METHODS A total of 180 healthy eyes and 193 glaucomatous eyes with OCTA and OCT imaging of the macula and optic nerve head (ONH) were studied. Four GBCs were evaluated that combined 1) all macula VD and thickness measurements (Macula GBC), 2) all ONH VD and thickness measurements (ONH GBC), 3) all VD measurements from the macula and ONH (vessel density GBC), and 4) all thickness measurements from the macula and ONH (thickness GBC). ROC curve (AUROC) analyses compared the diagnostic accuracy of GBCs to that of standard instrument-provided parameters. A fifth GBC that combined all parameters (full GBC) also was investigated. RESULTS GBCs had better diagnostic accuracy than standard OCTA and OCT parameters with AUROCs ranging from 0.90 to 0.93 and 0.64 to 0.91, respectively. The full GBC (AUROC = 0.93) performed significantly better than the ONH GBC (AUROC = 0.91; P = .036) and the vessel density GBC (AUROC = 0.90; P = .010). All other GBCs performed similarly. The mean relative influence of each parameter included in the full GBC identified a combination of macular thickness and ONH VD measurements as the greatest contributors. CONCLUSIONS GBCs that combine OCTA and OCT macula and ONH measurements can improve diagnostic accuracy for glaucoma detection compared to most but not all instrument provided parameters.
Collapse
|
15
|
Devalla SK, Liang Z, Pham TH, Boote C, Strouthidis NG, Thiery AH, Girard MJA. Glaucoma management in the era of artificial intelligence. Br J Ophthalmol 2019; 104:301-311. [DOI: 10.1136/bjophthalmol-2019-315016] [Citation(s) in RCA: 36] [Impact Index Per Article: 7.2] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/30/2019] [Revised: 09/07/2019] [Accepted: 10/05/2019] [Indexed: 12/20/2022]
Abstract
Glaucoma is a result of irreversible damage to the retinal ganglion cells. While an early intervention could minimise the risk of vision loss in glaucoma, its asymptomatic nature makes it difficult to diagnose until a late stage. The diagnosis of glaucoma is a complicated and expensive effort that is heavily dependent on the experience and expertise of a clinician. The application of artificial intelligence (AI) algorithms in ophthalmology has improved our understanding of many retinal, macular, choroidal and corneal pathologies. With the advent of deep learning, a number of tools for the classification, segmentation and enhancement of ocular images have been developed. Over the years, several AI techniques have been proposed to help detect glaucoma by analysis of functional and/or structural evaluations of the eye. Moreover, the use of AI has also been explored to improve the reliability of ascribing disease prognosis. This review summarises the role of AI in the diagnosis and prognosis of glaucoma, discusses the advantages and challenges of using AI systems in clinics and predicts likely areas of future progress.
Collapse
|
16
|
Abstract
PURPOSE OF REVIEW The use of computers has become increasingly relevant to medical decision-making, and artificial intelligence methods have recently demonstrated significant advances in medicine. We therefore provide an overview of current artificial intelligence methods and their applications, to help the practicing ophthalmologist understand their potential impact on glaucoma care. RECENT FINDINGS Techniques used in artificial intelligence can successfully analyze and categorize data from visual fields, optic nerve structure [e.g., optical coherence tomography (OCT) and fundus photography], ocular biomechanical properties, and a combination thereof to identify disease severity, determine disease progression, and/or recommend referral for specialized care. Algorithms have become increasingly complex in recent years, utilizing both supervised and unsupervised methods of artificial intelligence. Impressive performance of these algorithms on previously unseen data has been reported, often outperforming standard global indices and expert observers. However, there remains no clearly defined gold standard for determining the presence and severity of glaucoma, which undermines the training of these algorithms. To improve upon existing methodologies, future work must employ more robust definitions of disease, optimize data inputs for artificial intelligence analysis, and improve methods of extracting knowledge from learned results. SUMMARY Artificial intelligence has the potential to revolutionize the screening, diagnosis, and classification of glaucoma, both through the automated processing of large data sets, and by earlier detection of new disease patterns. In addition, artificial intelligence holds promise for fundamentally changing research aimed at understanding the development, progression, and treatment of glaucoma, by identifying novel risk factors and by evaluating the importance of existing ones.
Collapse
|
17
|
Yousefi S, Balasubramanian M, Goldbaum MH, Medeiros FA, Zangwill LM, Weinreb RN, Liebmann JM, Girkin CA, Bowd C. Unsupervised Gaussian Mixture-Model With Expectation Maximization for Detecting Glaucomatous Progression in Standard Automated Perimetry Visual Fields. Transl Vis Sci Technol 2016; 5:2. [PMID: 27152250 PMCID: PMC4855479 DOI: 10.1167/tvst.5.3.2] [Citation(s) in RCA: 42] [Impact Index Per Article: 5.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/27/2015] [Accepted: 03/06/2016] [Indexed: 11/24/2022] Open
Abstract
Purpose To validate Gaussian mixture-model with expectation maximization (GEM) and variational Bayesian independent component analysis mixture-models (VIM) for detecting glaucomatous progression along visual field (VF) defect patterns (GEM–progression of patterns (POP) and VIM-POP). To compare GEM-POP and VIM-POP with other methods. Methods GEM and VIM models separated cross-sectional abnormal VFs from 859 eyes and normal VFs from 1117 eyes into abnormal and normal clusters. Clusters were decomposed into independent axes. The confidence limit (CL) of stability was established for each axis with a set of 84 stable eyes. Sensitivity for detecting progression was assessed in a sample of 83 eyes with known progressive glaucomatous optic neuropathy (PGON). Eyes were classified as progressed if any defect pattern progressed beyond the CL of stability. Performance of GEM-POP and VIM-POP was compared to point-wise linear regression (PLR), permutation analysis of PLR (PoPLR), and linear regression (LR) of mean deviation (MD), and visual field index (VFI). Results Sensitivity and specificity for detecting glaucomatous VFs were 89.9% and 93.8%, respectively, for GEM and 93.0% and 97.0%, respectively, for VIM. Receiver operating characteristic (ROC) curve areas for classifying progressed eyes were 0.82 for VIM-POP, 0.86 for GEM-POP, 0.81 for PoPLR, 0.69 for LR of MD, and 0.76 for LR of VFI. Conclusions GEM-POP was significantly more sensitive to PGON than PoPLR and linear regression of MD and VFI in our sample, while providing localized progression information. Translational Relevance Detection of glaucomatous progression can be improved by assessing longitudinal changes in localized patterns of glaucomatous defect identified by unsupervised machine learning.
Collapse
Affiliation(s)
- Siamak Yousefi
- Hamilton Glaucoma Center and the Department of Ophthalmology University of California San Diego, La Jolla, CA, USA
| | - Madhusudhanan Balasubramanian
- Department of Electrical and Computer Engineering; Department of Biomedical Engineering, University of Memphis, Memphis, TN, USA
| | - Michael H Goldbaum
- Hamilton Glaucoma Center and the Department of Ophthalmology University of California San Diego, La Jolla, CA, USA
| | - Felipe A Medeiros
- Hamilton Glaucoma Center and the Department of Ophthalmology University of California San Diego, La Jolla, CA, USA
| | - Linda M Zangwill
- Hamilton Glaucoma Center and the Department of Ophthalmology University of California San Diego, La Jolla, CA, USA
| | - Robert N Weinreb
- Hamilton Glaucoma Center and the Department of Ophthalmology University of California San Diego, La Jolla, CA, USA
| | | | | | - Christopher Bowd
- Hamilton Glaucoma Center and the Department of Ophthalmology University of California San Diego, La Jolla, CA, USA
| |
Collapse
|
18
|
Michelessi M, Lucenteforte E, Oddone F, Brazzelli M, Parravano M, Franchi S, Ng SM, Virgili G. Optic nerve head and fibre layer imaging for diagnosing glaucoma. Cochrane Database Syst Rev 2015; 2015:CD008803. [PMID: 26618332 PMCID: PMC4732281 DOI: 10.1002/14651858.cd008803.pub2] [Citation(s) in RCA: 33] [Impact Index Per Article: 3.7] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 12/24/2022]
Abstract
BACKGROUND The diagnosis of glaucoma is traditionally based on the finding of optic nerve head (ONH) damage assessed subjectively by ophthalmoscopy or photography or by corresponding damage to the visual field assessed by automated perimetry, or both. Diagnostic assessments are usually required when ophthalmologists or primary eye care professionals find elevated intraocular pressure (IOP) or a suspect appearance of the ONH. Imaging tests such as confocal scanning laser ophthalmoscopy (HRT), optical coherence tomography (OCT) and scanning laser polarimetry (SLP, as used by the GDx instrument), provide an objective measure of the structural changes of retinal nerve fibre layer (RNFL) thickness and ONH parameters occurring in glaucoma. OBJECTIVES To determine the diagnostic accuracy of HRT, OCT and GDx for diagnosing manifest glaucoma by detecting ONH and RNFL damage. SEARCH METHODS We searched several databases for this review. The most recent searches were on 19 February 2015. SELECTION CRITERIA We included prospective and retrospective cohort studies and case-control studies that evaluated the accuracy of OCT, HRT or the GDx for diagnosing glaucoma. We excluded population-based screening studies, since we planned to consider studies on self-referred people or participants in whom a risk factor for glaucoma had already been identified in primary care, such as elevated IOP or a family history of glaucoma. We only considered recent commercial versions of the tests: spectral domain OCT, HRT III and GDx VCC or ECC. DATA COLLECTION AND ANALYSIS We adopted standard Cochrane methods. We fitted a hierarchical summary ROC (HSROC) model using the METADAS macro in SAS software. After studies were selected, we decided to use 2 x 2 data at 0.95 specificity or closer in meta-analyses, since this was the most commonly-reported level. MAIN RESULTS We included 106 studies in this review, which analysed 16,260 eyes (8353 cases, 7907 controls) in total. Forty studies (5574 participants) assessed GDx, 18 studies (3550 participants) HRT, and 63 (9390 participants) OCT, with 12 of these studies comparing two or three tests. Regarding study quality, a case-control design in 103 studies raised concerns as it can overestimate accuracy and reduce the applicability of the results to daily practice. Twenty-four studies were sponsored by the manufacturer, and in 15 the potential conflict of interest was unclear.Comparisons made within each test were more reliable than those between tests, as they were mostly based on direct comparisons within each study.The Nerve Fibre Indicator yielded the highest accuracy (estimate, 95% confidence interval (CI)) among GDx parameters (sensitivity: 0.67, 0.55 to 0.77; specificity: 0.94, 0.92 to 0.95). For HRT measures, the Vertical Cup/Disc (C/D) ratio (sensitivity: 0.72, 0.60 to 0.68; specificity: 0.94, 0.92 to 0.95) was no different from other parameters. With OCT, the accuracy of average RNFL retinal thickness was similar to the inferior sector (0.72, 0.65 to 0.77; specificity: 0.93, 0.92 to 0.95) and, in different studies, to the vertical C/D ratio.Comparing the parameters with the highest diagnostic odds ratio (DOR) for each device in a single HSROC model, the performance of GDx, HRT and OCT was remarkably similar. At a sensitivity of 0.70 and a high specificity close to 0.95 as in most of these studies, in 1000 people referred by primary eye care, of whom 200 have manifest glaucoma, such as in those who have already undergone some functional or anatomic testing by optometrists, the best measures of GDx, HRT and OCT would miss about 60 cases out of the 200 patients with glaucoma, and would incorrectly refer 50 out of 800 patients without glaucoma. If prevalence were 5%, e.g. such as in people referred only because of family history of glaucoma, the corresponding figures would be 15 patients missed out of 50 with manifest glaucoma, avoiding referral of about 890 out of 950 non-glaucomatous people.Heterogeneity investigations found that sensitivity estimate was higher for studies with more severe glaucoma, expressed as worse average mean deviation (MD): 0.79 (0.74 to 0.83) for MD < -6 db versus 0.64 (0.60 to 0.69) for MD ≥ -6 db, at a similar summary specificity (0.93, 95% CI 0.92 to 0.94 and, respectively, 0.94; 95% CI 0.93 to 0.95; P < 0.0001 for the difference in relative DOR). AUTHORS' CONCLUSIONS The accuracy of imaging tests for detecting manifest glaucoma was variable across studies, but overall similar for different devices. Accuracy may have been overestimated due to the case-control design, which is a serious limitation of the current evidence base.We recommend that further diagnostic accuracy studies are carried out on patients selected consecutively at a defined step of the clinical pathway, providing a description of risk factors leading to referral and bearing in mind the consequences of false positives and false negatives in the setting in which the diagnostic question is made. Future research should report accuracy for each threshold of these continuous measures, or publish raw data.
Collapse
Affiliation(s)
- Manuele Michelessi
- Ophthalmology, Fondazione G.B. Bietti per lo studio e la ricerca in Oftalmolologia-IRCCS, Via Livenza n 3, Rome, Italy, 00198
| | | | | | | | | | | | | | | |
Collapse
|
19
|
Evaluation of vermillion border descriptors and relevance vector machines discrimination model for making probabilistic predictions of solar cheilosis on digital lip photographs. Comput Biol Med 2015; 63:11-8. [DOI: 10.1016/j.compbiomed.2015.04.024] [Citation(s) in RCA: 6] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/17/2015] [Revised: 04/07/2015] [Accepted: 04/15/2015] [Indexed: 11/22/2022]
|
20
|
Yousefi S, Goldbaum MH, Balasubramanian M, Jung TP, Weinreb RN, Medeiros FA, Zangwill LM, Liebmann JM, Girkin CA, Bowd C. Glaucoma progression detection using structural retinal nerve fiber layer measurements and functional visual field points. IEEE Trans Biomed Eng 2014; 61:1143-54. [PMID: 24658239 DOI: 10.1109/tbme.2013.2295605] [Citation(s) in RCA: 58] [Impact Index Per Article: 5.8] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/17/2022]
Abstract
Machine learning classifiers were employed to detect glaucomatous progression using longitudinal series of structural data extracted from retinal nerve fiber layer thickness measurements and visual functional data recorded from standard automated perimetry tests. Using the collected data, a longitudinal feature vector was created for each patient's eye by computing the norm 1 difference vector of the data at the baseline and at each follow-up visit. The longitudinal features from each patient's eye were then fed to the machine learning classifier to classify each eye as stable or progressed over time. This study was performed using several machine learning classifiers including Bayesian, Lazy, Meta, and Tree, composing different families. Combinations of structural and functional features were selected and ranked to determine the relative effectiveness of each feature. Finally, the outcomes of the classifiers were assessed by several performance metrics and the effectiveness of structural and functional features were analyzed.
Collapse
|
21
|
Combining information from 3 anatomic regions in the diagnosis of glaucoma with time-domain optical coherence tomography. J Glaucoma 2014; 23:129-35. [PMID: 22828002 DOI: 10.1097/ijg.0b013e318264b941] [Citation(s) in RCA: 15] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/26/2022]
Abstract
PURPOSE To improve the diagnosis of glaucoma by combining time-domain optical coherence tomography (TD-OCT) measurements of the optic disc, circumpapillary retinal nerve fiber layer (RNFL), and macular retinal thickness. PATIENTS AND METHODS Ninety-six age-matched normal and 96 perimetric glaucoma participants were included in this observational, cross-sectional study. Or-logic, support vector machine, relevance vector machine, and linear discrimination function were used to analyze the performances of combined TD-OCT diagnostic variables. RESULTS The area under the receiver-operating curve (AROC) was used to evaluate the diagnostic accuracy and to compare the diagnostic performance of single and combined anatomic variables. The best RNFL thickness variables were the inferior (AROC=0.900), overall (AROC=0.892), and superior quadrants (AROC=0.850). The best optic disc variables were horizontal integrated rim width (AROC=0.909), vertical integrated rim area (AROC=0.908), and cup/disc vertical ratio (AROC=0.890). All macular retinal thickness variables had AROCs of 0.829 or less. Combining the top 3 RNFL and optic disc variables in optimizing glaucoma diagnosis, support vector machine had the highest AROC, 0.954, followed by or-logic (AROC=0.946), linear discrimination function (AROC=0.946), and relevance vector machine (AROC=0.943). All combination diagnostic variables had significantly larger AROCs than any single diagnostic variable. There are no significant differences among the combination diagnostic indices. CONCLUSIONS With TD-OCT, RNFL and optic disc variables had better diagnostic accuracy than macular retinal variables. Combining top RNFL and optic disc variables significantly improved diagnostic performance. Clinically, or-logic classification was the most practical analytical tool with sufficient accuracy to diagnose early glaucoma.
Collapse
|
22
|
Bowd C, Weinreb RN, Balasubramanian M, Lee I, Jang G, Yousefi S, Zangwill LM, Medeiros FA, Girkin CA, Liebmann JM, Goldbaum MH. Glaucomatous patterns in Frequency Doubling Technology (FDT) perimetry data identified by unsupervised machine learning classifiers. PLoS One 2014; 9:e85941. [PMID: 24497932 PMCID: PMC3907565 DOI: 10.1371/journal.pone.0085941] [Citation(s) in RCA: 30] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/13/2013] [Accepted: 12/04/2013] [Indexed: 12/12/2022] Open
Abstract
Purpose The variational Bayesian independent component analysis-mixture model (VIM), an unsupervised machine-learning classifier, was used to automatically separate Matrix Frequency Doubling Technology (FDT) perimetry data into clusters of healthy and glaucomatous eyes, and to identify axes representing statistically independent patterns of defect in the glaucoma clusters. Methods FDT measurements were obtained from 1,190 eyes with normal FDT results and 786 eyes with abnormal FDT results from the UCSD-based Diagnostic Innovations in Glaucoma Study (DIGS) and African Descent and Glaucoma Evaluation Study (ADAGES). For all eyes, VIM input was 52 threshold test points from the 24-2 test pattern, plus age. Results FDT mean deviation was −1.00 dB (S.D. = 2.80 dB) and −5.57 dB (S.D. = 5.09 dB) in FDT-normal eyes and FDT-abnormal eyes, respectively (p<0.001). VIM identified meaningful clusters of FDT data and positioned a set of statistically independent axes through the mean of each cluster. The optimal VIM model separated the FDT fields into 3 clusters. Cluster N contained primarily normal fields (1109/1190, specificity 93.1%) and clusters G1 and G2 combined, contained primarily abnormal fields (651/786, sensitivity 82.8%). For clusters G1 and G2 the optimal number of axes were 2 and 5, respectively. Patterns automatically generated along axes within the glaucoma clusters were similar to those known to be indicative of glaucoma. Fields located farther from the normal mean on each glaucoma axis showed increasing field defect severity. Conclusions VIM successfully separated FDT fields from healthy and glaucoma eyes without a priori information about class membership, and identified familiar glaucomatous patterns of loss.
Collapse
Affiliation(s)
- Christopher Bowd
- Hamilton Glaucoma Center, Department of Ophthalmology, University of California, San Diego, La Jolla, California, United States of America
- * E-mail:
| | - Robert N. Weinreb
- Hamilton Glaucoma Center, Department of Ophthalmology, University of California, San Diego, La Jolla, California, United States of America
| | - Madhusudhanan Balasubramanian
- Hamilton Glaucoma Center, Department of Ophthalmology, University of California, San Diego, La Jolla, California, United States of America
| | - Intae Lee
- Hamilton Glaucoma Center, Department of Ophthalmology, University of California, San Diego, La Jolla, California, United States of America
| | - Giljin Jang
- Hamilton Glaucoma Center, Department of Ophthalmology, University of California, San Diego, La Jolla, California, United States of America
- School of Electrical and Computer Engineering, Ulsan National Institute of Science and Technology, Ulsan, South Korea
| | - Siamak Yousefi
- Hamilton Glaucoma Center, Department of Ophthalmology, University of California, San Diego, La Jolla, California, United States of America
| | - Linda M. Zangwill
- Hamilton Glaucoma Center, Department of Ophthalmology, University of California, San Diego, La Jolla, California, United States of America
| | - Felipe A. Medeiros
- Hamilton Glaucoma Center, Department of Ophthalmology, University of California, San Diego, La Jolla, California, United States of America
| | - Christopher A. Girkin
- Department of Ophthalmology, University of Alabama at Birmingham, Birmingham, Alabama, United States of America
| | - Jeffrey M. Liebmann
- Department of Ophthalmology, New York University School of Medicine, New York, New York, United States of America
- New York Eye and Ear Infirmary, New York, New York, United States of America
| | - Michael H. Goldbaum
- Hamilton Glaucoma Center, Department of Ophthalmology, University of California, San Diego, La Jolla, California, United States of America
| |
Collapse
|
23
|
An optimized short-term wind power interval prediction method considering NWP accuracy. CHINESE SCIENCE BULLETIN-CHINESE 2014. [DOI: 10.1007/s11434-014-0119-7] [Citation(s) in RCA: 7] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/25/2022]
|
24
|
Glaucoma Diagnostic Accuracy of Machine Learning Classifiers Using Retinal Nerve Fiber Layer and Optic Nerve Data from SD-OCT. J Ophthalmol 2013; 2013:789129. [PMID: 24369495 PMCID: PMC3863536 DOI: 10.1155/2013/789129] [Citation(s) in RCA: 36] [Impact Index Per Article: 3.3] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/29/2013] [Accepted: 10/13/2013] [Indexed: 12/15/2022] Open
Abstract
Purpose. To investigate the diagnostic accuracy of machine learning classifiers (MLCs) using retinal nerve fiber layer (RNFL) and optic nerve (ON) parameters obtained with spectral domain optical coherence tomography (SD-OCT). Methods. Fifty-seven patients with early to moderate primary open angle glaucoma and 46 healthy patients were recruited. All 103 patients underwent a complete ophthalmological examination, achromatic standard automated perimetry, and imaging with SD-OCT. Receiver operating characteristic (ROC) curves were built for RNFL and ON parameters. Ten MLCs were tested. Areas under ROC curves (aROCs) obtained for each SD-OCT parameter and MLC were compared. Results. The mean age was 56.5 ± 8.9 years for healthy individuals and 59.9 ± 9.0 years for glaucoma patients (P = 0.054). Mean deviation values were −1.4 dB for healthy individuals and −4.0 dB for glaucoma patients (P < 0.001). SD-OCT parameters with the greatest aROCs were cup/disc area ratio (0.846) and average cup/disc (0.843). aROCs obtained with classifiers varied from 0.687 (CTREE) to 0.877 (RAN). The aROC obtained with RAN (0.877) was not significantly different from the aROC obtained with the best single SD-OCT parameter (0.846) (P = 0.542). Conclusion. MLCs showed good accuracy but did not improve the sensitivity and specificity of SD-OCT for the diagnosis of glaucoma.
Collapse
|
25
|
Theeraworn C, Kongprawechnon W, Kondo T, Bunnun P, Nishihara A, Manassakorn A. Automatic screening of narrow anterior chamber angle and angle-closure glaucoma based on slit-lamp image analysis by using support vector machine. ANNUAL INTERNATIONAL CONFERENCE OF THE IEEE ENGINEERING IN MEDICINE AND BIOLOGY SOCIETY. IEEE ENGINEERING IN MEDICINE AND BIOLOGY SOCIETY. ANNUAL INTERNATIONAL CONFERENCE 2013; 2013:5887-90. [PMID: 24111078 DOI: 10.1109/embc.2013.6610891] [Citation(s) in RCA: 4] [Impact Index Per Article: 0.4] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 11/09/2022]
Abstract
At present, Van Herick's method is a standard technique used to screen a Narrow Anterior Chamber Angle (NACA) and Angle-Closure Glaucoma (ACG). It can identify a patient who suffers from NACA and ACG by considering the width of peripheral anterior chamber depth (PACD) and corneal thickness. However, the screening result of this method often varies among ophthalmologists. So, an automatic screening of NACA and ACG based on slit-lamp image analysis by using Support Vector Machine (SVM) is proposed. SVM can automatically generate the classification model, which is used to classify the result as an angle-closure likely or an angle-closure unlikely. It shows that it can improve the accuracy of the screening result. To develop the classification model, the width of PACD and corneal thickness from many positions are measured and selected to be features. A statistic analysis is also used in the PACD and corneal thickness estimation in order to reduce the error from reflection on the cornea. In this study, it is found that the generated models are evaluated by using 5-fold cross validation and give a better result than the result classified by Van Herick's method.
Collapse
|
26
|
Vidotti VG, Costa VP, Silva FR, Resende GM, Cremasco F, Dias M, Gomi ES. Sensitivity and specificity of machine learning classifiers and spectral domain OCT for the diagnosis of glaucoma. Eur J Ophthalmol 2012; 23:0. [PMID: 22729440 DOI: 10.5301/ejo.5000183] [Citation(s) in RCA: 11] [Impact Index Per Article: 0.9] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Accepted: 06/04/2012] [Indexed: 01/30/2023]
Abstract
Purpose. To investigate the sensitivity and specificity of machine learning classifiers (MLC) and spectral domain optical coherence tomography (SD-OCT) for the diagnosis of glaucoma. Methods. Sixty-two patients with early to moderate glaucomatous visual field damage and 48 healthy individuals were included. All subjects underwent a complete ophthalmologic examination, achromatic standard automated perimetry, and RNFL imaging with SD-OCT (Cirrus HD-OCT; Carl Zeiss Meditec, Inc., Dublin, California, USA). Receiver operating characteristic (ROC) curves were obtained for all SD-OCT parameters. Subsequently, the following MLCs were tested: Classification Tree (CTREE), Random Forest (RAN), Bagging (BAG), AdaBoost M1 (ADA), Ensemble Selection (ENS), Multilayer Perceptron (MLP), Radial Basis Function (RBF), Naive-Bayes (NB), and Support Vector Machine (SVM). Areas under the ROC curves (aROCs) obtained for each parameter and each MLC were compared. Results. The mean age was 57.0±9.2 years for healthy individuals and 59.9±9.0 years for glaucoma patients (p=0.103). Mean deviation values were -4.1±2.4 dB for glaucoma patients and -1.5±1.6 dB for healthy individuals (p<0.001). The SD-OCT parameters with the greater aROCs were inferior quadrant (0.813), average thickness (0.807), 7 o'clock position (0.765), and 6 o'clock position (0.754). The aROCs from classifiers varied from 0.785 (ADA) to 0.818 (BAG). The aROC obtained with BAG was not significantly different from the aROC obtained with the best single SD-OCT parameter (p=0.93). Conclusions. The SD-OCT showed good diagnostic accuracy in a group of patients with early glaucoma. In this series, MLCs did not improve the sensitivity and specificity of SD-OCT for the diagnosis of glaucoma.
Collapse
Affiliation(s)
- Vanessa G Vidotti
- Glaucoma Service, Department of Ophthalmology, University of Campinas, Campinas - Brazil
| | | | | | | | | | | | | |
Collapse
|
27
|
Bowd C, Lee I, Goldbaum MH, Balasubramanian M, Medeiros FA, Zangwill LM, Girkin CA, Liebmann JM, Weinreb RN. Predicting glaucomatous progression in glaucoma suspect eyes using relevance vector machine classifiers for combined structural and functional measurements. Invest Ophthalmol Vis Sci 2012; 53:2382-9. [PMID: 22427577 DOI: 10.1167/iovs.11-7951] [Citation(s) in RCA: 27] [Impact Index Per Article: 2.3] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/24/2022] Open
Abstract
PURPOSE The goal of this study was to determine if glaucomatous progression in suspect eyes can be predicted from baseline confocal scanning laser ophthalmoscope (CSLO) and standard automated perimetry (SAP) measurements analyzed with relevance vector machine (RVM) classifiers. METHODS Two hundred sixty-four eyes of 193 participants were included. All eyes had normal SAP results at baseline with five or more SAP tests over time. Eyes were labeled progressed (n = 47) or stable (n = 217) during follow-up based on SAP Guided Progression Analysis or serial stereophotograph assessment. Baseline CSLO-measured topographic parameters (n = 117) and baseline total deviation values from the 24-2 SAP test-grid (n = 52) were selected from each eye. Ten-fold cross-validation was used to train and test RVMs using the CSLO and SAP features. Receiver operating characteristic (ROC) curve areas were calculated using full and optimized feature sets. ROC curve results from RVM analyses of CSLO, SAP, and CSLO and SAP combined were compared to CSLO and SAP global indices (Glaucoma Probability Score, mean deviation and pattern standard deviation). RESULTS The areas under the ROC curves (AUROCs) for RVMs trained on optimized feature sets of CSLO parameters, SAP parameters, and CSLO and SAP parameters combined were 0.640, 0.762, and 0.805, respectively. AUROCs for CSLO Glaucoma Probability Score, SAP mean deviation (MD), and SAP pattern standard deviation (PSD) were 0.517, 0.513, and 0.620, respectively. No CSLO or SAP global indices discriminated between baseline measurements from progressed and stable eyes better than chance. CONCLUSIONS In our sample, RVM analyses of baseline CSLO and SAP measurements could identify eyes that showed future glaucomatous progression with a higher accuracy than the CSLO and SAP global indices. (ClinicalTrials.gov numbers, NCT00221897, NCT00221923.).
Collapse
Affiliation(s)
- Christopher Bowd
- Hamilton Glaucoma Center, Department of Ophthalmology, University of California, San Diego, La Jolla, CA 92037-0946, USA.
| | | | | | | | | | | | | | | | | |
Collapse
|
28
|
A Two Stage Sequential Ensemble Applied to the Classification of Alzheimer’s Disease Based on MRI Features. Neural Process Lett 2011. [DOI: 10.1007/s11063-011-9200-2] [Citation(s) in RCA: 17] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/17/2022]
|
29
|
Combining functional and structural tests improves the diagnostic accuracy of relevance vector machine classifiers. J Glaucoma 2010; 19:167-75. [PMID: 19528827 DOI: 10.1097/ijg.0b013e3181a98b85] [Citation(s) in RCA: 21] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/31/2022]
Abstract
PURPOSE To investigate whether combining optic disc topography and short-wavelength automated perimetry (SWAP) data improves the diagnostic accuracy of relevance vector machine (RVM) classifiers for detecting glaucomatous eyes compared with using each test alone. METHODS One eye of 144 glaucoma patients and 68 healthy controls from the Diagnostic Innovations in Glaucoma Study were included. RVM were trained and tested with cross-validation on optimized (backward elimination) SWAP features (thresholds plus age; pattern deviation; and total deviation) and on Heidelberg retina tomograph II (HRT) optic disc topography features, independently and in combination. RVM performance was also compared with 2 HRT linear discriminant functions and to SWAP mean deviation and pattern standard deviation. Classifier performance was measured by the area under the receiver operating characteristic curves (AUROCs) generated for each feature set and by the sensitivities at set specificities of 75%, 90%, and 96%. RESULTS RVM trained on combined HRT and SWAP thresholds plus age had significantly higher AUROC (0.93) than RVM trained on HRT (0.88) and SWAP (0.76) alone. AUROCs for the SWAP global indices (mean deviation: 0.68; pattern standard deviation: 0.72) offered no advantage over SWAP thresholds plus age, whereas the linear discriminant functions AUROCs were significantly lower than RVM trained on the combined SWAP and HRT feature set and on HRT alone feature set. CONCLUSIONS Training RVM on combined optimized HRT and SWAP data improved diagnostic accuracy compared with training on SWAP and HRT parameters alone. Future research may identify other combinations of tests and classifiers that can also improve diagnostic accuracy.
Collapse
|
30
|
Goldbaum MH, Kozak I, Hao J, Sample PA, Lee T, Grant I, Freeman WR. Pattern recognition can detect subtle field defects in eyes of HIV individuals without retinitis under HAART. Graefes Arch Clin Exp Ophthalmol 2010; 249:491-8. [PMID: 20865422 PMCID: PMC3070878 DOI: 10.1007/s00417-010-1511-x] [Citation(s) in RCA: 8] [Impact Index Per Article: 0.6] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/25/2010] [Accepted: 09/01/2010] [Indexed: 12/31/2022] Open
Abstract
Objectives To use machine learning classifiers (MLCs) to seek differences in visual fields (VFs) between normal eyes and eyes of HIV+ patients; to find the effect of immunodeficiency on VFs and to compare the effectiveness of MLCs to commonly-used Statpac global indices in analyzing standard automated perimetry (SAP). Methods The high CD4 group consisted of 70 eyes of 39 HIV-positive patients with good immune status (CD4 counts were never <100/ml). The low CD4 group had 59 eyes of 38 HIV-positive patients with CD4 cell counts <100/ml at some period of time lasting for at least 6 months. The normal group consisted of 61 eyes of 52 HIV-negative individuals. We used a Humphrey Visual Field Analyzer, SAP full threshold program 24-2, and routine settings for evaluating VFs. We trained and tested support vector machine (SVM) machine learning classifiers to distinguish fields from normal subjects and high and CD4 groups separately. Receiver operating characteristic (ROC) curves measured the discrimination of each classifier, and areas under ROC were statistically compared. Results Low CD4 HIV patients: with SVM, the AUROC was 0.790 ± 0.042. SVM and MD each significantly differed from chance decision, with p < .00005. High CD4 HIV patients: the SVM AUROC of 0.664 ± 0.047 and MD were each significantly better than chance (p = .041, p = .05 respectively). Conclusions Eyes from both low and high CD4 HIV+ patients have VFs defects indicating retinal damage. Generalized learning classifier, SVM, and a Statpac classifier, MD, are effective at detecting HIV eyes that have field defects, even when these defects are subtle.
Collapse
Affiliation(s)
- Michael H. Goldbaum
- Jacobs Retina Center at the Shiley Eye Center, University of California San Diego, 9415 Campus Point Dr., 0946, La Jolla, CA 92093 USA
- Hamilton Glaucoma Center at the Shiley Eye Center, University of California San Diego, La Jolla, CA USA
- Institute for Neural Computation, University of California San Diego, La Jolla, CA USA
| | - Igor Kozak
- Jacobs Retina Center at the Shiley Eye Center, University of California San Diego, 9415 Campus Point Dr., 0946, La Jolla, CA 92093 USA
| | - Jiucang Hao
- Institute for Neural Computation, University of California San Diego, La Jolla, CA USA
| | - Pamela A. Sample
- Hamilton Glaucoma Center at the Shiley Eye Center, University of California San Diego, La Jolla, CA USA
| | - TeWon Lee
- Institute for Neural Computation, University of California San Diego, La Jolla, CA USA
| | - Igor Grant
- HIV Neurobehavioral Center, University of California San Diego, La Jolla, CA USA
| | - William R. Freeman
- Jacobs Retina Center at the Shiley Eye Center, University of California San Diego, 9415 Campus Point Dr., 0946, La Jolla, CA 92093 USA
| |
Collapse
|
31
|
Hamzah JC, Azuara-Blanco A. What is the best method for diagnosing glaucoma? EXPERT REVIEW OF OPHTHALMOLOGY 2010. [DOI: 10.1586/eop.10.33] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 11/08/2022]
|
32
|
Wu CC, Asgharzadeh S, Triche TJ, D'Argenio DZ. Prediction of human functional genetic networks from heterogeneous data using RVM-based ensemble learning. ACTA ACUST UNITED AC 2010; 26:807-13. [PMID: 20134029 DOI: 10.1093/bioinformatics/btq044] [Citation(s) in RCA: 23] [Impact Index Per Article: 1.6] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/12/2022]
Abstract
MOTIVATION Three major problems confront the construction of a human genetic network from heterogeneous genomics data using kernel-based approaches: definition of a robust gold-standard negative set, large-scale learning and massive missing data values. RESULTS The proposed graph-based approach generates a robust GSN for the training process of genetic network construction. The RVM-based ensemble model that combines AdaBoost and reduced-feature yields improved performance on large-scale learning problems with massive missing values in comparison to Naïve Bayes. CONTACT dargenio@bmsr.usc.edu SUPPLEMENTARY INFORMATION Supplementary material is available at Bioinformatics online.
Collapse
Affiliation(s)
- Chia-Chin Wu
- Department of Biomedical Engineering, University of Southern California, Los Angeles, 90089, USA
| | | | | | | |
Collapse
|
33
|
Bizios D, Heijl A, Hougaard JL, Bengtsson B. Machine learning classifiers for glaucoma diagnosis based on classification of retinal nerve fibre layer thickness parameters measured by Stratus OCT. Acta Ophthalmol 2010; 88:44-52. [PMID: 20064122 DOI: 10.1111/j.1755-3768.2009.01784.x] [Citation(s) in RCA: 54] [Impact Index Per Article: 3.9] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 02/01/2023]
Abstract
PURPOSE To compare the performance of two machine learning classifiers (MLCs), artificial neural networks (ANNs) and support vector machines (SVMs), with input based on retinal nerve fibre layer thickness (RNFLT) measurements by optical coherence tomography (OCT), on the diagnosis of glaucoma, and to assess the effects of different input parameters. METHODS We analysed Stratus OCT data from 90 healthy persons and 62 glaucoma patients. Performance of MLCs was compared using conventional OCT RNFLT parameters plus novel parameters such as minimum RNFLT values, 10th and 90th percentiles of measured RNFLT, and transformations of A-scan measurements. For each input parameter and MLC, the area under the receiver operating characteristic curve (AROC) was calculated. RESULTS There were no statistically significant differences between ANNs and SVMs. The best AROCs for both ANN (0.982, 95%CI: 0.966-0.999) and SVM (0.989, 95% CI: 0.979-1.0) were based on input of transformed A-scan measurements. Our SVM trained on this input performed better than ANNs or SVMs trained on any of the single RNFLT parameters (p < or = 0.038). The performance of ANNs and SVMs trained on minimum thickness values and the 10th and 90th percentiles were at least as good as ANNs and SVMs with input based on the conventional RNFLT parameters. CONCLUSION No differences between ANN and SVM were observed in this study. Both MLCs performed very well, with similar diagnostic performance. Input parameters have a larger impact on diagnostic performance than the type of machine classifier. Our results suggest that parameters based on transformed A-scan thickness measurements of the RNFL processed by machine classifiers can improve OCT-based glaucoma diagnosis.
Collapse
Affiliation(s)
- Dimitrios Bizios
- Department of Clinical Sciences, Ophthalmology, Malmö University Hospital, Lund University, Malmoe, Sweden.
| | | | | | | |
Collapse
|
34
|
|
35
|
Souza MB, Medeiros FW, Souza DB, Garcia R, Alves MR. Evaluation of machine learning classifiers in keratoconus detection from orbscan II examinations. Clinics (Sao Paulo) 2010; 65:1223-8. [PMID: 21340208 PMCID: PMC3020330 DOI: 10.1590/s1807-59322010001200002] [Citation(s) in RCA: 55] [Impact Index Per Article: 3.9] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 06/27/2010] [Revised: 07/27/2010] [Accepted: 09/02/2010] [Indexed: 02/05/2023] Open
Abstract
PURPOSE To evaluate the performance of support vector machine, multi-layer perceptron and radial basis function neural network as auxiliary tools to identify keratoconus from Orbscan II maps. METHODS A total of 318 maps were selected and classified into four categories: normal (n = 172), astigmatism (n = 89), keratoconus (n = 46) and photorefractive keratectomy (n = 11). For each map, 11 attributes were obtained or calculated from data provided by the Orbscan II. Ten-fold cross-validation was used to train and test the classifiers. Besides accuracy, sensitivity and specificity, receiver operating characteristic (ROC) curves for each classifier were generated, and the areas under the curves were calculated. RESULTS The three selected classifiers provided a good performance, and there were no differences between their performances. The area under the ROC curve of the support vector machine, multi-layer perceptron and radial basis function neural network were significantly larger than those for all individual Orbscan II attributes evaluated (p < 0.05). CONCLUSION Overall, the results suggest that using a support vector machine, multi-layer perceptron classifiers and radial basis function neural network, these classifiers, trained on Orbscan II data, could represent useful techniques for keratoconus detection.
Collapse
|
36
|
Huang ML, Chen HY, Huang WC, Tsai YY. Linear discriminant analysis and artificial neural network for glaucoma diagnosis using scanning laser polarimetry–variable cornea compensation measurements in Taiwan Chinese population. Graefes Arch Clin Exp Ophthalmol 2009; 248:435-41. [DOI: 10.1007/s00417-009-1259-3] [Citation(s) in RCA: 19] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/22/2009] [Revised: 11/10/2009] [Accepted: 11/19/2009] [Indexed: 11/29/2022] Open
|
37
|
Comparison of shape-based analysis of retinal nerve fiber layer data obtained From OCT and GDx-VCC. J Glaucoma 2009; 18:464-71. [PMID: 19680055 DOI: 10.1097/ijg.0b013e31818c6f2b] [Citation(s) in RCA: 4] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/09/2023]
Abstract
PURPOSE To directly compare in 1 population: (1) the performance of Optical Coherence Tomograph (OCT) and GDx-Variable Corneal Compensator (VCC) when using Wavelet-Fourier Analysis (WFA) and Fast-Fourier Analysis (FFA), (2) the performance of these shape-based and standard metrics, and (3) the shape of the retinal nerve fiber layer (RNFL) temporal, superior, nasal, inferior, temporal (TSNIT) curves obtained by the 2 different devices. METHODS RNFL estimates were obtained from 136 eyes of 136 individuals (73 healthy and 63 mild glaucoma). WFA and FFA with and without asymmetry measures were performed on the TSNIT RNFL estimates to identify glaucoma from healthy eyes. Performance of WFA, FFA, and the standard metrics of OCT (Inferior Average) and GDX-VCC (Nerve Fiber Indicator) was evaluated by calculating receiver operating characteristic area. Measurements were obtained at a custom radius (33 to 41 pixels) for GDx-VCC to match the OCT radius (1.73 mm). RESULTS WFA and FFA shape analysis significantly improved performance of both OCT (0.937) and GDx-VCC (0.913) compared with Inferior Average and Nerve Fiber Indicator (0.852 and 0.833, respectively). With either shape-based or standard metrics, OCT performance was slightly, but not significantly, better than GDx-VCC performance. Comparison of RNFL curves revealed that the GDx-VCC curves were more jagged and the peaks shifted more nasally when compared with the OCT RNFL curves. CONCLUSIONS Performance of both OCT and GDx-VCC devices are improved by shape-based analysis methods. Classification performance was greater when using WFA for the OCT, and greater with FFA for the GDx-VCC. Significant differences between the machines exist in the measured TSNIT thicknesses, possibly because of GDx-VCC's measurements being affected by polarization magnitude varying with angle.
Collapse
|
38
|
Huang ML, Chen HY. Glaucoma Classification Model Based on GDx VCC Measured Parameters by Decision Tree. J Med Syst 2009; 34:1141-7. [DOI: 10.1007/s10916-009-9333-2] [Citation(s) in RCA: 8] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/26/2009] [Accepted: 06/11/2009] [Indexed: 11/28/2022]
|
39
|
Sharma P, Sample PA, Zangwill LM, Schuman JS. Diagnostic tools for glaucoma detection and management. Surv Ophthalmol 2009; 53 Suppl1:S17-32. [PMID: 19038620 DOI: 10.1016/j.survophthal.2008.08.003] [Citation(s) in RCA: 119] [Impact Index Per Article: 7.9] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/25/2022]
Abstract
Early diagnosis of glaucoma is critical to prevent permanent structural damage and irreversible vision loss. Detection of glaucoma typically relies on examination of structural damage to the optic nerve combined with measurements of visual function. To aid the clinician in evaluation of visual function and structure, computer-based devices such as confocal scanning laser ophthalmoscopy, scanning laser polarimetry, and optical coherence tomography provide quantitative assessments of structural damage, and visual function testing includes standard automated perimetry as well as selective techniques, including short-wavelength automated perimetry and frequency-doubling technology perimetry are available. This article will review current literature on diagnostic modalities available for glaucoma with emphasis on the best evidence available in the literature to support their use in clinical practice.
Collapse
Affiliation(s)
- Pooja Sharma
- Drexel University College of Medicine, Philadelphia, Pennsylvania, USA
| | | | | | | |
Collapse
|
40
|
SBMDS: an interpretable string based malware detection system using SVM ensemble with bagging. ACTA ACUST UNITED AC 2008. [DOI: 10.1007/s11416-008-0108-y] [Citation(s) in RCA: 56] [Impact Index Per Article: 3.5] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/29/2022]
|
41
|
|
42
|
Ferreras A, Pablo LE, Pajarín AB, García-Feijoo J, Honrubia FM. Scanning laser polarimetry: logistic regression analysis for perimetric glaucoma diagnosis. Eye (Lond) 2008; 23:593-600. [DOI: 10.1038/eye.2008.50] [Citation(s) in RCA: 5] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/09/2022] Open
|
43
|
Boden C, Chan K, Sample PA, Hao J, Lee TW, Zangwill LM, Weinreb RN, Goldbaum MH. Assessing visual field clustering schemes using machine learning classifiers in standard perimetry. Invest Ophthalmol Vis Sci 2008; 48:5582-90. [PMID: 18055807 DOI: 10.1167/iovs.06-0897] [Citation(s) in RCA: 10] [Impact Index Per Article: 0.6] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/23/2022] Open
Abstract
PURPOSE To compare machine learning classifiers trained on three clustering schemes to determine whether distinguishing healthy eyes from those with glaucomatous optic neuropathy (GON) can be optimized by training with clustered data. METHODS Two machine learning classifiers-quadratic discriminant analysis (QDA) and support vector machines with Gaussian kernel (SVMg)-were trained separately using standard perimetry data from the Diagnostic Innovations in Glaucoma Study (DIGS), clustered using three clustering schemes on a training data set (123 eyes/123 glaucoma patients with GON; 135 eyes/135 normal control subjects). Trained classifiers were then applied to an independent data set containing 69 eyes of 69 glaucoma patients with early visual field loss and 83 eyes of 83 normal control subjects. Two control conditions were included: unclustered data and a random assignment of locations to clusters. RESULTS Areas under the receiver operating characteristic (ROC) curve ranged from 0.85 (SVMg, thresholds clustered by Glaucoma Hemifield Test sectors) to 0.92 (QDA, thresholds clustered by Garway-Heath mapping) for the training data set. Use of clustered data showed no significant optimization of sensitivity over use of unclustered data, and no single clustering method resulted in significantly higher performance in the independent data set. Sensitivities tended to be higher with QDA than with SVMg, regardless of specificity cutoff and clustering METHOD CONCLUSIONS QDA performed better with the early glaucoma data set than did the SVMg. Clustering may be advantageous when data-dimension reduction is needed-for example, when combining field results with other high-dimensional data (e.g., structural imaging data)-but it is not necessary for visual field data alone.
Collapse
Affiliation(s)
- Catherine Boden
- Department of Ophthalmology, University of California, San Diego, La Jolla, CA 92093-0946, USA
| | | | | | | | | | | | | | | |
Collapse
|
44
|
Medeiros FA, Vizzeri G, Zangwill LM, Alencar LM, Sample PA, Weinreb RN. Comparison of retinal nerve fiber layer and optic disc imaging for diagnosing glaucoma in patients suspected of having the disease. Ophthalmology 2008; 115:1340-6. [PMID: 18207246 DOI: 10.1016/j.ophtha.2007.11.008] [Citation(s) in RCA: 78] [Impact Index Per Article: 4.9] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/11/2007] [Revised: 11/07/2007] [Accepted: 11/07/2007] [Indexed: 11/25/2022] Open
Abstract
PURPOSE To compare retinal nerve fiber layer (RNFL) and optic disc topographic imaging for detection of optic nerve damage in patients suspected of having glaucoma. DESIGN Observational cohort study. PARTICIPANTS A cohort of 82 patients suspected of having glaucoma based on the appearance of the optic nerve. METHODS All patients were imaged using the GDx VCC scanning laser polarimeter and HRT (software version 3.0) confocal scanning laser ophthalmoscope. All patients had normal standard automated perimetry visual fields at the time of imaging and were classified based on history of documented stereophotographic evidence of progressive glaucomatous change in the appearance of the optic nerve occurring before the imaging sessions. MAIN OUTCOME MEASURES Areas under the receiver operating characteristic (ROC) curves were used to evaluate the diagnostic accuracies of GDx VCC and the HRT. RESULTS Forty eyes with progressive glaucomatous optic nerve change were included in the glaucoma group, and 42 eyes without any evidence of progressive damage to the optic nerve followed untreated for an average time of 8.97+/-3.08 years were included in the normal group. The area under the ROC curve for the best parameter from GDx VCC (nerve fiber indicator [NFI]) was significantly larger than that of the best parameter from the HRT (rim volume) (0.83 vs. 0.70; P = 0.044). The NFI parameter also had a larger ROC curve area than that of the contour line-independent parameter glaucoma probability score (0.83 vs. 0.68; P = 0.023). Assuming borderline results as normal, the Moorfields regression analysis classification had a sensitivity of 48% for specificity of 69%. For a similar specificity (70%), the parameter NFI had a significantly larger sensitivity (83%) (P = 0.003). CONCLUSIONS Retinal nerve fiber layer imaging with GDx VCC had a superior performance versus topographic optic disc assessment with the HRT for detecting early damage in patients suspected of having glaucoma. For glaucoma diagnosis, these results suggest that GDx VCC may offer advantage over the HRT when these tests are combined with clinical examination of the optic nerve.
Collapse
Affiliation(s)
- Felipe A Medeiros
- Hamilton Glaucoma Center and Department of Ophthalmology, University of California, San Diego, La Jolla, CA 92093-0946, USA.
| | | | | | | | | | | |
Collapse
|
45
|
The Quality of Reporting of Diagnostic Accuracy Studies in Glaucoma Using Scanning Laser Polarimetry. J Glaucoma 2007; 16:670-5. [DOI: 10.1097/ijg.0b013e3180457c6d] [Citation(s) in RCA: 13] [Impact Index Per Article: 0.8] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/25/2022]
|
46
|
Gunvant P, Zheng Y, Essock EA, Parikh RS, Prabakaran S, Babu JG, Shekar GC, Thomas R. Application of shape-based analysis methods to OCT retinal nerve fiber layer data in glaucoma. J Glaucoma 2007; 16:543-8. [PMID: 17873716 DOI: 10.1097/ijg.0b013e318050ab65] [Citation(s) in RCA: 7] [Impact Index Per Article: 0.4] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/26/2022]
Abstract
PURPOSE (1) To evaluate the performance of shape-based analysis [wavelet-Fourier analysis (WFA) and fast Fourier analysis (FFA)] applied to retinal nerve fiber layer (RNFL) thickness values obtained from the optical coherence tomograph (OCT) to discriminate healthy and glaucomatous eyes. (2) To compare the performance of the shape-based metrics to that of the standard OCT output measures (Inferior Average and Average Thickness). METHODS RNFL values were obtained from 152 eyes of 152 individuals (83 healthy and 69 "mild"-stage perimetric glaucoma). WFA and FFA were performed on the RNFL values and linear discriminant functions for both were obtained using Fisher linear discriminant analysis. Performance was evaluated by calculating sensitivity, specificity, and area under the receiver operating characteristic (ROC) curve (ROC area). RESULTS The ROC area of the shape-based methods [0.94 (WFA) and 0.88 (FFA)] was greater than that of OCT metrics [0.81 (Inferior Average) and 0.74 (Average Thickness)]. Specifically, WFAs performance was significantly better than both the FFA (P=0.009) and the Inferior Average (P=0.001). Inferior average performed significantly better than Average Thickness (P=0.006). CONCLUSIONS The ability to differentiate glaucomatous from healthy eyes using stratus OCT measurements is improved by using these analysis methods that emphasize the shape of the RNFL thickness pattern.
Collapse
|
47
|
Lin SC, Singh K, Jampel HD, Hodapp EA, Smith SD, Francis BA, Dueker DK, Fechtner RD, Samples JS, Schuman JS, Minckler DS. Optic nerve head and retinal nerve fiber layer analysis: a report by the American Academy of Ophthalmology. Ophthalmology 2007; 114:1937-49. [PMID: 17908595 PMCID: PMC3780976 DOI: 10.1016/j.ophtha.2007.07.005] [Citation(s) in RCA: 84] [Impact Index Per Article: 4.9] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/24/2007] [Revised: 05/24/2007] [Accepted: 07/05/2007] [Indexed: 12/14/2022] Open
Abstract
OBJECTIVE To evaluate the current published literature on the use of optic nerve head (ONH) and retinal nerve fiber layer (RNFL) measurement devices in diagnosing open-angle glaucoma and detecting progression. METHODS A search of peer-reviewed literature was conducted on February 15, 2006 in PubMed and the Cochrane Library for the period January 2003 to February 2006. The search was limited to studies of adults in English-language journals and yielded 442 citations. The panel reviewed the abstracts of these articles and selected 159 articles of possible clinical relevance for review. Of these 159 full-text articles, 82 were determined to be relevant for the first author and methodologist to review and rate according to the quality of evidence. RESULTS There were no studies classified as having the highest level of evidence (level I). The ONH and RNFL imaging instruments reviewed in this assessment were determined to be highly effective in distinguishing eyes with glaucomatous visual field (VF) loss from normal eyes without VF loss, based on level II evidence. In addition, some studies demonstrated that parameters from ONH or RNFL imaging predicted the development of VF defects among glaucoma suspects. Studies on detecting glaucoma progression showed that although there was often agreement on progression between the structural and functional (VF) tests, a significant proportion of glaucoma patients progressed by either the structural or the functional test alone. CONCLUSIONS The ONH and RNFL imaging devices provide quantitative information for the clinician. Based on studies that have compared the various available technologies directly, there is no single imaging device that outperforms the others in distinguishing patients with glaucoma from controls. Ongoing advances in imaging and related software, as well as the impracticalities associated with obtaining and assessing optic nerve stereophotographs, have made imaging increasingly important in many practice settings. The information obtained from imaging devices is useful in clinical practice when analyzed in conjunction with other relevant parameters that define glaucoma diagnosis and progression.
Collapse
|
48
|
Van Calster B, Timmerman D, Lu C, Suykens JAK, Valentin L, Van Holsbeke C, Amant F, Vergote I, Van Huffel S. Preoperative diagnosis of ovarian tumors using Bayesian kernel-based methods. ULTRASOUND IN OBSTETRICS & GYNECOLOGY : THE OFFICIAL JOURNAL OF THE INTERNATIONAL SOCIETY OF ULTRASOUND IN OBSTETRICS AND GYNECOLOGY 2007; 29:496-504. [PMID: 17444557 DOI: 10.1002/uog.3996] [Citation(s) in RCA: 21] [Impact Index Per Article: 1.2] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 05/15/2023]
Abstract
OBJECTIVES To develop flexible classifiers that predict malignancy in adnexal masses using a large database from nine centers. METHODS The database consisted of 1066 patients with at least one persistent adnexal mass for which a large amount of clinical and ultrasound data were recorded. The outcome of interest was the histological classification of the adnexal mass as benign or malignant. The outcome was predicted using Bayesian least squares support vector machines in comparison with relevance vector machines. The models were developed on a training set (n=754) and tested on a test set (n=312). RESULTS Twenty-five percent of the patients (n=266) had a malignant tumor. Variable selection resulted in a set of 12 variables for the models: age, maximal diameter of the ovary, maximal diameter of the solid component, personal history of ovarian cancer, hormonal therapy, very strong intratumoral blood flow (i.e. color score 4), ascites, presumed ovarian origin of tumor, multilocular-solid tumor, blood flow within papillary projections, irregular internal cyst wall and acoustic shadows. Test set area under the receiver-operating characteristics curve (AUC) for all models exceeded 0.940, with a sensitivity above 90% and a specificity above 80% for all models. The least squares support vector machine model with linear kernel performed very well, with an AUC of 0.946, 91% sensitivity and 84% specificity. The models performed well in the test sets of all the centers. CONCLUSIONS Bayesian kernel-based methods can accurately separate malignant from benign masses. The robustness of the models will be investigated in future studies.
Collapse
Affiliation(s)
- B Van Calster
- Department of Electrical Engineering (ESAT-SCD), Katholieke Universiteit Leuven, and Department of Obstetrics and Gynecology, University Hospitals K. U. Leuven, Belgium.
| | | | | | | | | | | | | | | | | |
Collapse
|
49
|
Abstract
PURPOSE To investigate the diagnostic accuracy of retinal nerve fiber layer thickness (RNFLT) summary report parameters from Stratus optical coherence tomography (OCT) in glaucoma patients. PATIENTS AND METHODS We obtained Stratus OCT measurements of peripapillary circle scans (average values) of the regular image resolution "FAST RNFLT" protocol, and of 1 circle scan of the high resolution "RNFLT" protocol in one eye of each of 62 glaucoma patients with mild or moderate visual field (VF) loss and 90 healthy subjects. Sensitivity, specificity, and diagnostic accuracy [(true positive+true negative)/all] were evaluated for all summary report parameters including the newer (eg, "Imax," the maximum thickness point in the inferior quadrant) at the normative limits of the Stratus OCT. RESULTS The diagnostic accuracy of full circle RNFLT using the 5% normal limit was 89% with the FAST RNFLT and 87% with the RNFLT protocol; this was at least as good as any other parameter. The diagnostic performance of the 2 protocols did not differ significantly for most parameters. In eyes with mild VF loss (n=39) diagnostic sensitivities reached 72% and 77% at specificities >or=95% using the FAST RNFLT and RNFLT protocol, respectively. CONCLUSIONS The diagnostic accuracy of the full circle RNFLT was as good as any Stratus OCT parameter on the basis of the peripapillary RNFL thickness measurements, including localized measurements. The sensitivity was moderately high in patients with mild glaucomatous VF loss. There seems to be room for further development of OCT interpretation tools for early diagnosis of glaucoma.
Collapse
Affiliation(s)
- Jesper Leth Hougaard
- Department of Clinical Sciences, Ophthalmology, Malmö University Hospital, Lund University, Sweden.
| | | | | |
Collapse
|
50
|
Bizios D, Heijl A, Bengtsson B. Trained artificial neural network for glaucoma diagnosis using visual field data: a comparison with conventional algorithms. J Glaucoma 2007; 16:20-8. [PMID: 17224745 DOI: 10.1097/ijg.0b013e31802b34e4] [Citation(s) in RCA: 46] [Impact Index Per Article: 2.7] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/31/2022]
Abstract
PURPOSE To evaluate and confirm the performance of an artificial neural network (ANN) trained to recognize glaucomatous visual field defects, and compare its diagnostic accuracy with that of other algorithms proposed for the detection of visual field loss. METHODS SITA Standard 30-2 visual fields, from 100 glaucoma patients and 116 healthy participants, formed the data set. Our ANN was a previously described fully trained network using scored pattern deviation probability maps as input data. Its diagnostic accuracy was compared to that of the Glaucoma Hemifield Test, the Pattern Standard Deviation index at the P<5% and <1%, and also to a technique based on the recognizing clusters of significantly depressed test points. RESULTS The included tests had early to moderate visual field loss (median MD=-6.16 dB). ANN achieved a sensitivity of 93% at a specificity level of 94% with an area under the receiver operating characteristic curve of 0.984. Glaucoma Hemifield Test attained a sensitivity of 92% at 91% specificity. Pattern Standard Deviation, with a cut off level at P<5% had a sensitivity of 89% with a specificity of 93%, whereas at P<1% the sensitivity and specificity was 72% and 97%, respectively. The cluster algorithm yielded a sensitivity of 95% and a specificity of 82%. CONCLUSIONS The high diagnostic performance of our ANN based on refined input visual field data was confirmed in this independent sample. Its diagnostic accuracy was slightly to considerably better than that of the compared algorithms. The results indicate the large potential for ANN as an important clinical glaucoma diagnostic tool.
Collapse
Affiliation(s)
- Dimitrios Bizios
- Department of Clinical Sciences, Ophthalmology, Malmö University Hospital, Lund University, SE-205 02 Malmö, Sweden.
| | | | | |
Collapse
|